METHODOLOGICAL APPROACHES IN EXPLORING TEXTBOOK STRUCTURES By Xuran Wang A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Measurement and Quantitative Methods – Doctor of Philosophy 2024 ABSTRACT This study addresses a significant gap in educational research by employing statistical methods to measure the sequencing of different components and content topics in textbooks. It is well-documented that the structure and sequencing features of textbooks play a crucial role in enhancing students’ learning. However, existing literature has predominantly relied on visual approaches to analyze and compare the sequencing of textbook materials—a method that has been in use for over two decades—with few studies employing statistical methods. This reliance on visual methods highlights a noticeable lack of statistical analyses in this area. Without quantitative indicators, the relationship between textbook sequencing features and students’ academic performance cannot be effectively studied. To advance the field, this research included 31 Algebra textbooks used in 9th grade across the U.S. and coded their content, supporting and motivational materials. The content coding was based on the Mathematics Curriculum Document Analysis content framework published in 2022. The coding for supporting and motivational materials in the textbooks was primarily based on motivation theories, reasoning demands of today’s society on students, and indications of the use of the Common Core State Standards in the textbooks. Two approaches were developed: visual mapping and statistical measurement, to analyze and compare the sequencing of three major components (Mathematics Topics, Motivational Materials, Mathematics Reasoning) and content topics. Chapter 4 provides a detailed explanation of the methods used in this study. The visual mappings offer clear representations of how content topics and motivational materials are distributed throughout the textbooks, revealing varied patterns across different textbooks, as discussed in Chapter 5. However, these observations rely on visual inspection rather than quantitative analysis. Chapters 6 and 7 present the statistical measurements using Markov chain techniques and model- based approach, showcasing the sequencing features of the three major components and content topics separately. By doing so, the differences in textbook structures were measured through quantitative indicators, describing pattern similarities and differences in students’ learning opportunities. This study moves beyond graphical analysis, offering a deeper understanding of textbook organization. More importantly, it provides quantitative indicators for future data analysis, enabling researchers to explore the relationship between sequencing features and academic outcomes. Chapter 8 provides an example of how to use these indicators in practice. This research has the potential to inform educators, curriculum developers, and policymakers about the effective ways to organize textbook content to enhance learning opportunities and improve educational outcomes for students. A full discussion is presented in Chapter 9. Copyright by XURAN WANG 2024 This dissertation is dedicated to Mom and Dad. Thank you for always loving and supporting me. v ACKNOWLEDGEMENTS First and foremost, I express my deepest gratitude to myself for undertaking this transformative journey and overcoming numerous challenges along the way. This dissertation stands as a reflection of my perseverance and dedication. I want to extend my sincere thanks to my advisor, Dr. William H. Schmidt, whose guidance and support have been invaluable throughout this journey. Our meetings in Beijing were significant, and I am deeply grateful for the opportunity to continue my studies at Michigan State University under his mentorship. His guidance has not only advanced my academic journey but has also profoundly influenced my approach to research, teaching and communication. Dr. Schmidt’s dedication to his students and his thoughtful communication style have been essential in my growth and career development. I am also deeply thankful to Dr. Richard T. Houang and Dr. Leland S. Cogan, who have been invaluable mentors to me. Their care, conversations, and support have greatly enriched my experience at MSU. Talks with Dr. Houang, whether walking around campus or over coffee, went beyond research to encompass broader educational insights. My heartfelt gratitude also extends to my other committee members, Dr. Kimberly Kelly and Dr. Chih-Li Sung, for their valuable contributions and encouragement. I am greatly appreciative of Dr. Tao Xin from Beijing Normal University, my previous supervisor, who consistently supported my decisions and introduced me to Dr. Schmidt. And I especially cherish the meaningful conversations with Dr. Cogan and his wife, Joella Cogan, who provided invaluable support during my most difficult moments. Gratitude extends to Yingyue Liu, my former roommate, whose discussions helped me see my data from a mapping perspective, offering new ways to conceptualize maps. vi Additionally, Yingyue’s recommendation to meet with Dr. Ashton M. Shortridge from the Department of Geography, Environment, and Spatial Sciences led to a crucial one-hour meeting that profoundly influenced how I approach my data in the future. Their insights played a fundamental role in refining the direction of my research. I am deeply grateful to Dr. Gee Lee from the Department of Statistics & Probability. Despite not knowing each other previously, his kindly response to my initial email led to regular meetings where he generously shared his expertise in sequential data analysis. His guidance was irreplaceable in completing this dissertation. Infinite gratitude to my mom, Hong Zhang, my dad, Zijie Wang, thank you for always giving me the freedom and encouragement to explore myself and the world. Your countless support have been my confidence, enabling me to have diverse experiences. Your belief in me to explore beyond boundaries has been an enduring source of strength. To my husband, Dr. Yiming Deng, whom I met during my doctoral journey, our path together has been marked by both hardship, growth and happiness. I am deeply appreciative of our shared experiences, our love, and I eagerly anticipate the future we will build together. Finally, to my counselor, as well as to those who unknowingly provided me emotional support during this process, Alan Heys, Yujia Liu, Wenyan Wang, Eun Kim, Jiachen Liu, Kun Xia, Hong Zhu, Jiawei Li, Siwen Guo, Jiahui Zhang, Hana Kang, Youngjun Lee, William Sullivan, Zixi Chen, Bixi Zhang, Zhao Peng, Shimeng Dai, Shuo Zhan, and many more. vii TABLE OF CONTENTS CHAPTER 1. INTRODUCTION ................................................................................................... 1 CHAPTER 2. LITERATURE REVIEW ........................................................................................ 5 CHAPTER 3. MEASURES IN THE TEXTBOOKS ................................................................... 17 CHAPTER 4. METHODOLOGICAL APPROACHES FOR ANALYZING AND COMPARING TEXTBOOK SEQUENCES: BOTH VISUAL AND STATISTICAL ........................................ 28 CHAPTER 5. RESULTS OF THE VISUAL ANALYSIS ........................................................... 40 CHAPTER 6. RESULTS OF THE STATISTICAL ANALYSIS: SEQUENCING OF THREE MAJOR COMPONENTS ............................................................................................................. 54 CHAPTER 7. RESULTS OF THE STATISTICAL ANALYSIS: SEQUENCING OF CONTENT TOPICS ..................................................................................................................... 78 CHAPTER 8. BRIDGING THEORY AND PRACTICE ............................................................ 88 CHAPTER 9. DISCUSSION AND LIMITATIONS ................................................................... 99 BIBLIOGRAPHY ....................................................................................................................... 110 APPENDIX A. MATHEMATICS CURRICULUM DOCUMENT ANALYSIS (MCDA) CONTENT FRAMEWORK ....................................................................................................... 114 APPENDIX B. SEQUENCING MAPPING DISPLAY OF THE THREE MAJOR COMPONENTS ......................................................................................................................... 116 APPENDIX C. SEQUENCING MAPPING DISPLAY OF CONTENT TOPICS .................... 118 APPENDIX D. SEQUENCING MAPPING DISPLAY OF MOTIVATIONAL MATERIALS .............................................................................................................................. 125 viii CHAPTER 1. INTRODUCTION 1.1 Background Textbooks are intentionally designed to support student learning in a school setting, distinguishing themselves from merely readable books or random assortments of material. Fundamentally, they are created with a clear educational purpose in mind. They not only present the content that students are expected to learn but also provide various exercises for student practice and employ language that supports learning from multiple perspectives. Generally, the language used in textbooks aims to enhance students’ motivation, introduce effective learning strategies and skills, and incorporate academic practices to facilitate their adoption. Materials extending support beyond the instructional content itself are typically included through margin notes, separate textboxes, and specific labels, enriching the learning experience. The organization and presentation of content and exercises in textbooks, along with materials provided to motivate and support learning, often dictate the instructional structure and approach likely to be adopted in the classroom (Valverde, Bianchi, Wolfe, Schmidt & Houang, 2002, p. 125). The sequence in which content topics are arranged in textbooks influences the likelihood of effective learning. The types of exercises provided serve as the primary learning activities for students both inside and outside of class. Materials designed to motivate and support learning are not only intertwined with content but are also specifically integrated with content-focused teaching pedagogies. These embedded materials can significantly impact students’ abilities to engage with and apply the content. Consequently, understanding and analyzing the sequence of various elements—subject content, exercises, and motivational and support materials—presented in textbooks is essential to grasp the breadth of learning opportunities available (Valverde et al., 2002). This is particularly 1 evident when students of the same grade level use different textbooks for identical subjects. For instance, consider 9th graders across different schools studying Algebra I and Algebra II. Even though they are learning the same subject matter, the use of textbooks can significantly impact their opportunities to learn (OTL) in the classroom, thereby influencing their academic achievement (Cogan & Schmidt, 2015; OECD, 2014; Schmidt, Burroughs, Zoido & Houang, 2015). Understanding the nuances between these textbooks is critical and can offer invaluable insights into the varied educational opportunities and learning trajectories of students. Despite the diverse components in textbooks and the potential insights their distribution sequences offer regarding learning opportunities, there is a notable scarcity of research exploring these aspects. Specifically, limited studies have delved into how these components are distributed throughout textbooks and how their sequences differ across textbooks used at the same grade level. Therefore, this study is committed to filling this critical gap. It aims to deepen our understanding of these components and their organization by conducting a thorough and statistical analysis of various components within textbooks. To the best of my knowledge, this is the first study that statistically addresses this issue in the field of textbook analysis and mathematics education. 1.2 The Current Study The current study is part of a collaborative research project with the University of Texas at Austin (UT-Austin), with our team at Michigan State University (MSU) primarily responsible for the coding and analysis of textbooks, as well as correlating the characteristics of these textbooks with students’ learning outcomes. With support from UT-Austin, we collected 55 textbooks utilized across 40 schools in 20 U.S. states (Figure 1.1). I independently completed all the supporting and motivational materials, as well as over 90% of the content coding activities, 2 within one and a half years, and conducted training sessions for two additional coders to assist in the process. The 55 textbooks include 18 for Algebra I, 13 for Algebra II, 18 for Geometry, and 6 covering other subjects such as Pre-Algebra, Math, Math I, Math II, Advanced Algebra, and Intermediate Algebra, all designated for use by 9th graders. In this study, the 31 Algebra textbooks (18 Algebra I and 13 Algebra II) are included and analyzed. The coverage of 9th-grade math content in the United States tends to vary based on local school district policies, state educational standards, and the specific math course in which a student is enrolled. Typically, the 9th-grade math curriculum centers around Algebra I. Nonetheless, some students may engage in Geometry or Algebra II courses, depending on their previous courses and proficiency. Figure 1.1 Textbooks Sample Distribution Across the United States In the next chapter, I will thoroughly examine why and how the sequencing of content topics in textbooks influences students’ learning outcomes. Also will identify the types of 3 effective exercises and materials included to enhance and motivate learning. Given the focus on math education, the supporting and motivational materials under discussion will specifically address the challenges and strategies pertinent to math learning. Chapter 3 will introduce the various types of materials used in this study, detailing how they are measured within the textbooks. This chapter will include a selection of illustrative examples to provide a clear understanding of the materials’ characteristics and roles in the learning process. Chapter 4 will then detail the methodological approaches employed in the study, encompassing both visual and statistical techniques to analyze the textbooks comprehensively. Chapters 5 through 7 will present the results of the textbook analysis conducted using the approaches developed in Chapter 4. These chapters will offer a detailed examination of the data, highlighting key findings and trends observed in the textbooks. Chapter 8 will provide a comprehensive example of how to apply these results in real-world contexts, illustrating the practical implications and potential benefits for educators and policymakers beyond purely academic research. Finally, Chapter 9 will discuss the implications of the study’s findings, address the limitations encountered, and propose potential directions for future research, thereby offering a holistic conclusion to the research endeavor. 4 CHAPTER 2. LITERATURE REVIEW This literature review will thoroughly examine how the sequencing of content topics in textbooks influences students’ learning outcomes and identify the types of effective exercises and materials included for enhancing and motivating learning. Given the focus on math education, the supporting and motivational materials under discussion will specifically address math learning. 2.1 Sequencing of Content Topics in Textbooks In discussing the significance of logically sequencing content topics within textbooks, it is essential to explore the underlying reason—why such organization matters for student learning in the classroom. This inquiry is deeply intertwined with the concept of coherence in curricula and textbooks, a principle of critical importance in mathematics education, or more specifically, in mathematics learning opportunity. A coherent and structured presentation of content topics substantially benefits mathematics education by enhancing students’ understanding and engagement. Bruner advocates for teaching depth and continuity rather than breadth, aiming to transform students’ initial weak grasp of a subject into a refined and powerful understanding (Bruner, 1995). This approach underscores the importance of diving deep into topics to foster a profound comprehension. The concept of coherence in curricula and textbooks, as defined by Schmidt and his colleagues, involves a logical sequence of topics and activities that reflect the discipline’s sequential or hierarchical nature (Schmidt, Wang & McKnight, 2005, p. 528). It ensures that teaching not only covers the relevant topics within an academic discipline but also embodies the core ideas that organize and generate knowledge within that discipline. A coherent set of content standards progresses from simple concepts to the discipline’s deeper structures, facilitating 5 students’ grasp of complex mathematical concepts and their application in real-world situations (Schmidt et al., 2005). Moreover, any instructional resource is recognized at the confluence of decisions regarding its coverage (the specific content topics to be presented), the sequence in which these topics are covered, the allocated instructional time, and the rigor of the content (Schmidt et al., 2005; Wan & Lee, 2022). This multifaceted approach underscores the importance of thoughtful curriculum design in enhancing educational effectiveness and student learning outcomes. Within this context, the logical sequencing of topics stands out not just as a critical aspect but arguably as the core component of coherence in curriculum and textbook design, as emphasized by Schmidt and Houang (2012). Its importance is emphasized across a wide array of research within curriculum studies, with significant contributions from scholars such as Cogan, Schmidt & Wiley (2001); Ferrini-Mundy, Burrill & Schmidt (2007); Schmidt (2004, 2008, 2012); Schmidt et al. (2005, 2012); Schmidt & Prawat (2006); and more recent studies by Wan et al. (2022) and Lee & Wan (2022). Drawing on data from the top-performing countries in the Third International Mathematics and Science Study (TIMSS), Schmidt et al. (2005) initially utilized a geometric shape, resembling an upper-triangle, as a nonquantitative indicator of coherence in curriculum design. The geometric representation effectively encapsulates both the extent of coverage and the pattern or sequence in which the topics were presented, serving as an indicator of coherence up until recent studies (Wan et al., 2022; Lee et al., 2022). Nonetheless, this nonquantitative measure was originally devised to compare the coverage of topics and the sequence in which these topics were presented from grade 1 to grade 8 across different countries, rather than to indicate the sequencing of topics within an individual textbook. Another geometric representation approach for examining each individual textbook’s sequencing features was first 6 introduced by Valverde et al. (2002). However, both methods lack quantitative indexes, making them unsuitable for use as quantitative variables in data analysis. This limitation constrains our capacity to rigorously examine the relationship between the sequencing features of textbooks and students’ academic achievements, thereby highlighting a considerable gap in empirical research on the impact of textbook sequencing on educational outcomes. In this study, I aim to statistically measure and analyze the logical sequencing of content topics within textbooks, an aspect that, to my knowledge, has received limited attention in the existing literature. In general terms, I believe that the logical sequencing of content topics should (a) introduce a series of grade-specific or subject-specific topics in mathematics, like Algebra I, and (b) organize these topics according to state standards or a rational sequence. For instance, within Algebra I textbooks, it is expected that students encounter simple linear equations prior to delving into quadratic equations. Hence, one of my objectives is to fill a gap in the existing groundwork laid by previous research on coherence in curricula and textbooks and the studies of measuring students’ learning opportunities in school. By investigating potential statistical measures for the logical sequencing of content topics within textbooks, I intend to refine and expand upon the current methods used to assess coherence, thereby filling a notable gap in the literature. The reliance on methodologies developed over two decades ago underscores the urgent need for innovative approaches in this field. 2.2 Enhancing Math Learning: Incorporating Mathematical Reasoning into Exercises The world in which we now live have become increasingly complicated, not only due to the advancements in artificial intelligence (AI), computers, robotics, and various technological formats but also in the manner we gather the necessary knowledge for living, studying, working, and addressing the challenges confronting the global population. For instance, we may encounter 7 varying “facts” about the same news event across different online platforms. These differing “facts” go beyond what we typically encounter in person-to-person exchanges, which generally have a more limited spread. Without concrete evidence, relying solely on internet-based information can lead us to form biased opinions about individuals or incorrect conclusions about certain matters. While this may not always lead to serious consequences, it typically reduces our capacity to thoroughly analyze facts pertinent to our own lives, studies, and works. Even more concerning is that it decreases our ability to engage in meaningful communication with the people in our lives; instead, we find ourselves constantly talking with strangers. Furthermore, in recent years, pandemics have swept across nations, economies have wrestled with recessions, and we’ve witnessed a surge in natural catastrophes—such as floods, hurricanes, tornadoes, and earthquakes—exacerbated by climate change. Navigating these scenarios has become exceedingly difficult. It’s imperative that we analyze these situations, choose suitable strategies, formulate logical conclusions, devise and articulate solutions, and then identify how these solutions can be implemented. This comprehensive process is referred to as mathematical reasoning (OECD, 2023). Mathematical reasoning serves as a critical process through which students learn to make sense of mathematical concepts, solve problems, and justify their solutions logically and coherently. It is more than just the ability to calculate numbers; it’s about understanding the why and how behind the numbers, which is essential for developing a deep and durable understanding of mathematics. The importance of mathematical reasoning in learning math cannot be overstated; it equips learners with the skills to approach complex problems creatively and critically, make connections between different mathematical concepts, and apply their knowledge in various contexts. With mathematical reasoning, students can arrive at results that 8 they can fully trust to be true in a wide variety of real-life contexts. It is also important that these conclusions are impartial, without any need for validation by an external authority (OECD, 2023). Within textbooks, cultivating students’ mathematical reasoning abilities can be effectively achieved through in-class and homework activities/exercises. Problems designed to engage higher-order thinking skills, known as Higher Order Problems (HOPs), require students to apply advanced mathematical reasoning involving sophisticated cognitive processes. These problems, when set against the backdrop of realistic, authentic, and real-world scenarios, push students to go beyond simply identifying the required mathematics for a solution, embodying higher-order real-world applications. Such problems are designed to reflect the intricate and often unpredictable nature of the real world, urging students to conceptualize, organize, and sift through information to craft a mathematical representation of the problem. In this context, arriving at the correct solution may not be as challenging or as crucial as developing mathematical literacy. Furthermore, problems that are grounded in mathematical concepts yet distinct from real- world scenarios also demand that students engage in a rigorous process of conceptualization, organization, extraction of relevant information, and the formulation of a logical approach to find solutions. These are known as higher-order math problems. An exemplar of this is the geometric proof, which requires a meticulous construction of a deductive argument, leveraging pertinent information and devising a coherent strategy for problem-solving. Identifying relevant theorems and axioms and understanding their logical integration within a proof is essential to these solutions. 9 Another strategy to enhance students’ mathematical reasoning abilities involves encouraging the use of mathematical reasoning to solve problems, facilitated by specific terms and phrases within the textbooks. The forthcoming chapter, Measures in the Textbooks will delve into the types of words that can foster this analytical thinking. Enhancing Math Learning: Materials that Motivate and Support Beyond 2.3 Instructional Content and Exercises Why do textbooks include materials beyond the core subject content and exercises? And how are these additional resources purposefully organized within them? Textbooks often contain a variety of materials beyond the core subject content and exercises, recognizing that the subject matter alone may not fully address the diverse needs of students to enhance their learning outcomes. Educators have identified several factors that can significantly impact students’ learning, including their motivation to learn math and their strategies for learning math. To address these factors, textbooks are thoughtfully organized to incorporate materials designed to motivate and support learning. The ensuing discussion will focus on describing various theories of motivation commonly embedded in textbooks to motivate learning. It will also highlight key factors aimed at enhancing mathematical learning specifically. Furthermore, the representation of each type of material within textbooks will be described in detail in the next chapter, Measures in the Textbooks. 2.3.1 The Theory of Intelligence and Ability Beliefs with Math Learning Students frequently encounter discouraging statements like “you are not smart enough to learn math” or “not everyone can do well in math.” Despite some progress in changing these perceptions, public attitudes often still reflect these negative sentiments. Such beliefs significantly influence policymakers’ considerations regarding math education within the public 10 school system, affecting their decision-making (Schmidt, 2012). In this context, it is imperative to challenge and address stereotypes like “some students cannot do mathematics” through rigorous scientific methods. Achieving proficiency in math, like mastering any other subject, comes with its own set of challenges. However, it is achievable if students are provided with equitable learning opportunities, effective teaching strategies, and the necessary support and motivation. The availability of learning opportunities and the effectiveness of teaching or learning strategies are largely external factors, beyond students’ control. These are typically provided through educational tools like math textbooks, while effective teaching strategies are imparted by teachers using their deep understanding of math and pedagogical knowledge to facilitate meaningful learning experiences in the classroom (Schmidt, 2012). Conversely, motivation is an internal aspect, driven by students’ personal attitudes towards math and their underlying reasons for engaging with it. This involves their beliefs about the value of math and their confidence in their abilities to succeed, which are crucial in shaping their learning approach and resilience in facing challenges, including complex problems. The belief among students that they are part of the group that “cannot do mathematics” can lead to avoidance of advanced math courses or a reluctance to engage with difficult problems. However, this mindset can be altered through intervention by educators, particularly math teachers, who play a crucial role in developing a positive learning attitude towards math. Motivation theories in learning seek to describe, explain, and predict the mechanisms of student motivation, providing educators with strategies to foster a conducive learning environment and motivation for math. 11 Among these theories is the theory of intelligence and ability beliefs (Dweck & Leggett, 1988; Blackwell, Trzesniewski & Dweck, 2007; Elliott & Dweck, 1988; Muenks & Miele, 2017). As popularized by Carol Dweck (2006), this theory posits that intelligence and abilities are not just innate traits but can be developed through dedication and hard work. This powerful perspective challenges the traditional paradigm that often labels students as “naturally gifted” or “inherently challenged” based on their initial performance. Within the context of education, this mindset holds particular significance as it shifts the focus from innate ability to the potential for growth, resilience, and continuous learning. Mathematics, a subject that many students often find daunting and impenetrable, stands as a critical area where the principles of the growth mindset can be particularly transformative. By understanding that abilities in mathematics can be nurtured and enhanced, students can approach challenges with a renewed sense of purpose and a decreased fear of failure. At the heart of Dweck’s groundbreaking work on mindset (2006) lies the distinction between two core beliefs: a fixed mindset, which views intelligence and abilities as immutable traits, and a growth mindset, which perceives them as qualities that can be developed. This distinction is crucial in an educational context. In a fixed mindset, students believe that their abilities are static, leading to either complacency or despair, depending on their perceived skill level. On the other hand, a growth mindset instills the belief that effort plays a pivotal role in improvement. This perspective is particularly salient in subjects like mathematics, a field that often demands persistence in the face of complex problems. Given the theoretical promise of a growth mindset in boosting academic achievement, numerous studies have sought to translate theory into practice by implementing growth mindset interventions in classroom settings. One compelling study by Yeager et al. (2019) utilized a brief 12 online growth mindset intervention, revealing that even short-term exposure to growth mindset principles could elevate students’ math achievements. Another study by Paunesku, Walton, Romero, Smith, Yeager & Dweck (2015) conducted an intervention where students were exposed to growth mindset instructions over an academic year. The results were promising; not only did students show an increased belief in the malleability of intelligence, but they also displayed enhanced perseverance in solving math problems and achieved improved grades. These studies, among others, underline the practical benefits of integrating growth mindset instructions in the educational process, particularly in the area of mathematics. 2.3.2 Exposure to Mathematical Concepts with Math Learning Another important motivation theory, self-efficacy, refers to the belief students hold about their capability to successfully perform and complete academic tasks (Bandura, 1997). Research has shown that self-efficacy is considered domain-specific, indicating that students may have different levels of self-efficacy according to different subject matters (Dweck, 2002; Muenks, Wigfield & Eccles, 2018). Specifically, in the math domain, mathematics self-efficacy is defined as “a situational or problem-specific assessment of an individual’s confidence in her or his ability to successfully perform or accomplish a particular [mathematics] task or problem” (Hackett & Betz, 1989, p. 262). Students who feel competent about performing well in math may persist and expend their effort to learn the knowledge, and this affects their achievement (Schunk & DiBenedetto, 2021). Research has widely demonstrated that mathematics self-efficacy affects students’ mathematical performance at different educational levels across different educational systems (Zimmerman, Bandura, & Martinez-Pons, 1992; Caprara, Fida, Vecchione, Del Bove, Vecchio, Barbaranelli, & Bandura, 2008; Lee, 2009; Stajkovic, Bandura, Locke, Lee, & Sergent, 2018). Generally, students with higher mathematics self-efficacy are linked with higher 13 mathematical performance (Zimmerman et al., 1992; Caprara et al., 2008), and countries with higher performance in math are those where students also report feeling more confident in solving a range of mathematics tasks (OECD, 2013a, p. 93). The relationship between self- efficacy and attainment in mathematics is demonstrated by Bandura’s theory (Bandura, 2002), which suggests that strongly perceived efficacy fosters high group performance attainments. Mathematics self-efficacy postulates that students develop their self-efficacy beliefs by interpreting information from four sources: (a) mastery experiences, which refer to the interpretation of one’s own previous performance; (b) vicarious experiences, which refer to the observation of others’ performances; (c) social persuasions, which refer to feedback received from others (e.g., encouragement from teachers); and (d) physiological states, which refer to emotional and somatic states, including anxiety and stress (Bandura, 1997). Regarding the relative strength of these four sources in relating to self-efficacy, research has documented that students’ mastery experiences show the strongest association with self-efficacy (Usher & Pajares, 2008). Further, the literature has suggested that exposure to specific mathematical concepts could enhance students’ feelings of competence in handling the same or similar concepts (Wang, Houang, Schmidt & Kelly, 2024), since this would provide opportunities for increasing mastery experiences (Zimmerman et al., 1992; Pajares, 1997; Borgonovi & Pokropek, 2019). 2.3.3 Standards for Mathematical Practices with Math Learning Following the release of the Common Core State Standards for Mathematical Practices (CCSS-MP) in 2010, numerous publishers endeavored to incorporate the eight prescribed mathematical practices into their textbook designs. Furthermore, they prominently featured the CCSS endorsement on the front covers of these textbooks and strategically integrated details of 14 the eight mathematical practices within the pages. This integration varied, with some practices appearing between chapters, others introduced in the opening pages before the first chapter, and some positioned at the conclusion of the last chapter, ensuring that the practices were shown throughout the educational material. The CCSS-MP represent a significant initiative in the United States aimed at providing a clear and consistent framework to prepare students for college and the workforce. The development of these standards was a collaborative effort with the goal of ensuring that students graduating from high school possess the necessary mathematical skills and knowledge to succeed in entry-level careers, introductory college courses, and workforce training programs, also ensuring greater equality in content coverage among students (Schmidt & Burroughs, 2013). These practices encourage students to develop a deeper understanding of mathematical concepts, enhance their problem-solving skills, and foster critical thinking and reasoning. By focusing on processes and proficiencies as much as on content, the CCSS-MP help cultivate a more robust mathematical mindset in students, preparing them for complex problem-solving and analytical tasks in their future academic and professional endeavors. Moreover, these practices underscore the importance of mathematical communication, allowing students to articulate their thought processes and reasoning effectively. 2.3.4 Study Habits with Math Learning Beyond the foundational theories of motivation and the guiding principles of the CCSS- MP in supporting students’ mathematics learning, the role of good study habits is critical in enhancing student learning outcomes in school. Effective study habits—such as organized study schedules, active engagement with materials, consistent practice, and self-assessment techniques—act as vital complements to the conceptual and procedural understandings promoted 15 by the CCSS-MP. These habits help students to internalize and apply mathematical concepts more efficiently, foster a positive attitude towards learning, and build confidence in their problem-solving capabilities. Moreover, good study habits encourage students to take initiative in their learning process, promoting a sense of responsibility and independence that is beneficial not only in mathematics but across all academic disciplines. Fundamentally, although motivation theories and the CCSS-MP lay a robust foundation for mathematical education, it is ultimately the development of disciplined study habits that frequently drives students to realize their full academic potential. Good study habits bridge the gap between theoretical understanding and practical application, enabling students to consolidate their learning and achieve sustained academic success. Through consistent practice and strategic learning approaches, students are better equipped to tackle complex mathematical problems, thereby enhancing their overall performance and fostering a lifelong appreciation for the subject. In summary, this chapter illustrates that a comprehensive approach to math education— one that includes well-sequenced content, the incorporation of mathematical reasoning, positive beliefs about intelligence and ability, self-efficacy, adherence to mathematical standards, and disciplined study habits—is essential for fostering students’ mathematical proficiency. The next chapter will introduce measures of various components within the textbook. In Chapter 4, the focus will be on describing approaches to both visual and statistical analysis in sequencing these components, which constitute the primary focus of this research. 16 CHAPTER 3. MEASURES IN THE TEXTBOOKS In this chapter, I introduce and discuss all the measures used in the current study, including the coding procedures and what is included in each measure. The coding procedures for content topics and supporting/motivating materials were treated separately. The content coding procedure includes only the pages of all lessons listed in the Table of Contents of each textbook. Pages such as lesson/chapter reviews, summaries, or tests, while related to content topics, are considered auxiliary materials and thus not included. In contrast, the coding for supporting and motivating materials encompasses pages from the very beginning to the end of the textbooks. 3.1 Coding Procedures and Measures for Content Topics The Mathematics Curriculum Document Analysis (MCDA) content framework (Schmidt, Houang, Sullivan & Cogan, 2022, p. 115) is detailed in Table A1 in Appendix A. Initially, the coder recorded the content code for each lesson based on the MCDA content framework, noting the corresponding chapter, lesson, title, beginning page, and ending page. Each lesson could contain a single content code or multiple content codes, with the coder trained to document all applicable codes. For instance, in Lesson 1.1 of the textbook BIL_ALGI_01, titled Solving Simple Equations, the coding details included the chapter, lesson, title, beginning page, ending page, and content code (assigned as “4.2.3” according to the content coding framework). Subsequently, within each lesson, the coder tallied the number of computational problems, word problems, and higher-order problems in the after-lesson problem session designed for student practice. Concerning the higher-order problems, also referred to as the Mathematics Reasoning component in this study, additional coding was included. Definitions for these higher-order problems are outlined below: 17 1. Higher-order application problems: These problems are situated in realistic, authentic, real-world contexts that extend beyond simple identification of necessary mathematics. Students are challenged to grapple with the complexities of real-world scenarios, requiring conceptualization, organization, and extraction of relevant information. They must formulate a mathematical equation that accurately represents the problem before arriving at a solution. In this context, the actual computation becomes a secondary aspect in developing mathematical literacy. 2. Higher-order mathematics problems: These problems are confined to the realm of mathematics without a real-world context. However, they demand students conceptualize, organize, extract pertinent information, and craft a logical approach to problem-solving. A classic example is a geometric proof, where the objective is to construct a deductive proof using relevant information and develop a coherent problem-solving approach. This process involves recognizing and applying relevant theorems and axioms coherently in the proof. The additional coding for higher-order problems included three key aspects: the problem type (Application or Mathematics), the type of quantitative reasoning necessary to solve the exercise (Mathematics, Algorithmic, Geometric, Statistic), and the associated content code for each higher-order problem. It’s important to note that an exercise may require multiple types of quantitative reasoning. In such cases, both the primary (most utilized/required) and secondary (less utilized/required) types were coded. 3.2 Coding Procedures and Measures for Supporting and Motivating Materials The supporting and motivating materials from the textbooks encompass several aspects: Growth Mindsets, Study Habits, Exposure, Reasoning, and labels relevant to these aspects. The 18 coding procedure involved collecting relevant information page by page, starting from the textbook’s introduction and continuing through to its end. Whenever phrases promoting the principles listed below were encountered, I recorded the page number and the original phrase or sentences for reference. 3.2.1 Growth Mindsets The core concept of a growth mindset posits that intelligence and ability can be cultivated through effort, strategic approaches, and support. This idea is rooted in the belief that individuals can develop their skills and understanding through perseverance and dedication, rather than seeing these attributes as fixed traits. In examining math textbooks, particular attention is given to identifying occurrences that promote and encourage the utilization of effort, strategies, and support. This involves assessing how textbooks facilitate and emphasize these essential components. For instance, passages that explicitly encourage students to persist through challenges are noted, such as phrases highlighting the value of making errors and overcoming mental roadblocks. These sections reinforce the idea that struggle and effort are integral to learning and growth. Furthermore, the analysis extends to how textbooks offer support, both in terms of guidance provided within the text and encouragement for students to experiment and share. By identifying and coding these elements, the aim is to understand how textbooks contribute to fostering a growth mindset, ultimately supporting students in developing a resilient and proactive approach to their mathematical education. Specific examples from the textbooks coded in the current study are illustrated in Figure 3.1. 19 Figure 3.1 Examples of Growth Mindsets Occurrences from Textbooks 20 3.2.2 Study Habits Additionally, I examined the presence of strategic approaches recommended within the textbooks, such as problem-solving techniques, study tips, and methods for organizing information, using appropriate tools. These strategies can empower students to tackle complex problems more effectively. Figure 3.2 provides examples from the textbooks coded in the current study. Figure 3.2 Examples of Study Habits Occurrences from Textbooks 21 Figure 3.2 (cont’d) 3.2.3 Exposure Students learn through exposure to the necessary content knowledge. Therefore, I collected instances related to sentences suggesting alternative methods for problem-solving or inquiries about previously encountered or solved problems. These sentences encourage students to recall previously learned material that could aid in solving current problems, reinforcing their understanding and ability to apply prior knowledge in new contexts. By prompting students to reflect on their prior learning, these statements facilitate connections between different concepts and problem-solving strategies. This approach not only helps solidify their grasp of the material but also fosters deeper comprehension and flexibility in applying various strategies to tackle mathematical challenges. Consequently, such instances are crucial for promoting a comprehensive and interconnected understanding of mathematics, encouraging students to draw on their full range of knowledge and experiences. Figure 3.3 provides examples from the textbooks coded in the current study. 22 Figure 3.3 Examples of Exposure Occurrences from Textbooks 3.2.4 Reasoning Reasoning forms a fundamental component of mathematical learning and problem- solving. This study focuses on identifying instances within the textbooks that promote reasoning skills. These include passages encouraging students to explain their thought processes, justify their answers, and engage in logical thinking. By prompting students to articulate their reasoning, textbooks contribute to developing critical thinking skills and a deeper understanding of mathematical concepts. Such sections often present problems requiring students to make connections, draw inferences, and apply principles in novel ways, fostering an environment where reasoning is not just encouraged but essential. Emphasizing the importance of reasoning in 23 mathematics education ensures that students do not merely memorize procedures but genuinely comprehend and engage with the material on a meaningful level. Figure 3.4 provides examples from the textbooks coded in the current study. Figure 3.4 Examples of Reasoning Occurrences from Textbooks 24 Figure 3.4 (cont’d) In addition to these principles, their associated labels, such as Reason and Apply, were also coded. However, these labels accompanied sets of problems for student practice. Examples are shown in Figure 3.5 below. Figure 3.5 Examples of Associated Labels from Textbooks 25 Figure 3.5 (cont’d) In conclusion, Chapter 3 has outlined the comprehensive measures used in the current study, detailing distinct coding procedures for content topics and supporting/motivating materials. I explored aspects of supporting and motivating materials, focusing on Growth Mindsets, Study Habits, Exposure, and Reasoning. By systematically coding these elements, this study aims to provide a detailed understanding of how math textbooks facilitate learning and promote essential educational goals. In Chapter 5, I will visually present the sequencing mapping display of supporting and motivating materials for each analyzed Algebra textbook. Furthermore, in the subsequent analysis aimed at statistically exploring the sequencing features of different components, I aggregated all supporting and motivating materials together and coded them as Motivational Materials. Pages categorized differently, excluding Motivational Materials and Mathematics Reasoning, were grouped under a separate category named Mathematics Topics regardless of their specific content topics. In the next chapter, I will delve into the approaches for analyzing the sequencing of different mapping displays. Chapters 5 and 6 will provide a thorough examination of how each 26 component is structured and presented across various textbooks, visually and statistically. Additionally, I will introduce the approach I applied to analyze the sequencing of different content topics throughout a textbook in Chapter 7. 27 CHAPTER 4. METHODOLOGICAL APPROACHES FOR ANALYZING AND COMPARING TEXTBOOK SEQUENCES: BOTH VISUAL AND STATISTICAL Textbooks present content and exercises’ activities in various structures, and teachers have flexibility in organizing their instruction. However, there is a belief that the sequence of classroom instruction often mirrors the sequence proposed in textbooks (Valverde et al., 2002). In essence, the structure of a textbook profoundly influences classroom experiences, and data show that textbooks vary in how they present components and integrate them. Textbooks are designed around distinct pedagogical models that outline the optimal sequence of educational opportunities to achieve curricular goals. Understanding textbook sequences unveils their instructional models and offers insights into how they shape unique educational configurations in classrooms. Thus, this research employs visual and statistical methods to examine the structure and sequence of textbooks comprehensively. This chapter addresses the identification of these textbook sequences. 4.1 Visually Mapping Educational Sequences from Textbooks To illustrate different sequences, I developed various graphical analyses: some represent the general sequencing of textbook components, while others specifically depict the sequence of content topics. These graphical displays effectively portray the sequence of components and content topics across an entire textbook. Here, I describe three types of these displays for a specific textbook, an Algebra I textbook published by Big Idea Learning LLC in 2016 for 9th- grade students. The original display (Figure 4.1) provides detailed elements, while two other displays (Figure 4.2 and Figure 4.3) align with current research goals and subsequent statistical analysis. Figure 4.1 depicts the detailed sequence of all elements in the Algebra I textbook. This two-dimensional legend illustrates the structure of the entire book from four perspectives. It 28 uniformly represents pages from beginning to end, left to right, divided into three sections top to bottom. Page “0” covers introductory pages before the first chapter, usually including pages like letters to students or pages outlining mathematical practices. Each appearance of a page is denoted by a darker blue cell. The upper section categorizes block types: Narrative, Worked Example, Problem Sets, and Other Sets, indicating the format on each page. The middle section corresponds to content topics from the MCDA Content Framework (see Appendix A). Rows represent topics, columns represent pages, and multiple content codes per page are common, shown by multiple darker blue cells in columns. The third section, a single row, uses darker cells to indicate the frequency of higher-order problems (Mathematics Reasoning) on each page. The bottom section uses darker cells to indicate Motivational Materials on each page. Unlike the first two sections, this one uses multiple colors; purple indicates frequent appearances of a specific type of motivational material. Multiple types on the same page result in multiple darker cells in the same column. Columns represent successive pages. Each section provides vertical and horizontal insights into page-level details and overall textbook progression. 29 Figure 4.1 First Sequencing Mapping Display from Sample Textbook In this example, the Algebra I textbook comprises 513 pages, categorizing into 261 (50.88%) narrative or worked example pages, 149 (29.04%) problem exercise pages, and 103 (20.08%) other pages, which include introductory, transfer pages between two lessons/chapters, quizzes, tests, multiple lesson reviews, or summaries. Certain content areas are prominently featured: the majority of the textbook (27.29%) addresses topics such as quadratics, polynomial 30 equations, and inequalities, with additional focus on simple linear equations (13.65%), slope and intercept (12.48%), and inequalities and linear functions (12.08%). There are a total of 7 higher- order problems appearing across six pages. Motivational materials are found 58 times throughout the textbook. Of these, 19 instances emphasize exploring alternative problem-solving approaches or prompting recall of previously encountered problems. Other instances include fostering abstract and quantitative reasoning, critiquing others’ reasoning, and encouraging regularity in problem-solving approaches. Figure 4.2 presents an alternative visualization of the same data, illustrating how Mathematics Reasoning (higher-order problems) and Motivational Materials interact with various Mathematics Topics in the textbook. For consistency in this study, Mathematics Reasoning is used to refer to higher-order problems below. Figure 4.2 Second Sequencing Mapping Display from Sample Textbook The above type of display has only one row. Similar to Figure 4.1, the columns from left to right represent the actual page numbers in the textbook, with page “0” indicating the introductory pages preceding the first chapter. There are three colors to indicate three different categories. Mathematics Topics refer to any page with a mathematics topic, regardless of the format—whether narrative, worked example, problem set, or non-coded quizzes, review, or summary pages. All types of motivational materials are combined into one category, Motivational Materials, with the focus being on whether the page includes Motivational 31 Materials rather than the frequency of their appearance. Finally, Mathematics Reasoning refers to the higher-order problems that appear on that page of the textbook. The display clearly shows how the Motivational Materials and Mathematics Reasoning interact with the Mathematics Topics in this textbook. For example, in this textbook, motivational sentences appear randomly throughout, with many concentrated in the first part and some appearing on several successive pages. If we assume that the difficulty level of the mathematics topics ranges from easy to hard from left to right, which follows logical progression, it can be seen that the authors try to motivate students’ learning from the beginning or at least before introducing harder topics. Besides, all the Mathematics Reasoning problems are found in the first three-quarters of the textbook, but not in the last quarter. Notably, pages with more motivational sentences tend to have more Mathematics Reasoning problems for students to practice in this textbook. In addition to Figure 4.1 and Figure 4.2, I developed another graphical method to display the distribution of content topics throughout the textbook. This new display focuses solely on the details of the Mathematics Topics, without considering the Motivational Materials and Mathematical Reasoning. The reason for this focus is the importance of textbook coherence to students’ learning that was mentioned in the literature review. Moreover, the sequence of topics has not been statistically studied yet. This research aims to fill that gap and find ways to capture sequencing features based on their distribution. Figure 4.3 shows the distribution of content topics across the same sample textbook as Figure 4.1 and Figure 4.2. 32 Figure 4.3 Third Sequencing Mapping Display from Sample Textbook Figure 4.3 is designed to illustrate the distribution of content topics within the Algebra I textbook. Given the textbook’s focus on Algebra I learning, some content topics are grouped into broader categories. For instance, subcategories like Number and System are combined, as are Geometry, Foundations, Probability, and Statistics. Additionally, the original category Change— encompassing topics such as infinite processes, calculus and analysis, linear and non-linear functions, and exponential growth models—is categorized under Advanced Algebra. However, details within the Algebra category remain individually listed to depict the distribution of various Algebra topics throughout the textbook. Similar to Figure 4.1, this two-dimensional display uniformly represents pages from the beginning to the last page, proceeding from left to right. Page “0” denotes the introductory pages preceding the first chapter of the textbook. Each appearance of a page is indicated by a darker blue cell. Pages with lighter blue cells denote that they are not coded for content, including quizzes, tests, multiple lesson reviews, or summaries. Notably, as discussed in Chapter 2, the coding procedure for Motivational Materials encompasses every page from the beginning to the end, whereas the content coding procedure excludes non-content pages. 33 From Figure 4.3, it becomes apparent that the sequence of topics in the first half of the sample Algebra I textbook follows a discernible pattern. Initially, the content progresses from simple linear equations to linear equations and inequalities, and subsequently to linear functions. It then revisits simple linear equations, followed by linear equations and inequalities, and once more focuses on linear functions. Interspersed among these topics are occasional lessons on subjects typically outside Algebra, such as Geometry. Conversely, the latter half of the textbook maintains a singular focus on Other Equations and Inequalities, encompassing topics like quadratics, polynomials, factorization, and expansion. Appendices B and C provide further details on the second and third sequencing mapping displays from the 31 textbooks analyzed in this study. 4.2 Statistically Measuring Educational Sequences from Textbooks The above provides three mapping approaches to display the same Algebra I textbook. In this section, statistical approaches that measure the sequencing features of the textbook will be introduced in detail. Markov chain analysis, widely used in economics, finance, biology, and computer science, models stochastic processes where states transition based on a defined set of probabilities. A Markov chain is characterized by memorylessness, where future state transitions depend solely on the current state and not on previous events, known as the Markov property. Mathematically, a Markov chain is represented by a transition matrix 𝑃, where each element 𝑃!" denotes the probability of transitioning from state 𝑖 to state 𝑗. If the chain is time-homogeneous, the transition probabilities do not change over time. Markov chains naturally model the probability of moving from one state to another, which is crucial for understanding sequential data. This is especially important in datasets where 34 certain categories follow a natural order and the transition probabilities between states need to be captured accurately. By assuming the Markov property, complex dependencies in sequential data can be simplified into manageable probabilistic transitions. This makes it easier to analyze and interpret the underlying patterns and trends in the data and leading to more informed decisions and strategies. Thus, Markov chain techniques are considered and applied in the current study. Based on one objective of this study—to understand the sequencing characteristics of textbooks and the distribution of mathematical reasoning and motivational materials within them—Figure 4.2 is utilized to explore these sequencing features. I will begin with an introduction to some key concepts and then use these concepts to assess and analyze the sequence features in Figure 4.2. 4.2.1 Transition Probability in Markov Chains Transition probability is a fundamental concept in the study of Markov chains, which are stochastic processes used to model systems that undergo transitions from one state to another. In a Markov chain, the transition probability defines the probability of moving from one state to another in a single step. Mathematically, it is expressed as: 𝑃!" = 𝑃(𝑋#$% = 𝑗 | 𝑋# = 𝑖) where 𝑋# represents the state of the system at step 𝑛. These probabilities are organized into a transition matrix 𝑃, which for a Markov chain with 𝑁 states, is an 𝑁´ 𝑁 matrix where each element 𝑃!" indicates the probability of transitioning from state 𝑖 to state 𝑗. Transition probabilities have several main properties. Firstly, each transition probability 𝑃!" is non-negative, ensuring that the probabilities are valid and meaningful. Secondly, the sum of the transition probabilities from any given state to all possible states, 35 including itself, equals one. This property is crucial as it reflects the fact that the process must transition to one of the possible states at each step. Lastly, in a time-homogeneous Markov chain, the transition probabilities are independent of time, meaning that the probability of transitioning from state 𝑖 to state 𝑗 is the same at any time step 𝑛. For calculating the transition matrix, let the states of the Markov chain be 𝑆%, 𝑆&, … , 𝑆#. Let 𝑓!" be the frequency of transition observed from state 𝑆! to state 𝑆". Then, construct the frequency matrix 𝐹: 𝐹 = 1 𝑓%& ⋯ 𝑓%# 𝑓%% 𝑓&% 𝑓&& ⋯ 𝑓&# ⋮ ⋮ 𝑓#% 𝑓#% ⋯ 𝑓## ⋱ ⋮ 5 For each state 𝑆!, calculate the sum of the frequencies of transitions from state 𝑆!: # 6 𝑓!" "’% Next, convert the frequency matrix to a transition probability matrix 𝑃 by dividing each entry in the frequency matrix by the corresponding row sum: 𝑃!" = 𝑓!" ∑ # (’% 𝑓!( This gives the transition probability matrix 𝑃: 𝑃 = 1 𝑝%% 𝑝%& ⋯ 𝑝%# 𝑝&% 𝑝&& ⋯ 𝑝&# ⋮ ⋮ 𝑝#% 𝑝#% ⋯ 𝑝## ⋱ ⋮ 5 The transition probability matrix in a Markov chain captures the sequence feature because it encodes the probability of moving from one state to another at each step. This encapsulation allows us to understand the behavior of the chain over one or multiple steps and thus, the sequence of states that the process may go through. 36 Absorption Probability in Markov Chains In the context of Markov chains, the absorption probability refers to the probability that starting from a given state 𝑖, the process will eventually reach an absorbing state 𝑗, which is a state from which once entered, there is no possibility of leaving. Consider a finite Markov chain {𝑋#, 𝑛 = 0, 1, 2, …} with state space 𝑆 = {0, 1, 2, … , 𝑟}. Suppose that all states are either absorbing or transient. Let 𝑙 ∈ 𝑆 be an absorbing state. Define 𝑎! = 𝑃(𝑎𝑏𝑠𝑜𝑟𝑝𝑡𝑖𝑜𝑛 𝑖𝑛 𝑙 | 𝑋) = 𝑖), 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑖 Î 𝑆 By the above definition, we have 𝑎* = 1, and 𝑎" = 0 if 𝑗 is any other absorbing state. To find the unknown values of 𝑎!′𝑠, we can use the following equations 𝑎! = 6 𝑎(𝑝!(, 𝑓𝑜𝑟 𝑖 Î 𝑆 ( Absorption probabilities provide information about the pathways the chain is likely to take before reaching an absorbing state. They help in identifying which states are more likely to be visited before absorption occurs. This is crucial for understanding how a system evolves over time. Expected Hitting Time in Markov Chains The expected hitting time in a Markov chain, refers to the average number of steps it takes for the process to reach a particular state starting from another state. Consider a finite Markov chain {𝑋#, 𝑛 = 0, 1, 2, …} with state space 𝑆 = {0, 1, 2, … , 𝑟}. Let 𝐴 Ì 𝑆 be a set of states. Let 𝑇 be the first time the chain visits a state in 𝐴. For all 𝑖 Î 𝑆, define 𝑡! = 𝐸[𝑇 | 𝑋) = 𝑖] 37 By the above definition, we have 𝑡" = 0, for all 𝑗 Î 𝐴. To find the unknown values of 𝑡!′𝑠, we can use the following equations 𝑡! = 1 + 6 𝑡(𝑝!(, 𝑓𝑜𝑟 𝑖 Î 𝑆 − 𝐴 ( Hitting times reveal the expected duration of sequences leading to a particular state. This helps in understanding the temporal aspects of state transitions. Expected Return Time in Markov Chains Another interesting random variable is the first return time. In particular, assuming the chain is in state 𝑙, we consider the expected time (number of steps) needed until the chain returns to state 𝑙. Assuming 𝑋) = 𝑙, let’s define 𝑟* as the expected number of steps needed until the chain returns to state 𝑙. Define Then, 𝑅* = min{𝑛 ³ 1: 𝑋# = 𝑙} 𝑟* = E[𝑅* | 𝑋) = 𝑙] Note that by definition, 𝑅* ³ 1, so we conclude 𝑟* ³ 1. In fact, 𝑟* = 1 if and only absorbing state. As before, we can apply the law of total probability to obtain 𝑟*. Again, let’s define 𝑡( as the expected time until the chain hits state 𝑙 for the first time, given that 𝑋) = 𝑘. We have already seen how to find 𝑡(’s (mean hitting times). Using the law of total probability, we can write 𝑟* = 1 + 6 𝑝*(𝑡( ( In summary, this chapter introduces three mapping displays of textbooks and various techniques used in Markov chain analysis. It demonstrates the effectiveness of Markov chain 38 techniques in simplifying and interpreting complex dependencies in sequential data, leading to more informed decisions and strategies in textbook analysis. The details of applying these methods to the textbook dataset will be presented and interpreted in Chapter 6. However, regarding the sequencing feature of content topics, which will be based on Figure 4.3, the Markov chain technique is not appropriate. For the flow of the whole dissertation and for the convenience of the readers, the statistical approach for analyzing this figure will be introduced and applied in Chapter 7. 39 CHAPTER 5. RESULTS OF THE VISUAL ANALYSIS As introduced in Chapter 4, I have developed two approaches—visual and statistical—to describe, assess, and analyze the sequence features of textbooks. In this chapter, I present the visual analysis results from three perspectives: patterns in the sequencing of Mathematics Topics, patterns in the sequencing of Motivational Materials, patterns in the sequencing of Mathematics Reasoning, and an integrated view across Mathematics Topics, Motivational Materials, and Mathematics Reasoning. 5.1 Patterns in the Sequencing of Mathematics Topics Content serves as a fundamental basis for classifying the structure of textbooks. Using the method illustrated in Figure 4.3, the rows represent content topics, while the columns represent pages from the beginning to the last page, arranged from left to right. Page “0” denotes the introductory pages preceding the first chapter of the textbook. Darker blue cells indicate the presence of specific content topic on a given page, while pages with only lighter blue cells represent they are not coded for their content, such as quizzes, tests, multiple lesson reviews, or summaries. The rows are organized from top to bottom by increasing difficulty level of the topics, with Number and System being less difficult than Probability and Statistics. Since all 31 books are Algebra I and II textbooks, understanding their different patterns involves dividing them into distinct topic segments and analyzing these segments across textbooks: algebra topics (ranging from Foundations to Advanced Algebra), prerequisite topics (Number and System), Geometry, and advanced topics (Probability and Statistics). Logically, most pages should focus on algebra topics, with advanced topics typically found towards the end of the textbooks, while prerequisite and Geometry topics should comprise a smaller portion of the pages. Within the algebra topics, we expect to see pages dedicated to higher difficulty level 40 topics following those of easier topics, not the reverse. However, my analysis indicates that textbooks often do not adhere to this expected structure. All results regarding patterns in the sequencing of content topics across all textbooks are shown in Appendix C. 5.1.1 Results of the Sequencing of Advanced Topics (Probability and Statistics) Reporting the analysis results regarding the sequencing of advanced topics is straightforward, as these topics ideally appear towards the end part of the textbook. This sequencing ensures that students have a solid foundation in algebraic concepts before delving into more complex material, which often builds upon these principles and logically progresses their learning. However, this order is not absolute in practice but rather a standard used in the current study to discuss the distributions. Students using textbooks from the same publisher consistently might have been exposed to some advanced topics before entering 9th grade, meaning the location of these advanced topics does not necessarily need to be at the end. Among the 31 Algebra I and II textbooks analyzed, only 2 do not include any Probability and Statistics topics. Additionally, one textbook covers a small proportion of pages in the middle of the textbook dedicated to Probability and Statistics, concurrently covering algebra topics (Expression) on those pages. In half of the textbooks (16 in total), pages exclusively covering Probability and Statistics topics are initially located in the middle of the entire textbook, as depicted in Figure 5.1, the first instance of Probability and Statistics topics appearing is in the middle of the textbook CPM_ALGI_01. Subsequently, these topics reappear for the second and third times in the last third of the book. However, textbooks where pages cover both advanced topics (Probability and Statistics) and any algebra topics, and are located in the middle or not in the last third part, are not the focus. In such cases, it is assumed that advanced topics would not be the primary classroom focus. 41 Only the remaining 12 out of the 31 textbooks adhere to the rationale that pages exclusively devoted to Probability and Statistics topics first appear towards the end of the textbook or at least in the last third, as depicted in Figure 5.2. These 12 textbooks, encompassing both Algebra I and Algebra II from various publishers, do not exhibit a consistent pattern in this regard. Figure 5.1 Example Textbook: Pages Exclusively Covering Probability and Statistics Firstly Located in the Middle Figure 5.2 Example Textbook: Pages Exclusively Covering Probability and Statistics Firstly Located at the End 5.1.2 Results of the Sequencing of Geometry Topic Another straightforward finding regarding the structure of Algebra textbooks involves examining topics from other subjects, such as Geometry. For pages covering Geometry topics, two situations are considered: whether the page also includes any algebra topics, and the exclusive coverage of Geometry topics along with the proportion of these pages. The former 42 scenario is not the focus, as Geometry topics would not likely be the main classroom focus in such cases. Instead, attention is directed towards the proportion of pages exclusively dedicated to Geometry topics. Among the 31 Algebra I and II textbooks analyzed, 28 include pages covering Geometry topics. These pages occupy only a very small proportion of the textbooks, and their locations vary. In some textbooks, they are situated in the middle (see Figure 5.3), while in others, they appear towards the end (see Figure 5.4). In both figures, it is evident that the pages covering Geometry exclusively do not include any other topics. In the current sample, 25 out of the 28 textbooks feature pages dedicated solely to Geometry topics without any Algebra topics. This suggests that during instructional periods, students are expected to engage with Geometry topics while enrolled in Algebra I or II classes. Figure 5.3 Example Textbook: Pages Exclusively Covering Geometry Firstly Located in the Middle 43 Figure 5.4 Example Textbook: Pages Exclusively Covering Geometry Firstly Located at the End 5.1.3 Results of the Sequencing of Prerequisite Topics (Number and System) Unlike advanced and Geometry topics, some prerequisite topics, such as Number and System, typically appear in Algebra textbooks. These topics are foundational and often interact with algebra topics throughout the textbook. However, it is not logical to have numerous pages solely dedicated to prerequisite topics in an Algebra textbook, especially when these pages are located towards the end. Such placement may appear irrelevant to students’ learning progression and could disrupt the continuity of the material. Therefore, the focus here is on pages exclusively dedicated to these prerequisite topics, specifically Number and System topics, and their distribution within the textbooks. Among the 31 Algebra I and II textbooks analyzed, 26 include pages dedicated solely to Number and System topics. These pages are dispersed throughout the textbooks, with many appearing both at the beginning and again in the middle, as depicted in Figure 5.5. In Figure 5.5, pages featuring Number and System topics first appear at the very beginning of the textbook. Subsequently, these topics reappear in the middle of the textbook after covering other topics. A few textbooks also place these pages towards the end (see Figure 5.6). In these instances, while Number and System topics initially appear at the beginning and reappear in the middle, they are exclusively located towards the end without any other topics on those pages. 44 Figure 5.5 Example Textbook: Pages Exclusively Covering Number and System Located at the Beginning and Reappearing in the Middle Figure 5.6 Example Textbook: Pages Exclusively Covering Number and System Located at the End 5.1.4 Results of the Sequencing of Algebra Topics In addition to examining the sequencing of advanced topics, Geometry, and prerequisite topics, this section focuses primarily on the sequencing of algebra topics. Among the 18 Algebra I textbooks analyzed, a prevalent pattern emerges where, in the first half of the textbooks, the sequencing alternates between two main categories identified by the MCDA project: Beginning Algebra and Algebra. Beginning Algebra topics include algebraic sequences and patterns, expressions, simple linear equations, slope and intercept. On the other hand, Algebra topics encompass linear equations and inequalities, trigonometric equations and identities, other equations and inequalities, linear functions, exponential functions, and other non-linear functions. Figure 5.7 illustrates this pattern. In the first half of the textbook, the topics alternate between Beginning Algebra topics (such as expressions) and Algebra topics (including linear 45 equations and inequalities, non-linear equations and inequalities, and linear functions). Interspersed among these topics are pages covering Number and System topics. In contrast, the second half of the textbooks predominantly focus on topics within the Algebra category. These sections contain extensive coverage of Algebra topics, with occasional pages dedicated to Number and System topics appearing in between. Figure 5.7 Example of Sequencing of Algebra Topics in Algebra I Textbooks Meanwhile, among the 13 Algebra II textbooks analyzed, the content trends predominantly within the Algebra category. Unlike Algebra I textbooks, which often feature alternating sections of Beginning Algebra and Algebra topics, Algebra II textbooks primarily focus on more advanced Algebra topics without extensive coverage of Beginning Algebra. Figure 5.8 exemplifies this trend, where most of the pages cover Algebra topics such as advanced algebraic sequences, complex equations, trigonometric identities, and functions. Interspersed among these topics are occasional pages dedicated to Number and System, Geometry, and Probability and Statistics topics. This sequencing highlights a shift in emphasis towards more complex algebraic concepts typical of Algebra II curricula, reflecting the progression from foundational algebra skills covered in Algebra I. 46 Figure 5.8 Example of Sequencing of Algebra Topics in Algebra II Textbooks 5.2 Patterns in the Sequencing of Motivational Materials Another critical aspect of textbook structure is the sequencing of Motivational Materials, detailed in Appendix D. This section outlines two predominant types of sequencing observed across the textbooks analyzed. The first type of textbook exhibits a consistent recurrence of specific types of motivational materials throughout its entirety. For instance, Figure 5.9 illustrates how a growth mindset label appears consistently across various exercises throughout the textbook. Similarly, Figure 5.10 showcases a single type of label, labeled Higher Order Math, recurrently used in exercises. In some textbooks, like the example in Figure 5.11, multiple types of motivational labels, such as reasoning and growth mindset-related labels, are used consistently throughout the exercises sections. In contrast, the second type of textbook features a more sporadic distribution of motivational materials throughout its pages. These textbooks incorporate various motivational sentences that encourage students to persist in problem-solving, adjust strategies, or maintain resilience, aligning with math practice standards like those outlined in the Common Core State Standards for Mathematics. Despite this randomness, many textbooks include motivational sentences in the introductory pages, often within letters or on the front page before the table of contents, as illustrated in Figure 5.12. 47 Moreover, some textbooks integrate both sequencing types. They may have certain motivational labels that recur consistently throughout the textbook, while other motivational sentences appear randomly across its pages. It’s noteworthy that very few textbooks omit motivational materials altogether. Figure 5.9 Example Textbook with Single Kind of Motivational Materials Figure 5.10 Another Example of a Textbook with Single Kind of Motivational Materials Figure 5.11 Example Textbook with Multiple Kinds of Motivational Materials Figure 5.12 Example Textbook with Random Appearance of Motivational Materials In summary, among the 31 Algebra I and II textbooks analyzed, the majority incorporate motivational materials in their introductory pages (19 textbooks), with 18 textbooks utilizing both randomly distributed motivational materials throughout the text and recurring motivational labels. Five textbooks feature randomly distributed motivational materials, while three exclusively use motivational labels within their exercises sections. Remarkably, only one 48 textbook among the analyzed set does not include any form of motivational materials. These findings underscore the varied approaches taken by textbook authors to integrate motivational support, ranging from structured recurring labels to more sporadic, context-specific encouragements dispersed throughout the text. 5.3 Patterns in the Sequencing of Mathematics Reasoning Using the display method shown in Figure 4.3 to explore the patterns in the sequencing of mathematical reasoning, the columns from left to right represent the actual page numbers in the textbook, with page “0” indicating the introductory pages preceding the first chapter. The visualization results are shown in Figure 5.13 and Figure 5.14. Three colors are used to indicate different categories: 1. Light blue cells represent Mathematics Topics, which include any page with a mathematics topic, regardless of the format—narrative, worked example, problem set, non-coded quizzes, review, or summary pages. 2. Darker blue cells represent Motivational Materials, combining all types of motivational materials into one category, focusing on whether the page includes such content. 3. Green cells represent Mathematics Reasoning, indicating the presence of higher-order problems on that page of the textbook. Findings indicate that not all analyzed textbooks include higher-order problems. Specifically, two textbooks do not contain any higher-order problems. Additionally, there are no obvious patterns in the distribution of higher-order problems; they are randomly distributed throughout the textbooks. 49 Figure 5.13 Sequencing of Three Major Components in 18 Algebra I Textbooks 50 Figure 5.14 Sequencing of Three Major Components in 13 Algebra II Textbooks 51 An Integrated Look Across Mathematics Topics, Motivational Materials, and 5.4 Mathematics Reasoning The preceding discussion has explored how different parts of these textbooks reveal groups of textbooks sharing analogous approaches to content exploration, motivational materials, and the use of mathematical reasoning practices for increasing students’ ability to solve complex problems. This section illustrates how these components interact using the results from Figures 5.13 and 5.14 to demonstrate the sequencing of the three major categories within each textbook. In the sample of 31 textbooks, these components operate independently of each other. Visual inspection does not reveal apparent relationships between patterns of content presentation and strategies for motivating students’ learning. There do not appear to be preferences for presenting particular content in conjunction with specific motivational materials deemed of highest priority for that content. However, in some textbooks, motivational materials recur regularly throughout the entire textbook. As mentioned in the section on Patterns in the Sequencing of Motivational Materials, this regular recurrence is due to labels that frequently appear in the lesson exercise sections. Thus, these materials are not associated with particular content but are a result of the textbook’s format design. An interesting result from this integrated analysis is that some textbooks, such as PEAR_ALGII_01 (published in 2015) and PEAR_ALGII_02 (published in 2011), although published in different years, show quite similar sequencing of the components despite being from the same publisher. Moreover, the location of the Motivational Materials and Mathematics Reasoning in these textbooks is almost identical, at least in the first three-fourths of the content. The same pattern is observed in textbooks like HMD_ALGII_01 (published in 2011) and HRW_ALGII_01 (published in 2007). These textbooks were published in different years and by 52 different publishers. Despite these differences, the sequencing of components in these textbooks is remarkably similar. In summary, the visualized integrated sequencing figures reveal some common patterns among the three major components, along with other types of patterns discussed in this chapter. However, these observations rely on visual inspection rather than quantitative analysis. The next chapter will present results using the statistical approach introduced in Chapter 3, providing a statistical perspective and deriving relevant quantitative indicators to describe pattern similarities and differences. 53 CHAPTER 6. RESULTS OF THE STATISTICAL ANALYSIS: SEQUENCING OF THREE MAJOR COMPONENTS In Chapter 5, the visualizations provided us with intuitive insights into the similarities and differences in the sequencing of components in mathematics textbooks from various perspectives. However, these visualizations have limitations in quantifying the sequence characteristics, akin to descriptive statistics. For instance, while a Box and Whisker plot can describe the distribution of a sample, it requires quantitative metrics such as mean and variance. Moreover, advanced analyses, like examining correlations between sequence characteristics and student performance, necessitate quantitative indicators. Therefore, both visualizations and statistical approaches are essential for a comprehensive understanding of textbook structures in real-world contexts. In this chapter and the next, I will summarize the results obtained from different statistical perspectives. I will apply and analyze two types of displays from Chapter 4. The first display (Figure 4.2) uses three colors to encode the three major components (Mathematics Topics, Motivational Materials, Mathematics Reasoning) across textbooks, providing a general overview of their structure. The second display (Figure 4.3) focuses on the sequencing of content topics within textbooks, which is particularly pertinent for textbook analysis researchers, educators, and mathematics teachers. This chapter will concentrate on the statistical findings derived from the mapping of the three major components, while Chapter 7 will shift focus to the mapping of content topics. I will start with a descriptive analysis of all 31 Algebra textbooks (18 Algebra I and 13 Algebra II), including their page counts (e.g., means, standard deviations), and the percentage distribution of components across textbook pages. These descriptive features are directly linked 54 to the sequencing displays. Subsequently, I will summarize the statistical characteristics of the sequencing. 6.1 Descriptive Analysis Number of pages across 31 Algebra textbooks is shown in Table 6.1. Table 6.1 Number of Pages in 31 Algebra Textbooks Algebra I Textbooks Algebra II Textbooks TextbookID Length of Pages TextbookID Length of Pages BIL_ALGII_01 CL_ALGII_01 CPM_ALGII_01 HMD_ALGII_01 HMD_ALGII_02 HRW_ALGII_01 MDL_ALGII_01 MGH_ALGII_01 MGH_ALGII_02 PEAR_ALGII_01 PEAR_ALGII_02 PEAR_ALGII_03 PEAR_ALGII_04 639 623 777 1034 870 1034 963 908 908 958 958 650 812 BIL_ALGI_01 BIL_ALGI_02 CL_ALGI_01 CPM_ALGI_01 GM_ALGI_01 HMC_ALGI_01 HMD_ALGI_01 HMD_ALGI_02 HRW_ALGI_01 MDL_ALGI_01 MGH_ALGI_01 MGH_ALGI_02 PEAR_ALGI_01 PEAR_ALGI_02 PEAR_ALGI_03 PEAR_ALGI_04 PEAR_ALGI_05 PEAR_ALGI_06 513 623 447 622 491 633 926 748 906 900 810 810 783 771 712 502 771 700 Note: The letters before the first “_” in the TextbookID represent the publisher’s abbreviation; the middle letters denote the subject abbreviation, such as Algebra I (ALGI) and Algebra II (ALGII); the number at the end of each TextbookID indicates the sequence number of the subject textbook published by the publisher. The introductory pages (page 0) are included. 55 According to the data from the analyzed textbooks, Algebra I textbooks vary in length from a minimum of 447 pages to a maximum of 926 pages, with an average length of 704 pages per textbook (SD = 148.57). Conversely, Algebra II textbooks range from 623 pages to 1034 pages, averaging 856 pages per textbook (SD = 145.26). Tables 6.2 and 6.3 present the percentage distribution of components across the pages of the 18 Algebra I textbooks and the 13 Algebra II textbooks, respectively. Table 6.2 Percentage Distribution of Components in 18 Algebra I Textbooks TextbookID Mathematics Topics Motivational Materials Mathematics Reasoning BIL_ALGI_01 BIL_ALGI_02 CL_ALGI_01 CPM_ALGI_01 GM_ALGI_01 HMC_ALGI_01 HMD_ALGI_01 HMD_ALGI_02 HRW_ALGI_01 MDL_ALGI_01 MGH_ALGI_01 MGH_ALGI_02 PEAR_ALGI_01 PEAR_ALGI_02 PEAR_ALGI_03 PEAR_ALGI_04 PEAR_ALGI_05 PEAR_ALGI_06 89.08% 95.02% 99.11% 90.03% 99.80% 96.37% 88.88% 89.30% 89.18% 94.78% 81.98% 88.15% 84.29% 73.41% 87.78% 88.45% 71.60% 87.57% 9.75% 4.65% 0.22% 5.79% 0.00% 2.84% 10.69% 9.63% 10.26% 4.67% 17.28% 11.11% 15.20% 25.81% 12.22% 10.56% 27.76% 12.43% 1.17% 0.32% 0.67% 4.18% 0.20% 0.79% 0.43% 1.07% 0.55% 0.56% 0.74% 0.74% 0.51% 0.78% 0.00% 1.00% 0.65% 0.00% Note: The letters before the first “_” in the TextbookID represent the publisher’s abbreviation; the middle letters denote the subject abbreviation, such as Algebra I (ALGI) and Algebra II (ALGII); the number at 56 the end of each TextbookID indicates the sequence number of the subject textbook published by the publisher. The introductory pages (page 0) are included. Table 6.3 Percentage Distribution of Components in 13 Algebra II Textbooks TextbookID Mathematics Topics Motivational Materials Mathematics Reasoning BIL_ALGII_01 CL_ALGII_01 CPM_ALGII_01 HMD_ALGII_01 HMD_ALGII_02 HRW_ALGII_01 MDL_ALGII_01 MGH_ALGII_01 MGH_ALGII_02 PEAR_ALGII_01 PEAR_ALGII_02 PEAR_ALGII_03 PEAR_ALGII_04 93.74% 99.68% 93.69% 90.04% 90.11% 90.14% 95.85% 89.76% 83.04% 77.97% 78.18% 94.46% 87.68% 5.63% 0.16% 4.76% 9.38% 9.08% 9.48% 3.53% 10.13% 16.85% 21.09% 20.88% 5.23% 10.96% 0.63% 0.16% 1.54% 0.58% 0.80% 0.39% 0.62% 0.11% 0.11% 0.94% 0.94% 0.31% 1.35% Note: The letters before the first “_” in the TextbookID represent the publisher’s abbreviation; the middle letters denote the subject abbreviation, such as Algebra I (ALGI) and Algebra II (ALGII); the number at the end of each TextbookID indicates the sequence number of the subject textbook published by the publisher. The introductory pages (page 0) are included. Based on the analysis of Table 6.2 and Table 6.3, the distribution of three components across pages in Algebra I and Algebra II textbooks reveals consistent trends with slight variations. In Algebra I textbooks, Mathematics Topics dominate the pages, constituting an average of 88.60% (SD = 7.53%), ranging from 71.60% to 99.80%. Motivational Materials occupy an average of 10.60% (SD = 7.57%) of the pages, with variability from 0.00% to 27.76%. Mathematics Reasoning, while less prevalent, averages 0.80% (SD = 0.91%) of the pages, peaking at 4.18% and occasionally absent. 57 Comparatively, Algebra II textbooks exhibit a similar pattern but with slightly higher averages across components. Mathematics Topics average 89.57% (SD = 6.53%) of the pages, ranging from 77.97% to 99.68%, indicating a slightly broader distribution compared to Algebra I. Motivational Materials average 9.78% (SD = 6.44%) of the pages, with a range of 0.16% to 21.09%, showing a consistent but slightly reduced presence compared to Algebra I. Mathematics Reasoning maintains a low but discernible presence, averaging 0.65% (SD = 0.46%) of the pages and varying from 0.11% to 1.54%. In sum, the descriptive findings highlight significant variability in length across both Algebra I and Algebra II textbooks. Moreover, they underscore a consistent emphasis on Mathematics Topics in both types of textbooks, with Motivational Materials serving a supportive role. Although Mathematics Reasoning appears in smaller proportions, it nonetheless contributes to the overall structure of the textbooks. 6.2 Measures From Markov Chain Analysis To capture the sequencing features of the textbooks, ideas from Markov chain analysis were adopted. As introduced in Chapter 3, the first step is to calculate the transition probability matrix for each textbook. Transition Probability Matrices The transition probability matrices of the 18 Algebra I and 13 Algebra II textbooks are shown in Table 6.4 and Table 6.5, respectively. Additionally, for a more intuitive visualization of the transitions between the three components, Figure 6.1 and Figure 6.2 present the corresponding transition probability diagrams. Explanations of the results will follow these figures. 58 Table 6.4 Transition Probability Matrices of 18 Algebra I Textbooks BIL_ALGI_01 BIL_ALGI_02 1 0.92 0.66 1.00 2 0.07 0.34 0.00 CPM_ALGI_01 1 0.91 0.83 0.81 2 0.05 0.17 0.00 HMD_ALGI_01 1 0.88 1.00 0.00 2 0.12 0.00 0.75 MDL_ALGI_01 1 0.95 0.93 1.00 2 0.05 0.07 0.00 PEAR_ALGI_01 1 0.82 0.96 0.75 2 0.17 0.04 0.25 PEAR_ALGI_04 1 0.89 0.89 0.80 2 0.10 0.11 0.20 3 0.01 0.00 0.00 3 0.04 0.00 0.19 3 0.00 0.00 0.25 3 0.01 0.00 0.00 3 0.01 0.00 0.00 3 0.01 0.00 0.00 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 0.95 0.93 1.00 2 0.04 0.07 0.00 3 0.00 0.00 0.00 GM_ALGI_01 3 2 1 1.00 0.00 0.00 NaN NaN NaN 0.00 0.00 1.00 HMD_ALGI_02 1 0.89 1.00 0.25 2 0.10 0.00 0.50 MGH_ALGI_01 1 0.79 0.98 1.00 2 0.21 0.02 0.00 PEAR_ALGI_02 1 0.68 0.89 0.60 2 0.31 0.10 0.40 PEAR_ALGI_05 1 0.66 0.86 0.50 2 0.33 0.14 0.50 3 0.01 0.00 0.25 3 0.01 0.00 0.00 3 0.01 0.01 0.00 3 0.01 0.00 0.00 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 CL_ALGI_01 1 1.00 1.00 0.33 2 0.00 0.00 0.00 HMC_ALGI_01 1 0.96 1.00 1.00 2 0.03 0.00 0.00 HRW_ALGI_01 1 0.88 1.00 0.20 2 0.11 0.00 0.40 MGH_ALGI_02 1 0.87 1.00 1.00 2 0.13 0.00 0.00 3 0.00 0.00 0.67 3 0.01 0.00 0.00 3 0.00 0.00 0.40 3 0.01 0.00 0.00 PEAR_ALGI_03 3 2 1 0.00 0.14 0.86 0.00 0.00 1.00 NaN NaN NaN PEAR_ALGI_06 3 2 1 0.00 0.14 0.86 1.00 0.00 0.00 NaN NaN NaN 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Note: The letters before the first “_” in the TextbookID represent the publisher’s abbreviation; the middle letters denote the subject abbreviation, such as Algebra I (ALGI) and Algebra II (ALGII); the number at the end of each TextbookID indicates the sequence number of the subject textbook published by the publisher. 1 = Mathematics Topics; 2 = Motivational Materials; 3 = Mathematics Reasoning. Since all original values were rounded to two digits, the sum of the rows is not always exactly equal to 1. “NaN” means no such component in the textbook data. 59 Table 6.5 Transition Probability Matrices of 13 Algebra II Textbooks CL_ALGII_01 1 1.00 1.00 1.00 1 2 3 2 0.00 0.00 0.00 HMD_ALGII_02 2 1 0.10 0.89 0.00 1.00 0.29 0.71 1 2 3 MGH_ALGII_01 2 1 0.11 0.89 0.00 1.00 0.00 1.00 1 2 3 PEAR_ALGII_02 1 0.74 0.95 0.78 2 0.25 0.05 0.22 1 2 3 3 0.00 0.00 0.00 3 0.01 0.00 0.00 3 0.00 0.00 0.00 3 0.01 0.00 0.00 CPM_ALGII_01 2 1 0.04 0.94 0.17 0.83 0.00 0.92 1 2 3 HRW_ALGII_01 2 1 0.10 0.89 0.00 1.00 0.50 0.50 1 2 3 MGH_ALGII_02 2 1 0.20 0.80 0.01 0.99 0.00 1.00 1 2 3 PEAR_ALGII_03 1 0.95 0.91 1.00 2 0.05 0.09 0.00 1 2 3 3 0.02 0.00 0.08 3 0.00 0.00 0.00 3 0.00 0.00 0.00 3 0.00 0.00 0.00 BIL_ALGII_01 1 0.94 0.94 1.00 2 0.06 0.06 0.00 HMD_ALGII_01 1 0.89 1.00 0.67 2 0.10 0.00 0.33 MDL_ALGII_01 1 0.96 0.91 1.00 2 0.03 0.09 0.00 3 0.01 0.00 0.00 3 0.01 0.00 0.00 3 0.01 0.00 0.00 PEAR_ALGII_01 1 0.73 0.95 0.78 2 0.25 0.05 0.22 3 0.01 0.00 0.00 PEAR_ALGII_04 1 0.86 0.98 1.00 2 0.12 0.00 0.00 3 0.01 0.02 0.00 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Note: The letters before the first “_” in the TextbookID represent the publisher’s abbreviation; the middle letters denote the subject abbreviation, such as Algebra I (ALGI) and Algebra II (ALGII); the number at the end of each TextbookID indicates the sequence number of the subject textbook published by the publisher. 1 = Mathematics Topics; 2 = Motivational Materials; 3 = Mathematics Reasoning. Since all original values were rounded to two digits, the sum of the rows is not always exactly equal to 1. “NaN” means no such component in the textbook data. 60 Figure 6.1 Transition Probability Diagrams of 18 Algebra I Textbooks 61 Figure 6.2 Transition Probability Diagrams of 13 Algebra II Textbooks 62 Based on the transition probability matrices in Table 6.4 and Table 6.5, the values primarily denote the probability of transitioning from one component to another on the subsequent page within the textbooks. In these tables, the first column signifies the initial components, while the first row denotes the end components. For example, a cell value of 0.66 in the transition probability matrix for BIL_ALGI_01 indicates that in this textbook, there is a 66% probability of transitioning from Motivational Materials to Mathematics Topics on the next page. These values are directional, indicating the sequencing from the current component to the next, rather than vice versa. Higher values suggest a greater probability of transitioning to that component, emphasizing continuity, while lower values indicate less frequent transitions. Upon examining the probabilities of transitions within the same component in the subsequent page, distinct patterns emerge within both Algebra I and Algebra II textbooks. In Algebra I, Mathematics Topics exhibit a strong average transition probability of 87.61% (SD = 9.34%), illustrating substantial continuity within this component across textbooks. Motivational Materials show lower but notable variability, ranging from 0% to 34%, with some textbooks completely excluding this component like GM_ALGI_01. Mathematics Reasoning displays wider variability, ranging from 0% to 67%, with two textbooks omitting this component entirely. Conversely, Algebra II textbooks demonstrate slightly higher average transition probabilities within Mathematics Topics at 88.31% (SD = 8.27%), albeit with greater variability ranging from 73% to 100%. Motivational Materials and Mathematics Reasoning show lower probabilities overall, with Motivational Materials ranging from 0% to 17% and Mathematics Reasoning from 0% to 8%. Additionally, these transition values within the same component highlight the degree of continuity specific to each component. Higher probabilities indicate longer stretches of 63 consecutive pages within that component before transitioning, while lower probabilities suggest more frequent transitions. For example, the textbook BIL_ALGI_02 demonstrates a higher continuity of Mathematics Topics pages compared to PEAR_ALGI_05, with corresponding transition probabilities of 0.95 and 0.66, respectively. This distinction can be observed through their mapping displays (see Figure 5.13), similarly noted in other textbooks such as BIL_ALGI_02 and MGH_ALGI_01. Furthermore, transitions between different components, such as from Motivational Materials to Mathematics Topics, also reveal consistent patterns across both Algebra I and Algebra II textbooks. In Algebra I, the average probability of transitioning from Motivational Materials to Mathematics Topics is 93.71% (SD = 9.11%), with transitions ranging from 66% to 100% among textbooks. Notably, eight textbooks demonstrate a perfect transition probability of 100%. Similarly, in Algebra II, the average probability slightly increases to 95.85% (SD = 5.15%), with transitions ranging from 83% to 100% across different textbooks. This consistency underscores a strong tendency for Motivational Materials to lead into Mathematics Topics across both types of textbooks, highlighting a structured sequencing approach in educational materials. These findings, represented by the colors coding the three components, indicate that high transition probabilities from Motivational Materials to Mathematics Topics signify a high probability that a page of Motivational Materials will be followed by a page on Mathematics Topics (dark blue followed by light blue). For instance, BIL_ALGI_02 exhibits a stronger continuity between Motivational Materials and Mathematics Topics compared to BIL_ALGI_01, with corresponding transition probabilities of 0.93 and 0.66, respectively. This difference is evident in their mapping displays (see Figure 5.13). Despite BIL_ALGI_01 containing more Motivational Materials pages than BIL_ALGI_02, many of these are followed by another 64 Motivational Materials page rather than a Mathematics Topics page, reflected in their transition probabilities from Motivational Materials to Motivational Materials, which are 34% and 7%, respectively. Expected Hitting Time Matrices The expected hitting time refers to the average number of pages required for the learning process to reach a specific component for the first time, starting from a given initial component. Formally, for a textbook with components’ set S, the expected hitting time ℎ(𝑖, 𝑗) is the average number of pages to reach component 𝑗 from component 𝑖. Expected hitting times provide a quantitative measure of component reachability and dynamics in textbooks, offering a structured way to interpret and explain patterns in the textbook dataset. Tables 6.6 and 6.7 present the expected hitting times for the 18 Algebra I and 13 Algebra II textbooks, respectively. 65 Table 6.6 Expected Hitting Time Matrices of Algebra I Textbooks (Excluding Three with Missing Transition Probability Matrices) BIL_ALGI_01 BIL_ALGI_02 CL_ALGI_01 1 0 2 1 2 14 0 15 CPM_ALGI_01 1 0 1 1 2 20 0 21 HMD_ALGI_02 1 0 1 2 2 10 0 5 MGH_ALGI_01 1 0 1 1 2 5 0 6 PEAR_ALGI_02 1 0 1 1 2 3 0 3 3 84 86 0 3 28 30 0 3 123 124 0 3 134 135 0 3 127 128 0 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 0 1 1 2 23 0 24 HMC_ALGI_01 1 0 1 1 2 34 0 35 HRW_ALGI_01 1 0 1 2 2 9 0 5 MGH_ALGI_02 1 0 1 1 2 8 0 9 PEAR_ALGI_04 1 0 1 1 2 10 0 9 3 309 311 0 3 126 127 0 3 299 300 0 3 134 135 0 3 99 100 0 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 0 1 3 2 0 0 1 HMD_ALGI_01 1 0 1 2 2 8 0 1 MDL_ALGI_01 1 0 1 1 2 22 0 23 PEAR_ALGI_01 1 0 1 1 2 6 0 5 PEAR_ALGI_05 1 0 1 2 2 3 0 3 3 442 443 0 3 306 307 0 3 179 180 0 3 194 195 0 3 152 154 0 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Note: The letters before the first “_” in the TextbookID represent the publisher’s abbreviation; the middle letters denote the subject abbreviation, such as Algebra I (ALGI) and Algebra II (ALGII); the number at the end of each TextbookID indicates the sequence number of the subject textbook published by the publisher. 1 = Mathematics Topics; 2 = Motivational Materials; 3 = Mathematics Reasoning. “NaN” means no such component in the textbook data. 66 Table 6.7 Expected Hitting Time Matrices of Algebra II Textbooks BIL_ALGII_01 CL_ALGII_01 CPM_ALGII_01 1 0 1 1 2 0 0 0 HMD_ALGII_02 1 0 1 1 2 10 0 8 MGH_ALGII_01 1 0 1 1 2 9 0 10 PEAR_ALGII_02 1 0 1 1 2 4 0 4 3 620 621 0 3 123 124 0 3 906 907 0 3 105 106 0 1 2 3 1 2 3 1 2 3 1 2 3 1 0 1 1 2 25 0 26 HRW_ALGII_01 1 0 1 2 2 10 0 6 MGH_ALGII_02 1 0 1 1 2 5 0 6 PEAR_ALGII_03 1 0 1 1 2 20 0 21 3 69 71 0 3 257 258 0 3 906 907 0 3 323 324 0 1 2 3 1 2 3 1 2 3 1 2 3 1 0 1 1 2 18 0 19 HMD_ALGII_01 1 0 1 1 2 10 0 7 MDL_ALGII_01 1 0 1 1 2 30 0 31 PEAR_ALGII_01 1 0 1 1 2 4 0 4 PEAR_ALGII_04 1 0 1 1 2 8 0 9 3 158 160 0 3 171 172 0 3 159 160 0 3 105 106 0 3 73 72 0 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 Note: The letters before the first “_” in the TextbookID represent the publisher’s abbreviation; the middle letters denote the subject abbreviation, such as Algebra I (ALGI) and Algebra II (ALGII); the number at the end of each TextbookID indicates the sequence number of the subject textbook published by the publisher. 1 = Mathematics Topics; 2 = Motivational Materials; 3 = Mathematics Reasoning. “NaN” means no such component in the textbook data. The expected hitting time is a statistical measure used to assess the sequential patterns within textbooks. A hitting time of “0” indicates that once a component is reached for the first time starting from itself, it takes zero pages to reach that component again. This occurs because 67 the component has already been visited, so the first instance of reaching it is considered immediate. The values in the matrices signify how quickly each component can be accessed. Across the analysis of 18 Algebra I textbooks, the average expected number of pages required for the process to reach Motivational Materials from Mathematics Topics for the first time is 13 pages (SD = 9.05), ranging from 3 to 34 pages. Three textbooks with missing probability matrix data and one without such hitting behavior were excluded from this analysis. The average expected number of pages required to reach Motivational Materials from Mathematics Reasoning for the first time is 11 pages (SD = 10.26), spanning from 1 to 35 pages, with two textbooks excluded due to missing data. Among the 12 Algebra II textbooks examined, one textbook lacking such hitting behavior being excluded, the average expected number of pages required for the process to reach Motivational Materials from Mathematics Topics for the first time stands at 13 pages (SD = 8.53), with variability from 4 to 30 pages. The average expected number of pages required to reach Motivational Materials from Mathematics Reasoning for the first time is also 13 pages (SD = 9.23), ranging from 4 to 31 pages. The analysis reveals notable variability in how quickly the component Motivational Materials is accessed, quantified in pages per textbook. Lower expected hitting times suggest that the page tends to reach Motivational Materials relatively quickly from other components. This could indicate high accessibility or frequent occurrence of that component. Components with lower hitting times are more critical to the dynamics of the textbook and could represent important transitions that influence overall behavior, as observed in textbooks like PEAR_ALGI_02 and PEAR_ALGI_05. For instance, compared to textbooks like 68 MDL_ALGI_01, where the expected hitting times to first reach Motivational Materials from Mathematics Topics and Mathematics Reasoning are 22 and 23 pages, respectively, PEAR_ALGI_02 and PEAR_ALGI_05 have notably lower hitting times at only 3 pages. Figure 5.13 illustrates that there are indeed more pages dedicated to Motivational Materials in PEAR_ALGI_02 and PEAR_ALGI_05 throughout the entire textbooks compared to MDL_ALGI_01. Furthermore, these Motivational Materials pages appear intermittently in both PEAR_ALGI_02 and PEAR_ALGI_05, as indicated by transition probabilities suggesting a low probability of directly moving to another Motivational Materials page in the subsequent page, approximately 10%. This variability underscores the diverse approaches in component organization across different textbooks, influencing the pacing and accessibility of critical educational opportunities. Overall, these findings contribute to a deeper understanding of how educational materials are structured and navigated, offering implications for curriculum design and student engagement strategies in mathematics education. From the analysis of expected hitting times above, it becomes evident that relying solely on these measures is insufficient to fully characterize the sequencing features of textbooks. These measures are typically interpreted alongside transition probability matrices, as discussed in the preceding section. Similarly, transition probability matrices alone are inadequate for describing the entire structural complexity of a textbook. It’s important to clarify that the values in the transition probability matrix indicate the probability of transitioning from one component to another on the next page. Lower values, such as around 0% transition probability, typically correspond to a larger average number of pages required for the process to reach that component. This suggests that reaching the component is not as quick or immediate compared to components with higher transition probabilities. Additionally, value 0 in the transition probability matrix 69 denotes a very low average transition probability from one component to another in the next step, whereas 0 in the expected hitting time matrix indicates either no expectation of hitting that component from a specific starting component or it represents hitting that component from itself. This distinction highlights the different roles these matrices play in understanding the dynamics and accessibility of components within textbooks. Using different kinds of measures are particularly crucial when comparing sequencing features across different textbooks. Furthermore, Figures 6.3 and 6.4 provide heatmaps of the expected return times for each textbook. These heatmaps visually represent the average number of pages needed to revisit a particular component after leaving it. Each cell in the heatmap corresponds to a specific component, using a blue-to-red color scheme to represent data. In this scheme, blue indicates shorter expected return times, signifying components that are revisited more frequently, while red indicates longer expected return times, highlighting components that take longer to be revisited. The gradient from blue to red provides an intuitive visual cue to quickly identify patterns within the textbooks. 70 Figure 6.3 Heatmaps of Expected Hitting Time for 15 Algebra I Textbooks (Excluding Three with Missing Transition Probability Matrices) 71 Figure 6.3 (cont’d) 72 Figure 6.4 Heatmaps of Expected Hitting Time for 13 Algebra II Textbooks 73 Figure 6.4 (cont’d) 74 Expected Return Time Another useful indicator reflecting sequencing features is the expected return time. In the context of a textbook dataset, expected return time refers to the average number of pages or transitions needed for the learning process to return to a particular component for the first time after leaving it. This concept is crucial in the study of textbook datasets as it provides insight into the long-term behavior of the system. Components with higher expected return times are visited less frequently, while components with lower expected return times are visited more frequently. Table 6.8 and Table 6.9 present the expected return time to each of the three components for the first time after leaving them. Table 6.8 Expected Return Time to Three Components in Algebra I Textbooks (Excluding Three with Missing Transition Probability Matrices) TextbookID Mathematics Topics Motivational Materials BIL_ALGI_01 BIL_ALGI_02 CL_ALGI_01 CPM_ALGI_01 HMC_ALGI_01 HMD_ALGI_01 HMD_ALGI_02 HRW_ALGI_01 MDL_ALGI_01 MGH_ALGI_01 MGH_ALGI_02 PEAR_ALGI_01 PEAR_ALGI_02 1 1 1 1 1 1 1 1 1 1 1 1 1 11 22 - 18 35 9 11 10 21 6 9 7 4 75 Mathematics Reasoning 85 310 - 24 127 232 93 181 180 135 135 195 128 Table 6.8 (cont’d) PEAR_ALGI_04 PEAR_ALGI_05 1 1 10 4 100 154 Note: “-” indicates very few occurrences of the components result in large expected return time values. Table 6.9 Expected Return Time to Three Components in Algebra II Textbooks TextbookID Mathematics Topics Motivational Materials Mathematics Reasoning BIL_ALGII_01 CL_ALGII_01 CPM_ALGII_01 HMD_ALGII_01 HMD_ALGII_02 HRW_ALGII_01 MDL_ALGII_01 MGH_ALGII_01 MGH_ALGII_02 PEAR_ALGII_01 PEAR_ALGII_02 PEAR_ALGII_03 PEAR_ALGII_04 1 1 1 1 1 1 1 1 1 1 1 1 1 18 - 22 11 11 11 28 10 6 5 5 20 9 159 - 65 172 124 259 160 907 907 106 106 324 74 Note: “-” indicates very few occurrences of the components result in large expected return time values. Based on the results in Table 6.8 and Table 6.9, it is evident that Mathematics Topics is the most frequently visited component in all 31 textbooks, with the expected number of pages needed for the process to return to it for the first time after leaving it being only one page. In contrast, the average expected number of pages required for the process to return to Motivational Materials is 13 pages across both Algebra I and Algebra II textbooks. The analysis also reveals 76 that the least frequently visited component across both Algebra I and Algebra II textbooks is Mathematics Reasoning. Specifically, in Algebra II textbooks, the average number of pages needed to return to this component is almost twice that in Algebra I textbooks, with 280 pages and 149 pages, respectively. In summary, this chapter provides a comprehensive analysis of the statistical findings derived from the mapping of the three major components of 31 Algebra textbooks (18 Algebra I and 13 Algebra II). The analysis begins with a descriptive overview of the textbooks, focusing on their page counts and the percentage distribution of components across pages. The findings reveal notable variability in textbook length and percentage distribution of three major components. The sequencing features, as analyzed through Markov chain analysis, demonstrate continuity within components and structured transitions between them. The expected hitting and return times offer a deeper understanding of the accessibility and frequency of components, contributing to a nuanced interpretation of the textbook structure. 77 CHAPTER 7. RESULTS OF THE STATISTICAL ANALYSIS: SEQUENCING OF CONTENT TOPICS In this chapter, I will focus on the content topics mapping display introduced in Figure 4.3. This display has attracted the interest of researchers, educators, and textbook designers because it focuses purely on the content itself and illustrates how the content is structured throughout the entire textbook. Due to our coding procedure, some pages were not coded and will be excluded from this analysis. To present the results of 31 Algebra textbooks (18 Algebra I and 13 Algebra II), I will first describe the percentage of coded and non-coded content, and explain how I cleaned the data before applying further analysis. Following this, I will introduce and present the results from model-based approach to illustrate their sequential features. 7.1 Descriptive Analysis Table 7.1 and Table 7.2 display the proportion of pages that were not coded for content in each textbook. Table 7.1 Proportion of Non-coded Pages in 18 Algebra I Textbooks TextbookID Original Pages Deleted Pages Percentage of Deleted Pages BIL_ALGI_01 BIL_ALGI_02 CL_ALGI_01 CPM_ALGI_01 GM_ALGI_01 HMC_ALGI_01 HMD_ALGI_01 HMD_ALGI_02 513 623 447 622 491 633 926 748 19.88% 20.22% 14.32% 24.6% 6.72% 21.48% 31.32% 35.03% 102 126 64 153 33 136 290 262 78 Table 7.1 (cont’d) HRW_ALGI_01 MDL_ALGI_01 MGH_ALGI_01 MGH_ALGI_02 PEAR_ALGI_01 PEAR_ALGI_02 PEAR_ALGI_03 PEAR_ALGI_04 PEAR_ALGI_05 PEAR_ALGI_06 Table 7.2 906 900 810 810 783 771 712 502 771 700 283 221 189 199 180 193 227 82 193 43 31.24% 24.56% 23.33% 24.57% 22.99% 25.03% 31.88% 16.33% 25.03% 6.14% Proportion of Non-coded Pages in 13 Algebra II Textbooks TextbookID Original Pages Deleted Pages Percentage of Deleted Pages BIL_ALGII_01 CL_ALGII_01 CPM_ALGII_01 HMD_ALGII_01 HMD_ALGII_02 HRW_ALGII_01 MDL_ALGII_01 MGH_ALGII_01 MGH_ALGII_02 PEAR_ALGII_01 PEAR_ALGII_02 PEAR_ALGII_03 PEAR_ALGII_04 639 623 777 1034 870 1034 963 908 908 958 958 650 812 126 72 245 305 265 298 220 229 229 233 239 89 167 79 19.72% 11.56% 31.53% 29.5% 30.46% 28.82% 22.85% 25.22% 25.22% 24.32% 24.95% 13.69% 20.57% The descriptive analysis above reveals that the percentage of deleted pages due to not being coded ranged from 6.14% (33 pages) to 35.03% (290 pages) across the 18 Algebra I textbooks, averaging 22.48% (165 pages). Among the 13 Algebra II textbooks, the average percentage of deleted pages due to not being coded is 23.72% (209 pages), with a range from 11.56% (72 pages) to 31.53% (305 pages). The relatively lower minimum proportion in Algebra I textbooks is attributed to the inclusion of two practice books, whereas Algebra II textbooks have only one, which typically does not contain pages such as lesson/chapter reviews or summaries. After removing the non-coded pages from the data, Figure 7.1 illustrates an example of the content topics mapping. The columns represent the actual page numbers in the textbook, while the rows remain unchanged. Figure 7.1 Example of Content Topics Mapping Display after Removing Non-coded Pages With the updated content topics mapping display, specific rules have been established for further analysis. Columns containing only one “1” or one darker blue cell already indicate the corresponding content topic on those pages and require no further action. However, for columns with two or more “1”s, the following rules are applied to retain only one “1” in each column, ensuring consistency of variables across all sequencing time points represented by the columns: 80 1. If there are “1”s between the rows Foundations and Advanced_Algebra, retain the “1” located closest to Advanced_Algebra (which is also the lowest row) and replace other “1”s with “0”. This approach is based on the assumption that when a page in Algebra textbooks contains any algebra topic, that topic is likely the main focus, especially when other subjects’ topics, such as Geometry, also appear on the same page. Additionally, if there are multiple algebra topics on a page, the most advanced topic is marked as “1”, covering topics not previously learned, aligning with the classroom learning process. 2. If there is no “1” between the rows Foundations and Advanced_Algebra, but “1”s are located in other rows, retain the lowest “1” in the column. This approach ensures that only one “1” is kept per column, emphasizing the relatively advanced topic that reflects how the textbook distributes different content topics. Following these rules, the example textbook’s content topics mapping display can be revised as shown in Figure 7.2. This display format will also be used for the final analysis. Figure 7.2 Example of Content Topics Mapping Display for Final Analysis e.g., a jump, and using number -1 in the new comparison dataset to indicate the changing direction e.g., a jump, and using number 1 in the new comparison dataset to indicate the changing direction 7.2 Measures From Model-Based Approach The objective of this research is to statistically capture the sequencing features of the textbook based on its display. After following the previous steps, the display for conducting the 81 final analysis is shown in Figure 7.2. Unlike the three major components mapping display, Figure 7.2 presents a binary matrix where “1” indicates that the corresponding content topic is covered on that page. The content topics listed in the rows exhibit a naturally ordered characteristic; for instance, students typically learn linear equations before non-linear equations. To capture the sequencing features based on Figure 7.2, a different approach is needed compared to our previous analysis, which treated each major component as an equal and individual state. In this case, the focus is not on the change between different individual content topics, but rather on whether the next page transitions to a more advanced topic or reverts to a relatively less advanced one. Given the naturally ordered nature of these content topics, I compare each consecutive pair of pages to capture the level of difficulty between them. In Figure 7.2, this involves identifying if there is a change (“jump”) between the topics covered on two consecutive pages. Specifically, I use “1” to indicate a jump to a relatively advanced topic compared to the previous page, “0” to indicate the continuity of the same topic, and “-1” to indicate a jump to a relatively less advanced topic. The resulting comparison dataset for each textbook consists of a row vector with values of -1, 0, and 1. Each value represents the comparison result between the current page and the previous page, reflecting whether there was a jump or no change in difficulty level in the content topic distribution. This approach mirrors the methodology applied in Chapter 6, where I analyzed transitions between the current page and its previous page. As in Chapter 6, my goal is to determine the probabilities of different jumps and no jumps between two consecutive pages. This analysis will reveal the sequencing or continuity features of the content topics distribution in the textbook. 82 To achieve this measure, I calculated the probabilities for each category based on statistical model. The outcome variable for each textbook is the comparison dataset, which is a categorical outcome with three values (-1, 0, 1). Since this dataset exhibits sequential features where the current value is influenced by its previous value, the previous value is included in the model when calculating the probabilities. Here is the equation of a multinomial logistic regression model: 𝑙𝑜𝑔𝑖𝑡(𝑃(𝑦+ = 𝑘 | 𝑦+,%)) = 𝛽)( + 𝛽%(𝑦+,% Where 𝑘 represents each category (-1, 0, 1), and 𝑃(𝑦+ = 𝑘 | 𝑦+,%) is the probability of 𝑦+ being in category 𝑘 given the previous value 𝑦+,%. The model is usually expressed using a baseline category (e.g., category -1), and the logits for the other categories are relative to this baseline. For instance, if category -1 is the baseline, the model for categories 0 and 1 would be: 𝑙𝑜𝑔𝑖𝑡(𝑃(𝑦+ = 0 | 𝑦+,%)) = 𝛽)) + 𝛽%)𝑦+,% = log ( 𝑙𝑜𝑔𝑖𝑡(𝑃(𝑦+ = 1 | 𝑦+,%)) = 𝛽)% + 𝛽%%𝑦+,% = log ( 𝑃(𝑦+ = 0 | 𝑦+,%) 𝑃(𝑦+ = −1 | 𝑦+,%) ) 𝑃(𝑦+ = 1 | 𝑦+,%) 𝑃(𝑦+ = −1 | 𝑦+,%) ) And the probabilities for category -1, 0, 1 can then be calculated as: 𝑃(𝑦+ = −1 | 𝑦+,%) = 1 % 1 + ∑ "’) exp (𝛽)" + 𝛽%"𝑦+,%) 𝑃(𝑦+ = 0 | 𝑦+,%) = 𝛽)) + 𝛽%)𝑦+,% = 𝑃(𝑦+ = 1 | 𝑦+,%) = 𝛽)% + 𝛽%%𝑦+,% = 𝑒𝑥𝑝(𝛽)) + 𝛽%)𝑦+,%) % "’) exp (𝛽)" + 𝛽%"𝑦+,%) 1 + ∑ exp(𝛽)% + 𝛽%%𝑦+,%) % "’) exp (𝛽)" + 𝛽%"𝑦+,%) 1 + ∑ Table 7.3 and Table 7.4 below show the results of all 31 Algebra textbooks (18 Algebra I and 13 Algebra II) 83 Table 7.3 Probabilities of Each Transition in 18 Algebra I Textbooks TextbookID BIL_ALGI_01 BIL_ALGI_02 CL_ALGI_01 CPM_ALGI_01 GM_ALGI_01 HMC_ALGI_01 HMD_ALGI_01 HMD_ALGI_02 HRW_ALGI_01 MDL_ALGI_01 MGH_ALGI_01 MGH_ALGI_02 PEAR_ALGI_01 PEAR_ALGI_02 PEAR_ALGI_03 PEAR_ALGI_04 PEAR_ALGI_05 PEAR_ALGI_06 -1 2.21% 2.23% 1.32% 4.08% 2.64% 3.64% 1.74% 1.86% 2.58% 1.77% 1.62% 1.64% 2.33% 3.48% 3.11% 2.64% 3.48% 1.83% 0 95.10% 95.14% 97.11% 91.20% 94.73% 93.12% 96.52% 95.45% 95.32% 96.01% 96.60% 96.55% 95.00% 92.70% 92.95% 93.77% 92.70% 95.57% 1 2.70% 2.63% 1.58% 4.72% 2.64% 3.24% 1.74% 2.69% 2.10% 2.22% 1.78% 1.81% 2.67% 3.83% 3.94% 3.60% 3.83% 2.60% Note: “-1” = transit to less advanced topic in next page; “0” = transit to same topic in next page; “1” = transit to more advanced topic in next page The analysis of 18 Algebra I textbooks reveals that the average probability of transitioning to a less advanced topic on the next page across these textbooks is 2.46% (SD = 0.79%), ranging from 1.32% in CL_ALGI_01 to 4.08% in CPM_ALGI_01. Conversely, the average probability of transitioning to a more advanced topic on the next page is 2.80% (SD = 0.87%), varying from 1.58% in CL_ALGI_01 to 4.72% in CPM_ALGI_01. Additionally, the 84 average probability of staying on the same topic on the next page is 94.75% (SD = 1.61%), ranging from 91.20% in CPM_ALGI_01 to 97.11% in CL_ALGI_01. Figure 7.3 and Figure 7.4 depict the mapping displays for two Algebra I textbooks, CL_ALGI_01 and CPM_ALGI_01, respectively, providing a visual representation for final analysis. These figures illustrate the differences between textbooks with fewer transitions (e.g., CL_ALGI_01) and those with more transitions (e.g., CPM_ALGI_01). The visual mappings highlight how content progression varies across textbooks, offering insights into the structure and flow of topics within each textbook. Figure 7.3 Mapping Display for Final Analysis of CL_ALGI_01 Figure 7.4 Mapping Display for Final Analysis of CPM_ALGI_01 85 Table 7.4 Probabilities of Each Transition in 13 Algebra II Textbooks TextbookID BIL_ALGII_01 CL_ALGII_01 CPM_ALGII_01 HMD_ALGII_01 HMD_ALGII_02 HRW_ALGII_01 MDL_ALGII_01 MGH_ALGII_01 MGH_ALGII_02 PEAR_ALGII_01 PEAR_ALGII_02 PEAR_ALGII_03 PEAR_ALGII_04 -1 1.37% 0.91% 1.89% 1.93% 1.33% 1.91% 2.57% 1.92% 1.92% 2.22% 2.23% 1.97% 2.18% 0 97.25% 97.63% 96.22% 96.01% 96.84% 96.04% 94.59% 95.71% 95.71% 95.15% 95.11% 95.88% 95.33% 1 1.37% 1.46% 1.89% 2.07% 1.83% 2.05% 2.84% 2.37% 2.37% 2.63% 2.65% 2.15% 2.49% Note: “-1” = transit to less advanced topic in next page; “0” = transit to same topic in next page; “1” = transit to more advanced topic in next page The analysis of 13 Algebra II textbooks shows that the average probability of transitioning to a less advanced topic on the next page across these textbooks is 1.87% (SD = 0.42%), ranging from 0.91% in CL_ALGII_01 to 2.57% in MDL_ALGII_01. Similarly, the average probability of transitioning to a more advanced topic on the next page is 2.17% (SD = 0.43%), varying from 1.37% in BIL_ALGII_01 to 2.84% in MDL_ALGII_01. Additionally, the average probability of remaining on the same topic on the next page is 95.96% (SD = 0.84%), with values ranging from 94.59% in MDL_ALGII_01 to 97.63% in CL_ALGII_01. Figure 7.5 and Figure 7.6 present the mapping displays for two Algebra II textbooks, BIL_ALGII_01 and CL_ALGII_01, respectively, providing visual insights for final analysis. 86 These figures illustrate how these textbooks differ in transitioning to more advanced topics on subsequent pages, with BIL_ALGII_01 showing fewer transitions compared to MDL_ALGII_01. The visual mappings offer a clear depiction of the structural variations in topic progression within each textbook, highlighting distinct approaches to content complexity. Figure 7.5 Mapping Display for Final Analysis of BIL_ALGII_01 Figure 7.6 Mapping Display for Final Analysis of MDL_ALGII_01 Based on the comprehensive analysis presented in Chapter 7, it becomes evident that the sequencing of content topics in Algebra textbooks is characterized by a balance between continuity and variability. Through detailed mapping displays and statistical models, this chapter has revealed how textbooks manage transitions between more advanced and less advanced topics. The findings highlight consistent patterns of staying on the same topic and occasional shifts to more or less advanced content across both Algebra I and Algebra II textbooks. These insights provide educators and curriculum developers with a deeper understanding of how each textbook structures educational content and supports learning progression. 87 CHAPTER 8. BRIDGING THEORY AND PRACTICE Describing a textbook effectively to those concerned with it—such as teachers, parents, and students—is crucial because it directly influences educational decisions and outcomes. Typically, textbooks are evaluated based on surface content and structure, such as the number of topics and chapters/lessons. However, this approach may overlook the deeper, more nuanced aspects of a textbook’s effectiveness. In this chapter, I will demonstrate the practical application of measures derived from my textbook analysis by comparing two math textbooks. This comparison will illustrate how findings can enhance educational practices. Through specific examples, I aim to offer valuable insights for teachers, parents, and students focused on optimizing learning outcomes. I randomly selected two textbooks (BIL_ALGI_02 and HMD_ALGI_02) to assist a female teacher in selecting one for teaching 9th grade Algebra I this academic year. At this stage, assuming cost is not a concern, her priority is to understand the structure before delving into content details. However, relying solely on examining the table of contents and flipping through pages offers only a basic understanding. This approach lacks direct visualization and meaningful quantitative data essential for a well-informed decision. The basic information and descriptive analysis of the two textbooks is listed in Table 8.1 and Table 8.2: 88 Table 8.1 Basic Information of Two Textbooks for Analyzing and Comparing Textbook ISBN-10 ISBN-13 Title Publisher BIL_ALGI_02 1608408388 9781608408382 HMD_ALGI_02 0547647034 9780547647036 BIG IDEAS MATH Algebra 1: Common Core Curriculum Holt McDougal Algebra 1. Common Core Edition Big Ideas Learning, LLC. Houghton Mifflin Publishing Company Public Year 2015 2012 Table 8.2 Descriptive Analysis of Two Textbooks Variables BIL_ALGI_02 HMD_ALGI_02 Number of Original Pages Percentage of Mathematics Topics Percentage of Motivational Materials Percentage of Mathematics Reasoning 623 95.02% 4.65% 0.32% 748 89.30% 9.63% 1.07% These two textbooks are designed for teaching Algebra I under the Common Core State Standards (CCSS). BIL_ALGI_02, published by Big Ideas Learning, LLC. in 2015, spans 623 pages, with a predominant focus on Mathematics Topics, occupying 95.02% of its content. In contrast, HMD_ALGI_02, published by Houghton Mifflin Publishing Company in 2012, covers 748 pages, with 89.30% dedicated to Mathematics Topics. BIL_ALGI_02 includes 4.65% for Motivational Materials and allocates 0.32% to Mathematics Reasoning, while HMD_ALGI_02 devotes 9.63% and 1.07% respectively to these aspects. From this information, the teacher can identify several key points. HMD_ALGI_02, with its additional 125 pages compared to BIL_ALGI_02, requires students to cover approximately one extra page per day over the typical 180 instructional days for 9th graders in the United States. This difference underscores the potential for deeper coverage and exploration of content 89 within HMD_ALGI_02. Moreover, HMD_ALGI_02’s higher allocation to Mathematics Reasoning offers enhanced opportunities for students to develop critical thinking and problem- solving skills through complex problem-solving tasks. Additionally, the integration of Motivational Materials at a higher frequency in HMD_ALGI_02 can help sustain student engagement and motivation throughout the learning process. However, based on the current information alone, the teacher cannot yet determine how the three components (Mathematics Topics, Motivational Materials, Mathematics Reasoning) are distributed throughout the textbooks, nor can she grasp the overall distribution of different content topics. Therefore, Figures 8.1 to 8.3 present three distinct types of mapping displays for both textbooks, offering clear representations of their structures. Unlike flipping through pages, which can be cumbersome for gaining a holistic view, these mapping displays provide a straightforward 2-D image that unfolds the content in an accessible manner. This visual approach allows for a direct and comprehensive understanding of how the textbooks are organized, aiding in informed decision-making regarding their suitability for educational use. Figure 8.1 Three Components Mapping Displays of Two Textbooks 90 Figure 8.2 Original Content Topics Mapping Displays of Two Textbooks Figure 8.3 Final Analysis Content Topics Mapping Displays of Two Textbooks Now, let’s consider what insights the teacher can collect from the three figures. First, Figure 8.1 offers a visual representation of how the three major components are organized within each textbook. It becomes apparent that BIL_ALGI_02 contains more consecutive pages dedicated to Mathematics Topics and Motivational Materials compared to HMD_ALGI_02. However, in HMD_ALGI_02, Motivational Materials pages are interspersed at regular intervals and typically transition back to Mathematics Topics on the subsequent page. In contrast, the distribution of Motivational Materials pages in BIL_ALGI_02 appears more scattered, with some pages followed directly by another Motivational Materials page. Additionally, there is a 91 noticeable disparity in the presence of Mathematics Reasoning pages, with HMD_ALGI_02 featuring more than BIL_ALGI_02. Moving to Figure 8.2, the focus shifts to the distribution of different content topics, providing a detailed view of their organization throughout the textbooks. Unlike Figure 8.1, which outlines the overall layout of the three components, Figure 8.2 specifically illustrates how the content topics are structured and sequenced within each textbook. This visual allows the teacher to assess the flow and progression of mathematical concepts presented to students. She observes that both textbooks cover Probability and Statistics towards the end, while Geometry topics are included in both. However, BIL_ALGI_02 also incorporates Number and System topics alongside algebraic concepts. In contrast, HMD_ALGI_02 maintains a more focused approach on linear and non-linear equations, as well as linear and non-linear functions. Finally, Figure 8.3 provides mapping displays for statistically analyzing the sequencing of content topics in the two textbooks. After excluding pages with reviews, summaries, tests, and other non-content materials, it is revealed that both textbooks contain a similar number of instructional pages, with BIL_ALGI_02 slightly exceeding HMD_ALGI_02—contrary to their original total page counts. Moreover, BIL_ALGI_02 exhibits more transitions between topics compared to HMD_ALGI_02, suggesting a less linear progression of topics. Therefore, at this stage, the teacher need not be concerned about the total number of pages in each textbook. Our descriptive analysis indicates that BIL_ALGI_02 contains 497 instructional pages, while HMD_ALGI_02 contains 486 instructional pages, indicating comparable lengths in core instructional content despite their different total page counts. While these straightforward mapping approaches highlight differences between the textbooks, they do not provide a quantitative comparison. This limitation makes it challenging to 92 effectively communicate these differences to stakeholders who rely on data-driven insights. Without quantitative indicators, describing variations between the textbooks remains less informative and nuanced. Tables 8.3 to 8.6 and Figure 8.4 present the statistical results derived from the displays. Table 8.3 Transition Probability Matrices of Two Textbooks BIL_ALGI_02 Mathematics Topics Motivational Materials Mathematics Reasoning Mathematics Topics Motivational Materials Mathematics Reasoning 0.95 0.93 1.00 0.04 0.07 0.00 0.00 0.00 0.00 HMD_ALGI_02 Mathematics Topics Motivational Materials Mathematics Reasoning Mathematics Topics Motivational Materials Mathematics Reasoning 0.89 1.00 0.25 0.10 0.00 0.50 0.01 0.00 0.25 93 Figure 8.4 Transition Probability Diagrams of Two Textbooks The relatively higher continuity of Mathematics Topics and Motivational Materials pages in BIL_ALGI_02, with transition probabilities of 0.95 and 0.07 respectively, suggests that this textbook offers a more consistent and focused approach to these components. In contrast, HMD_ALGI_02, with a transition probability of 0.89 for Mathematics Topics and 0 for Motivational Materials, presents a more varied structure that integrates Motivational Materials regularly throughout the textbook. This distinction implies that BIL_ALGI_02 may appeal to educators seeking a predictable and structured learning progression, while HMD_ALGI_02 could benefit educators aiming to provide diverse instructional materials that include motivational elements strategically. However, the transition probabilities for Mathematics Reasoning pages reveal a different dynamic. HMD_ALGI_02 demonstrates higher continuity for Mathematics Reasoning, with a probability of 0.25 compared to 0.00 in BIL_ALGI_02. This indicates that students using HMD_ALGI_02 are more frequently exposed to higher-order thinking and problem-solving exercises, crucial for developing deeper mathematical understanding. The clear transitions within 94 Mathematics Reasoning in HMD_ALGI_02, as opposed to their absence in BIL_ALGI_02, underscore the textbook’s emphasis on reinforcing these complex skills. Table 8.4 Expected Hitting Time Matrices of Two Textbooks BIL_ALGI_02 Mathematics Topics 0 1 1 Motivational Materials 23 0 24 Mathematics Reasoning 309 311 0 Mathematics Topics Motivational Materials Mathematics Reasoning HMD_ALGI_02 Mathematics Topics 0 1 2 Motivational Materials 10 0 5 Mathematics Reasoning 123 124 0 Mathematics Topics Motivational Materials Mathematics Reasoning The expected hitting time matrices in Table 8.4 indicate that in BIL_ALGI_02, it takes an average of 309 pages to transition from Mathematics Topics to Mathematics Reasoning, whereas in HMD_ALGI_02, this transition occurs in 123 pages on average. This disparity suggests that HMD_ALGI_02 includes more higher-order problems compared to BIL_ALGI_02, implying that students using HMD_ALGI_02 will encounter more frequent and potentially more challenging problems. A similar trend is observed for transitioning from Mathematics Topics to Motivational Materials, highlighting HMD_ALGI_02’s emphasis on integrating engaging and motivating materials. 95 Table 8.5 Expected Return Time to Three Components in Two Textbooks TextbookID Mathematics Topics Motivational Materials BIL_ALGI_02 HMD_ALGI_02 1 1 22 11 Mathematics Reasoning 310 93 Moreover, based on the results in Table 8.5, we can conclude that Mathematics Topics is visited most frequently in both textbooks, with the expected number of pages needed to return to it for the first time after leaving being only one page. In contrast, Motivational Materials is visited more frequently in HMD_ALGI_02 than in BIL_ALGI_02, as indicated by the smaller number of pages needed to return to it for the first time. The same trend applies to Mathematics Reasoning; in BIL_ALGI_02, the average number of pages needed to return to this component is over three times that in HMD_ALGI_02, with 310 pages and 93 pages, respectively. These findings highlight the different emphases and frequencies of these key components within each textbook’s structure, providing insights into how often students are exposed to motivational materials and higher-order reasoning problems in each textbook. Table 8.6 Probabilities for Each Transition in Two Textbooks TextbookID BIL_ALGI_02 HMD_ALGI_02 -1 2.23% 1.86% 0 95.14% 95.45% 1 2.63% 2.69% Note: “-1” = transit to less advanced topic in next page; “0” = transit to same topic in next page; “1” = transit to more advanced topic in next page Finally, Table 8.6 provides the probabilities of different topic transitions between two consecutive pages in both textbooks, after excluding pages that were lesson/chapter reviews, 96 summaries, tests, or transitional pages between chapters. BIL_ALGI_02 uses 497 pages for content analysis, while HMD_ALGI_02 designates 486 pages. Notably, a higher proportion of pages are excluded from content analysis in HMD_ALGI_02 (35.03%) compared to BIL_ALGI_02 (20.22%). This indicates that HMD_ALGI_02 contains more ancillary materials, such as reviews and tests, which are not directly part of the core instructional content. The analysis reveals that there are more topic jumps between consecutive pages in BIL_ALGI_02 compared to HMD_ALGI_02. The probability of transitioning to a less advanced topic on the next page is 2.23% in BIL_ALGI_02, compared to 1.86% in HMD_ALGI_02. The probabilities of transitioning to a more advanced topic on the next page are similar in both textbooks, around 2.6%. Additionally, the probability of remaining on the same topic on the next page is approximately 95%, with HMD_ALGI_02 having a slightly higher probability. This suggests that HMD_ALGI_02 offers a more stable progression in topic difficulty, while BIL_ALGI_02 exhibits more variability in content topics transitions. So far, the teacher has received a thorough analysis of the structure and sequencing of various components and content topics within each textbook. In deciding between the two textbooks, she should first consider her teaching objectives regarding structure, diversity of materials, emphasis on higher-order thinking, and the balance between core instructional content and ancillary materials. Each textbook offers distinct advantages that align differently with educational goals, student needs, and instructional preferences. For instance, BIL_ALGI_02 features more frequent topic transitions between consecutive pages compared to HMD_ALGI_02. This suggests broader exposure to diverse mathematical topics but may lead to a less consistent progression in difficulty, especially when transitions to less advanced topics on the next page frequently recall previous knowledge. Conversely, 97 HMD_ALGI_02 stands out with its integration of Motivational Materials and emphasis on higher-order thinking skills, fostering a more varied and supportive learning environment. However, it includes a higher proportion of ancillary materials, such as reviews and tests, which may not directly contribute to core instructional content. Educators’ preferences may vary based on how these additional materials align with their teaching strategies and curriculum goals. 98 CHAPTER 9. DISCUSSION AND LIMITATIONS The current study undertook a comprehensive analysis of 31 Algebra textbooks used in 9th-grade classrooms across the United States, focusing on the structure and sequencing of different components and content topics to emphasize the variation in learning opportunities provided by different textbooks. By applying both visual mapping and statistical analysis, including Markov chain techniques and a model-based approach, I have been able to simplify and interpret the complex dependencies within various educational sequences. The findings highlight the impact of different textbook structures on the learning experiences of students. This chapter will discuss the implications of these findings, their alignment with existing literature, and potential recommendations for educators and researchers. Mathematics textbooks represent carefully designed learning opportunities intentionally provided to students by policymakers. Anything studied from these textbooks aims to help educators understand these opportunities. For instance, the total number of pages in a mathematics textbook, which varies significantly across countries. Previous studies have shown that U.S. mathematics textbooks typically have far more pages than those in other countries (Valverde et al., 2002). The current analysis also revealed significant variation in the number of pages within U.S. textbooks, with the average number being 704 pages for Algebra I and 865 pages for Algebra II. While, in some countries, such as China, the limited number of pages is due to the belief that more pages in a textbook would place additional pressure on students, rather than extra pages being unnecessary. This belief also contributes to the relatively few real-world practice problems in Chinese mathematics textbooks, as including more would increase the textbook’s length. Another important aspect is the content topics covered in the mathematics textbook. As indicated by previous studies (Valverde et al., 2002), U.S. textbooks cover more 99 content topics than those in other countries. In contrast, countries like Singapore cover fewer topics but delve into each topic more deeply, dedicating more pages to each. The above mentioned aspects reflect some of the opportunities provided by the textbook, but they are not enough to reveal all the opportunities that a textbook offers. Policymakers make deliberate efforts to shape and improve mathematics education through well-structured and thoughtfully designed learning materials (Rezat, Fan & Pepin, 2021). However, studies on the structure of textbooks are limited. The current study filled the gap through combined both visual and statistical analyses and found several key findings. 9.1 Sequencing Patterns of Different Component in Algebra Textbooks Firstly, visual and Markov chain analysis was applied to uncover the sequencing features of 18 Algebra I and 13 Algebra II textbooks, focusing on: the transition probabilities, expected hitting time, and expected return time between three different components (Mathematics Topics, Motivational Materials, Mathematics Reasoning) within the textbooks. These statistical measures and corresponding visualizations reveal significant insights into how these textbooks’ structure and transition between various types of components. Students using different textbooks encounter various components at differing frequencies. For example, compared to other textbooks, students using CPM_ALGI_01 are more likely to encounter higher-order problems more frequently and earlier in the learning process, with an average of 28 pages needed to do so, also this textbook has consecutive pages featuring higher- order problems, as indicated by the probability of transitioning from Mathematics Reasoning to Mathematics Reasoning is 19%. While existing literature has confirmed the role of higher-order thinking skills in students’ learning (OECD, 2023), it is evident that CPM_ALGI_01 contains more higher-order problems compared to other analyzed textbooks. However, the relationship 100 between the frequency and organization of these skills and students’ academic performance has not been thoroughly explored. There is still a lack of evidence to indicate how textbook design can maximize the importance of higher-order thinking skills. The same situation applies to Motivational Materials. Students using certain textbooks, such as PEAR_ALGI_02 and PEAR_ALGI_05, are more likely to encounter motivational phrases or sentences more frequently and earlier in the learning process. These materials encourage students to try harder when faced with challenging problems or suggest alternative strategies for problem-solving, with an average frequency of every 3 pages. Additionally, the probability of transitioning to another page with Motivational Materials in the next page is around 10%, indicating that these phrases or sentences often appear on consecutive pages. One interesting observation is that in both PEAR_ALGI_02 and PEAR_ALGI_05, the average number of pages needed to reach a higher-order problem during the learning process is over 100 pages, significantly more than CPM_ALGI_01. Additionally, students are not exposed to these problems on consecutive pages, with the corresponding transition probabilities being 0%. This indicates that the three textbooks employ very different strategies for organizing their components. CPM_ALGI_01 focuses more on cultivating students’ critical thinking skills and encouraging in complex cognitive processes, such as conceptualization, organization, and application of mathematical principles to real-world scenarios through introducing higher-order problems more frequently and consecutively. In contrast, PEAR_ALGI_02 and PEAR_ALGI_05 place a greater emphasis on psychological factors in students’ math learning, using motivational materials to encourage perseverance and alternative problem-solving strategies. However, we can observe the variability in sequencing patterns, but cannot determine which organization is more beneficial to students’ learning improvement, as no student 101 achievement data were linked to these findings. Further research is needed to evaluate the impact of these different instructional approaches on student performance. Controlled studies that track academic outcomes in relation to the specific design of each textbook would provide valuable insights. Such data would help educators and textbook developers understand the effectiveness of various approaches and guide the development of more effective educational opportunities. The application of Markov chain analysis in this study has elucidated the sequencing features of Algebra I and II textbooks, highlighting patterns of continuity and variability in different component progression. These findings contribute to a deeper understanding of how educational materials are structured and navigated, offering valuable implications for curriculum design and instructional practices in mathematics education. 9.2 Transition Patterns Between Different Levels of Topics in Algebra Textbooks Secondly, this study provides valuable insights into the progression and sequencing of content topics within Algebra textbooks, using transition probabilities between different levels of topic difficulty to reveal the patterns for people who might have difficulty. The findings indicate that in Algebra textbooks, the average probability of transitioning to a less advanced topic on the subsequent page is relatively low, suggesting a pedagogical strategy that favors maintaining a consistent level of difficulty with minimal regression to simpler content. Conversely, the average probability of transitioning to a more advanced topic is slightly higher, implying a propensity to introduce more complex topics as students’ progress through the textbook. In addition to the transition probabilities between different levels of topic difficulty, the model-based approach applied in this analysis also indicates the frequency of “jumps” within each textbook. The stability or variability in topic sequencing could influence students’ ability to follow and retain the material. More stable sequencing might facilitate better comprehension and 102 retention, whereas frequent “jumps” might necessitate additional instructional support. Visual mappings provide clear insights into the varying approaches of different textbooks regarding content flow and structure. For instance, CPM_ALGI_01 shows more “jumps” in its mapping, as indicated by the sum of transition probabilities to different topics on the next page being around 9%, compared to other analyzed textbooks in this study. Moreover, the mapping displays show different patterns of the content topics, as mentioned in Chapter 5. For example, some among the 31 Algebra I and II textbooks analyzed, only two do not include any Probability and Statistics topics. One textbook covers a small proportion of pages dedicated to these topics in the middle, while concurrently covering algebra topics. In half of the textbooks (16 in total), pages exclusively covering Probability and Statistics topics first appear in the middle, with CPM_ALGI_01 being a prime example. These topics reappear later in the last third of the book. The remaining 12 textbooks adhere to a rationale that pages exclusively devoted to Probability and Statistics topics first appear towards the end or at least in the last third of the textbook. These 12 textbooks, from various publishers and covering both Algebra I and Algebra II, do not exhibit a consistent pattern in this regard. While different patterns of content topics are observed, it is still unclear what these differences mean. In other words, the question of what kind of sequencing of content topics is most beneficial to students cannot be answered until these characteristics can be linked with students’ performance data. One thing we can confirm is that the order of content topics in a textbook does matter (Cogan et al., 2001; Ferrini-Mundy et al., 2007; Schmidt, 2004, 2008, 2012; Schmidt et al., 2005, 2006, 2012). However, there is no single optimal ordering; different states may have different rationales for their sequencing choices. The importance of this approach in the study is to provide a statistical method for using their own standards to study the 103 impact of content topic sequencing on their students’ learning outcomes. The order of topics in the current study is just one option rather than a definitive critique. 9.3 The Relationship with Students’ Opportunity to Learn (OTL) Consider a situation where a textbook designer, a teacher who used the textbook in the classroom, and a student in the teacher’s classroom were asked whether they have taught or learned the concept of linear equations. The textbook designer might confidently assert that the concept of linear equations is thoroughly covered in the textbook, including various examples and exercises. The teacher might explain that while the topic is included in the curriculum, the extent of coverage and emphasis in the classroom may vary depending on the pace of the class and the students’ comprehension levels. The student, on the other hand, might have a different perspective, possibly recalling whether they remember or understood the concept based on their personal learning experience, there are diverse factors that influence the student’s perceptions and retention of the knowledge. However, one primary goal of schooling is to teach academic knowledge and skills, such as literacy, numeracy, scientific understanding, and critical thinking. This is typically achieved through classroom instruction by teachers, supplemented by learning materials like textbooks. Since the final recipients of academic knowledge are students, everyone involved in the educational process should consider how to effectively convey this knowledge. Recall the situation mentioned above: a textbook designer, a teacher, and a student each play a role in the educational pathway. The textbook designer must carefully consider where and how to present the concept of linear equations in the textbook, as well as what supporting materials can enhance the learning of the concept. Their design serves as a model for teachers who use the textbook, guiding them in structuring lessons, highlighting learning tips, and 104 assigning practices to students according to the textbook’s layout. Consequently, every aspect of the textbook—what content or any material is included, where it is placed, and how it is sequenced—can influence classroom teaching and, ultimately, the learning experiences of students. Thus, both the sequencing of content topics and the sequencing of other materials play equally important roles in shaping students’ learning opportunities in schools. Sequencing can significantly influence the retention and transfer of knowledge. Bransford, Brown, and Cocking (2000) argue that well-sequenced materials facilitate the formation of mental models, which are crucial for understanding and applying knowledge in new contexts. When textbooks are sequenced effectively, they help students build a coherent understanding of the subject matter, making it easier for them to retrieve and apply knowledge in different situations. Similarly, Schmidt et al. (2005) highlighted the impact of textbook sequencing on learning opportunities in mathematics. Their study using visual analysis found that textbooks with a coherent and well-structured sequence of topics were associated with higher student achievement. Specifically, textbooks that followed a logical progression, with topics building on one another and revisiting key concepts, were more effective in promoting deep understanding and mastery of mathematical skills. Furthermore, integrating various types of materials, such as motivational texts and higher-order thinking tasks, within the sequence can create a more supportive learning environment. For example, Boaler (2002) demonstrated that including real-world applications and problem-solving activities throughout the textbook sequence can enhance students’ ability to apply mathematical concepts to practical situations. This approach not only enriches the learning experience but also helps students see the relevance and value of the subject matter. 105 Capturing these sequencing features helps reveal the textbook’s structure, allowing educators and designers to assess its alignment with state or national standards. This insight is crucial for enhancing textbook design to ensure it effectively supports educational goals and maximizes learning opportunities for students. However, the real world is not this straightforward. The current findings indicate that students in different schools using different textbooks are likely to have varied learning experiences and opportunities, regardless of their family background or other demographic factors, which are typically included in data analysis when exploring factors that influence students’ academic outcomes. This analysis highlights the profound impact of textbook structure on students’ opportunities to learn (OTL). Variations in textbook sequencing reveal disparities in learning opportunities among students, underscoring an important aspect of educational inequality. Such disparities can significantly mediate the impact of students’ family backgrounds on their academic outcomes (Schmidt et al., 2015). Addressing these inconsistencies in textbook design is crucial for ensuring equitable educational experiences and improving academic achievement across diverse student populations. In conclusion, the sequencing of content topics and materials in textbooks is a crucial factor that influences students’ learning opportunities in the classroom. By aligning with cognitive development stages, supporting motivation, and providing a logical progression of topics, well-sequenced textbooks can enhance students’ understanding, retention, and application of knowledge. Future research and textbook development should continue to explore and implement effective sequencing strategies to maximize educational opportunities. 106 9.4 Strengths, Limitations and Suggestions for Future Studies 9.4.1 Strengths The use of graphical analyses in this study to illustrate the sequences of textbook components and content topics proved to be a powerful tool in understanding textbook structures. This methodological approach is invaluable for educators and researchers seeking to understand and improve the effectiveness of educational materials. Additionally, the application of Markov chain analysis and model-based analysis firstly offered an approach for statistically measuring the sequencing features of textbooks. These statistical methods allowed for a detailed examination of transition probabilities between different components and difficulty level of topics, providing quantitative insights into the structure and flow of textbooks. This approach enabled the study in this field to move beyond graphical analysis, offering a deeper understanding of how textbooks are organized. More important, this provides a quantitative indicator for future data analysis if researcher would like to explore the relationship between the sequencing features and academic outcome. Finally, the most important contribution of this study is not merely providing a statistical method based on the current coding procedure, but rather offering a general approach to analyzing a textbook’s sequencing features based on researchers’ specific needs. For example, instead of focusing on the motivational materials covered in textbooks, some scholars might be more interested in the graphs used in the textbook. In such cases, they can assign a category to pages containing graphs. Once they have established their own categories for textbook pages, they can apply the approaches used in this study to explore their own sequencing features. This flexibility allows researchers to attach the analysis to their specific interests and investigate 107 various aspects of textbook structure and understand different learning opportunities provided by the textbooks. 9.4.2 Limitations and Suggestions for Future Studies While the study provides valuable insights, it is essential to acknowledge its limitations. The research focused on 31 Algebra textbooks, which may not represent the full range of textbooks used across the country. Future research should expand the scope to include more subjects and grade levels to provide a more comprehensive understanding of textbook design and its impact on learning. Besides, this study provided a statistical way to measure the sequencing features; however, it did not illustrate what these differing probabilities mean in practical terms. For example, while the analysis quantified the transition probabilities, it did not establish a threshold to indicate the extent of the differences’ impact on learning outcomes, similar to how correlation coefficients have thresholds to indicate the strength of relationships. Further research is needed to interpret these probabilities meaningfully with students’ learning outcome dataset. Additionally, the current study used the original number of pages as time points in the analysis, which means the length of the pages varies. Future research could standardize the mapping display into a more practical format, such as converting it to the typical 180 instructional days academic year. Lastly, as mentioned many times in this paper, further research could explore the impact of these sequencing patterns on student learning outcomes, providing additional insights into the effectiveness of different textbook designs. Investigating how specific sequencing strategies influence students’ understanding of mathematical concepts could inform the development of textbooks that better support student learning. Moreover, longitudinal studies could examine how 108 the use of different textbook structures affects students’ academic performance over time, considering various demographic and socio-economic backgrounds. 109 BIBLIOGRAPHY Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman. Bandura, A. (2002). Social cognitive theory in cultural context. Applied Psychology: An International Review, 51(2), 269–290. Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit Theories of Intelligence Predict Achievement Across an Adolescent Transition: A Longitudinal Study and an Intervention. Child Development, 78(1), 246–263. Boaler, J. (2002). Experiencing School Mathematics: Traditional and Reform Approaches to Teaching and Their Impact on Student Learning. Lawrence Erlbaum Associates. Borgonovi, F., & Pokropek, A. (2019). Seeing is believing: Task-exposure specificity and the development of mathematics self-efficacy evaluations. Journal of Educational Psychology, 111(2), 268–283. Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How People Learn: Brain, Mind, Experience, and School. National Academy Press. Bruner, J. S. (1995). On learning mathematics. Mathematics Teacher, 88(4), 330-335. Caprara, G. V., Fida, R., Vecchione, M., Del Bove, G., Vecchio, G. M., Barbaranelli, C., & Bandura, A. (2008). Longitudinal analysis of the role of perceived self-efficacy for self- regulated learning in academic continuance and achievement. Journal of Educational Psychology, 100(3), 525–534. Cogan, L. S., Schmidt, W. H., & Wiley, D. E. (2001). Who takes what math and in which track? Using TIMSS to characterize U.S. students’ eighth-grade mathematics learning opportunities. Educational Evaluation and Policy Analysis, 23(4), 323-341. Cogan, L. S., & Schmidt, W. H. (2015). The Concept of Opportunity to Learn (OTL) in International Comparisons of Education. In K. Stacey & R. Turner (Eds.), Assessing mathematical literacy (pp. 207–216). Springer. Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95(2), 256–273. Dweck, C. S. (2002). The development of ability conceptions. In J. Eccles & A. Wigfield (Eds.), Development of achievement motivation (pp. 57-88). San Diego, CA: Academic Press. Dweck, C. S. (2006). Mindset: The new psychology of success. Random House. Elliott, E. S., & Dweck, C. S. (1988). Goals: An approach to motivation and achievement. Journal of Personality and Social Psychology, 54(1), 5–12. 110 Ferrini-Mundy, J., Burrill, G., & Schmidt, W. H. (2007). Building teacher capacity for implementing curricular coherence: mathematics teacher professional development tasks. Journal of Mathematics Teacher Education, 10, 311-324. Hackett, G., & Betz, N. E. (1989). An exploration of the mathematics self-efficacy/mathematics performance correspondence. Journal for Research in Mathematics Education, 20(3), 261–273. Lee, Y. & Wan, D. (2022). Disciplinary emphasis and coherence of integrated science textbooks: a case study from mainland China. International Journal of Science Education, 44(1), 156-177. Lee, J. (2009). Universals and specifics of math self-concept, math self-efficacy, and math anxiety across 41 PISA 2003 participating countries. Learning and Individual Differences, 19, 355-365. Muenks, K., & Miele, D. B. (2017). Students’ thinking about effort and ability: The role of developmental, contextual, and individual difference factors. Review of Educational Research, 87(4), 707–735. Muenks, K., Wigfield, A., & Eccles, J. S. (2018). I can do this! The development and calibration of children’s expectations for success and competence beliefs. Developmental Review, 48, 24–39. OECD. (2013a). PISA 2012 results: Ready to learn. Students’ engagement, drive and self-beliefs. Paris, France: OECD Publishing. OECD. (2014). PISA 2012 technical report. OECD Publishing. OECD. (2023). PISA 2022 Assessment and Analytical Framework, PISA, OECD Publishing, Paris. Paunesku, D., Walton, G. M., Romero, C., Smith, E. N., Yeager, D. S., & Dweck, C. S. (2015). Mind-set interventions are a scalable treatment for academic underachievement. Psychological science, 26(6), 784–793. Pajares, F. (1997). Current directions in self-efficacy research. In M. Maehr & P. R. Pintrich (Eds.), Advances in motivation and achievement (Vol. 10, pp. 1–49). Greenwich, CT: JAI Press. Rezat, S., Fan, L. & Pepin, B. (2021). Mathematics textbooks and curriculum resources as instruments for change. ZDM: Mathematics Education, 53(6), 1189-1206. Schmidt, W. H. (2004). A vision for mathematics. Educational Leadership, 37(5), 525-559. 111 Schmidt, W. H., Wang, H. C., & McKnight, C. C. (2005). Curriculum coherence: an examination of US mathematics and science content standards from an international perspective. Journal of Curriculum Studies, 37(5), 525-559. Schmidt, W. H. & Prawat, R. S. (2006). Curriculum coherence and national control of education: Issue or non-issue? Journal of Curriculum Studies, 38(6), 641-658. Schmidt, W. H. (2008). What’s missing from math standards? Focus, rigor, and coherence. American Educator, (Spring), 22-24. Schmidt, W. H. & Houang, R. T. (2012). Curriculum coherence and the Common Core State Standards for Mathematics. Educational Researcher, 41, 103-130. Schmidt, W. H. (2012). At the precipice: The story of mathematics education in the United States. Peabody Journal of Education, 87(1), 133-156. Schmidt, W. H., & Burroughs, N. A. (2013). How the common core boosts quality and equality. Educational Leadership, 70(4), 54-58. Schmidt, W. H., Burroughs, N. A., Zoido, P., & Houang, R. T. (2015). The role of schooling in perpetuating educational inequality: An international perspective. Educational Researcher, 44(7), 371–386. Schmidt, W., Houang, R. T., Sullivan, W. F., & Cogan, L. S. (2022). “When practice meets policy in mathematics education: A 19 country/jurisdiction case study”, OECD Education Working Papers, No. 268, OECD Publishing, Paris. Stajkovic, A. D., Bandura, A., Locke, E. A., Lee, D., & Sergent, K. (2018). Test of three conceptual models of influence of the big five personality traits and self-efficacy on academic performance: A meta-analytic path-analysis. Personality and Individual Differences, 120, 238–245. Usher, E. L., & Pajares, F. (2008). Sources of self-efficacy in school: Critical review of the literature and future directions. Review of Educational Research, 78(4), 751–796. Valverde, G. A., Bianchi, L. J., Wolfe, R. G., Schmidt, W. H., & Houang, R. T. (2002). According to the book: Using TIMSS to investigate the translation of policy into practice through the world of textbook. Dordrecht, The Netherlands: Kluwer. Wan, D. & Lee, Y. (2022). Coherence of topics from middle-school integrated science textbooks from Taiwan and Korea. International Journal of Science and Mathematics Education, 20, 881–899 (2022). Yeager, D. S., Hanselman, P., Walton, G. M., Murray, J. S., Crosnoe, R., Muller, C. Tipton, E., Schneider, B., Hulleman, C. S., Hinojosa, C. P., Paunesku, D., Romero, C., Flint, K., Roberts, A., Trott, J., Iachan, R., Buontempo, J., Yang, S. M., Carvalho, C. M., … 112 Dweck, C. S. (2019). A national experiment reveals where a growth mindset improves achievement. Nature, 573, 364–369. Zimmerman, B. J., Bandura, A., & Martinez-Pons, M. (1992). Self-motivation for academic attainment: The role of self-efficacy beliefs and personal goal setting. American Educational Research Journal, 29(3), 663–676. 113 APPENDIX A. MATHEMATICS CURRICULUM DOCUMENT ANALYSIS (MCDA) CONTENT FRAMEWORK Table A1 Mathematics Curriculum Document Analysis (MCDA) Content Framework 1.1 Whole Number 1.1.1 Meaning (place value, ordering, comparison) 1.1.2 Operations (meaning and computations) 1.1.3 Properties of operations (order of operation, 1 Number 2 System 3 Space and Shape 4 Change and Relationships 1.2 Fractions and Decimals 1.3 Number Sense and Estimation 2.1 Number Systems 2.2 Other Number Concepts 3.1 Position, Visualization and Shape 3.2 Symmetry, Congruence and Similarity 4.1 Algebra Foundations relationship among operations) 1.2.1 Common fractions 1.2.2 Decimal fractions & percentages 1.2.3 Properties and relationships of common & decimal fractions Integers, negative numbers & their properties 1.3.1 Measurement units, estimation & errors 1.3.2 Rounding & significant figures 1.3.3 Estimating computations 1.3.4 Exponents & orders of magnitude 2.1.1 2.1.2 Rational numbers & their properties 2.1.3 Real numbers, their subsets & properties 2.1.4 Complex numbers 2.2.1 Simple number patterns and sequences 2.2.2 Binary arithmetic &/or other number bases 2.2.3 Roots, radicals and complex numbers 2.2.4 Combinatorics (permutations and combinations) 2.2.5 Computational thinking: Algorithmic mathematics & computer simulations 2.2.6 Computer coding (including both formal and informal (pseudocode) syntax) 3.1.1 2-D Geometry: Basics (points, lines, segments, rays, angles) 3.1.2 2-D Geometry: Polygons & circles (formulas, properties, perimeter, area) 3.1.3 3-D Geometry (shapes, volume, surfaces, cross- sections) 3.1.4 Co-ordinate geometry (analytical geometry) 3.1.5 Trigonometry of right-angled triangles including the Pythagorean Theorem 3.1.6 Vectors and matrices 3.1.7 Geometric approximation for irregular shapes 3.2.1 Symmetry 3.2.2 Transformations (including geometric patterns) 3.2.3 Congruence & similarity 1.1.1 Rates and ratios 1.1.2 Proportionality 114 Table A1 (cont’d) 4.2 Beginning Algebra 4.3 Algebra 4.4 Change 5.1 Descriptive Statistics 5.2 Probability Distributions 5.3 Statistical Inference 4.2.1 Algebraic sequences and patterns 4.2.2 Expressions 4.2.3 Simple linear equations 4.2.4 Slope and intercept 4.3.1 Linear equations and inequalities 4.3.2 Trigonometric equations and identities 4.3.3 Other equations and inequalities (quadratics, polynomials, including factorization and expansion) 4.3.4 Linear functions 4.3.5 Exponential functions 4.3.6 Other non-linear functions 4.4.1 Infinite processes (e.g. sequence, series, limits and convergence) 4.4.2 Calculus and analysis 4.4.3 Linear, non-linear, and exponential for modelling growth and change 5.1.1 Mean, mode, median, variance, etc. 5.1.2 Displays of distributions 5.2.1 Definition of discrete probability and related theorems 5.2.2 Conditional probability and independent events 5.2.3 Bayes Theorem 5.2.4 Discrete and continuous random variables and their distributions 5.3.1 Populations and their parameters 5.3.2 Sampling from the population/random sampling 5.3.3 Estimation of parameters (e.g. mean, variance) 5.3.4 Sampling distributions (standard errors, bias) 5.3.5 Confidence intervals 5.3.6 Hypothesis testing 5.3.7 Definition of correlation coefficient 5.3.8 Relationship among categorical variables (contingency tables) 5.3.9 Relationship involving continuous variables (regression) 5.3.10 Relationship involving categorical and continuous variables (ANOVA) 5.3.11 History of Mathematics as a Human Activity 115 5 Statistics, Probability and Data APPENDIX B. SEQUENCING MAPPING DISPLAY OF THE THREE MAJOR COMPONENTS Figure B1 Sequencing Mapping Display of Three Major Components in 18 Algebra I Textbooks 116 Figure B2 Sequencing Mapping Display of Three Major Components in 13 Algebra II Textbooks 117 APPENDIX C. SEQUENCING MAPPING DISPLAY OF CONTENT TOPICS Figure C1 Sequencing Mapping Display of Content Topics in 18 Algebra I Textbooks 118 Figure C1 (cont’d) 119 Figure C1 (cont’d) 120 Figure C1 (cont’d) 121 Figure C2 Sequencing Mapping Display of Content Topics in 13 Algebra II Textbooks 122 Figure C2 (cont’d) 123 Figure C2 (cont’d) 124 APPENDIX D. SEQUENCING MAPPING DISPLAY OF MOTIVATIONAL MATERIALS Figure D1 Sequencing Mapping Display of Motivational Materials in 18 Algebra I Textbooks 125 Figure D1 (cont’d) 126 Figure D1 (cont’d) 127 Figure D2 Sequencing Mapping Display of Motivational Materials in 13 Algebra II Textbooks 128 Figure D2 (cont’d) 129