CHARACTERIZATION OF MANUAL EXPLORATORY BEHAVIORS IN EARLY CHILDHOOD By Priya Prakashbhai Patel A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Kinesiology – Doctor of Philosophy 2022 ABSTRACT CHARACTERIZATION OF MANUAL EXPLORATORY BEHAVIORS IN EARLY CHILDHOOD By Priya Prakashbhai Patel Manual exploratory behaviors observed during early childhood have critical functional and clinical role in the motor development of a child (Lockman & Kahrs, 2017). This dissertation is aimed to (1) address the challenges faced in the quantitative analysis of these behaviors, (2) conduct quantitative analysis of two important manual exploratory behaviors, (3) extend the current knowledge on the effects of age and object properties on these behaviors beyond infancy by assessing them in preschoolers. In Study 1, a machine learning (ML) -based automated classification system was developed as a proof-of-concept for the classification of manual exploratory behaviors that address the challenges encountered in the quantitative analysis of these behaviors. This system was developed using data from adult participants and it can currently classify three manual exploratory behaviors namely- rotation, throwing and fingering with substantially higher accuracy than chance level (average accuracy = 85.0 + 4.16%). Based on these findings, ML - based approach appears to be both- a feasible and a scalable alternative to conventional video coding for identifying the manual exploratory behaviors on time series; thereby, facilitating their quantitative assessment. In Study 2, quantitative assessment of two important manual exploratory behaviors- rotation and throwing was conducted along with the assessment of ML classifiers on data from children (3 – 5 years old). The ML classifiers showed substantial decrease in performance owing to differences in movements between children and adults as well as technical difficulties. Rotation behaviors became more variable and faster with increasing age while the characteristics of throwing behaviors were inconclusive of developmental differences across the three ages. In Study 3, the effects of age and three object properties (size, shape and texture) were assessed on the qualitative characteristics of manual exploratory behaviors in children (3 – 5 years old). Manual exploration of objects was driven at different levels by age and object properties in preschoolers. In terms of age, throwing behaviors were more common in the 3-year group while rotational behaviors in the 5-year group. In terms of the three object properties, object size and shape directed child’s hand preference in reaching objects while object size and texture influenced their manual exploratory behaviors. In addition, object texture was found to mainly influence child’s first interactions with the objects as the squeezing and fingering behaviors occurred more often during the first interactions with the objects. The findings suggest that the dynamic interplay between learning to perceive object properties and manually exploring them continues to develop and adapt beyond infancy. In summary, manual exploratory behaviors, similar to other motor behaviors, are influenced by different individual, task and environment factors. These effects continue beyond infancy and throughout early childhood. A thorough qualitative and quantitative assessment is required to fully understand their functional and clinical role in early childhood. For this, ML - based approach is recommended to address the challenges in their quantitative analysis and to facilitate the overall scope of investigating these behaviors. This dissertation is dedicated to my husband, Harsh Pandya, whose unwavering support and confidence in my abilities have led me to pursue my dream of becoming a researcher. Thank you for your unconditional love and support! iv ACKNOWLEDGEMENTS Firstly, I would like to thank my advisor, Dr. Mei-Hua Lee, whose vision and expertise in motor development field instilled the foundations of becoming a competent researcher in me. She has been my toughest critic that has driven me to strive for quality and perfection. Thank you, Mei, for your support, mentoring and helping me become an independent researcher. I would also like to thank members of my committee- Dr. Rajiv Ranganathan, Dr. Florian Kagerer and Dr. Subir Biswas, for their guidance, cooperation and encouragement on this dissertation work. I would specially like to thank Dr. Rajiv Ranganathan for extending his guidance outside of the scope of this dissertation and throughout my doctoral journey. Special thanks to my husband, Harsh Pandya, who has supported me by all means throughout my doctoral journey, right from giving me mental support and care to being my engineering consultant and proofreading countless drafts and documents. Thank you for believing in me that helped me pursue my dreams. I would also like to thank my parents, family and friends- Aimee Tomlinson, Yuemei Lu for their continued love and support. My gratitude extends to all members of the Sensorimotor Development lab and MOTRE lab at Michigan State University for their help and contributions. Last but not the least, I would like to shout out to all the participants and their parents who participated in this research. Special thanks to all the parents who wore an additional experimenter’s hat on top of their existing countless hats during the pandemic time in order to help me with the data collection process. This dissertation would not have been possible without them. v TABLE OF CONTENTS LIST OF TABLES ....................................................................................................................... viii LIST OF FIGURES ....................................................................................................................... ix CHAPTER 1. INTRODUCTION ................................................................................................... 1 Focus of this dissertation ............................................................................................................. 4 CHAPTER 2. LITERATURE REVIEW ........................................................................................ 6 Developmental course of exploration in childhood .................................................................... 6 Significance of manual exploratory behaviors in early childhood .............................................. 7 Functional significance: Tool use development in early childhood ............................................ 8 State of the art techniques and challenges in conducting quantitative assessment of manual exploratory behaviors ................................................................................................................ 10 Quantitative analysis of manual exploratory behaviors ............................................................ 12 Effect of object properties on manual exploratory behaviors ................................................... 14 Clinical significance: Manual exploratory behaviors in Autism Spectrum Disorder (ASD) .... 15 Summary ................................................................................................................................... 17 CHAPTER 3. AUTOMATED CLASSIFICATION OF MANUAL EXPLORATORY BEHAVIORS USING SMART OBJECTS AND MACHINE LEARNING ............................... 18 Abstract ..................................................................................................................................... 18 Introduction ............................................................................................................................... 19 Method ...................................................................................................................................... 23 Participants ............................................................................................................................ 23 Apparatus ............................................................................................................................... 23 Protocol.................................................................................................................................. 25 Data analysis ............................................................................................................................. 26 Classifier training .................................................................................................................. 27 Classifier testing and validation ............................................................................................ 31 Results ....................................................................................................................................... 32 5-fold cross validation method .............................................................................................. 34 70-30% data split ................................................................................................................... 34 Behavior classification of sequence trials ............................................................................. 35 Discussion ................................................................................................................................. 36 CHAPTER 4. EFFECT OF AGE ON THE QUANTITATIVE CHARACTERISTICS OF ROTATION AND THROWING BEHAVIORS IN PRESCHOOLERS ..................................... 42 Abstract ..................................................................................................................................... 42 Introduction ............................................................................................................................... 43 Method ...................................................................................................................................... 46 Participants ............................................................................................................................ 46 Apparatus ............................................................................................................................... 46 vi Protocol.................................................................................................................................. 47 Data Analysis ............................................................................................................................ 48 ML -based classification........................................................................................................ 48 Quantitative analysis.............................................................................................................. 48 Statistical analysis ................................................................................................................. 50 Results ....................................................................................................................................... 50 Descriptive measures ............................................................................................................. 50 ML -based classification performance .................................................................................. 51 Quantitative analysis.............................................................................................................. 52 Discussion ................................................................................................................................. 55 CHAPTER 5. EFFECT OF AGE AND OBJECT PROPERTIES ON THE CHARACTERISTICS OF MANUAL EXPLORATORY BEHAVIORS IN EARLY CHILDHOOD ............................ 60 Abstract ..................................................................................................................................... 60 Introduction ............................................................................................................................... 61 Method ...................................................................................................................................... 63 Participants ............................................................................................................................ 63 Apparatus ............................................................................................................................... 64 Protocol.................................................................................................................................. 64 Data Analysis ............................................................................................................................ 67 Qualitative analysis................................................................................................................ 67 Statistical analysis ................................................................................................................. 68 Results ....................................................................................................................................... 69 Descriptive measures ............................................................................................................. 69 Hand preference during reaching an object: frequency percentage of 1 hand vs. 2 hands ... 69 Initial interactions with the object: frequency percentage of first three behaviors in a trial . 71 Manual exploratory behaviors performed with each object: frequency percentage of total behavior category during each trial ....................................................................................... 74 Discussion ................................................................................................................................. 77 CHAPTER 6. GENERAL DISCUSSION .................................................................................... 82 ML -based automated classification system.............................................................................. 82 Characterization of Manual exploratory behaviors ................................................................... 84 Future directions........................................................................................................................ 87 Conclusions ............................................................................................................................... 87 BIBLIOGRAPHY ......................................................................................................................... 89 vii LIST OF TABLES Table 3.1. List of formulae used to calculate five statistical features for classifier training…….30 Table 3.2. List of evaluation metrics with their formulae………………………………………..31 Table 3.3. Accuracies of all 22 trained models across the three different testing methods……...33 Table 4.1. Mean (SD) of quantitative characteristics of rotation and throwing behaviors as a function of age…………………………………………………………………………………...53 Table 5.1. Summary of manual exploratory behaviors divided into operational behavior categories………………………………………………………………………………………...68 Table 5.2. Summary results of the repeated measures of analysis of variance on the three behavior categories as a function of age, object size, texture and shape………………………...74 viii LIST OF FIGURES Figure 3.1. (a) Four objects made of firm Styrofoam in two different shapes and two different sizes- 5 cm hard ball, 5 cm hard cube, 10 cm hard cube, 10 cm hard ball. (b) 3-space mini– Bluetooth Inertial Measurement Units (IMU) 3 cm x 3 cm x 1.3 cm in size, weighed 9 grams. (c) Every object had a slit like cavity to fit the sensor in it. (d) Sensor was securely placed in the cavity of the object and covered with a custom-made red colored sleeve to hide sensor placement from the participants………………………………………………………………….24 Figure 3.2. (a) One remote experimental session was conducted per participant. Experimenter communicated with the participant via video call and controlled the trials on the participant’s computer using TeamViewer. (b) Experimenter and participant’s computer screenshots demonstrating what they saw during the experimental session. Since experimenter-controlled sensors on participant’s end via TeamViewer and communicated with the participant via video call, the left side of the experimenter’s screen displays the software used to collect sensor data and right side displays the video call. Participants only saw experimenters during the session, so their screen displays video calls with the experimenter. (c) Every participant performed 5 blocks of 15 trials each. During each block, participants performed the three behaviors 5 times (one behavior per trial) in a random order. Participants also performed one sequence trial at the end of each block in which they performed all three behaviors sequentially instead of one behavior per trial. Sequence trials were used for testing purposes………………………………26 Figure 3.3. (a) Various classification algorithms were trained using a 4-step process: data pre- processing, segmentation, feature extraction and training classifiers. (b) Trimming of sample data from a throwing trial. The start and end of throwing trials had idle data which was collected to avoid missing the actual throw due to potential internet lag. So, the start and end of throwing trials were trimmed by applying an operational threshold of 0.3 m/s2. (c) Sample data depicting implementation of segmentation process. Segmentation is a process of dividing time series into windows of data that can be represented by a common characteristic. Sliding window approach was used in which the time series was divided into 1 second windows with 50% overlap between two consecutive windows………………………………………………..27 Figure 3.4. SVM Quadratic, SVM Medium Gaussian and Narrow Neural Network were the best models that consistently showed higher performance than the other 19 models across all three testing methods They showed high accuracy, precision, recall and f1 scores on 5-fold cross validation and 70-30% data split method. Their performance metrics were moderate on the behavior classification of sequence trials. (a) Accuracy of highest three models in the three testing methods. (b) Average precision of models across the three behaviors. (c) Average recall of the models across the three behaviors. (d) Average f1 scores of the models across the three behaviors. Error bars represent one standard deviation………………………………………….36 Figure 4.1. Study 2 protocol. There were 5 sequence trials performed in one block per participant. In each sequence trial, participants performed three manual exploratory behaviors (rotation, throwing and fingering) one after another in a random sequence…………………….48 ix Figure 4.2. (a) Precision (%) (b) recall (%) and (c) f1-scores of the three ML classifiers for classifying fingering, rotation and throwing behaviors. Classifiers showed substantially poor performance for classifying throwing behaviors compared to the other two behaviors for which the performances were moderate. This mainly accounted for their overall performance reduction........................................................................................................................................52 Figure 4.3. (a) Average rotational variability as a function of age used for statistical analysis. There was no main effect of age on the rotational variability across three ages. (b) Frequency distribution of rotational variability plotted for the three age groups. Rotations with low variability were highest in the 3-year group while those with high variability were found more in the 5-year group. (c) Average dimensionless angular jerk as a function of age used for statistical analysis. There was no main effect of age on the angular jerk, but average angular jerk was highest in the 5-year group and lowest in the 3-year group. (d) Average dimensionless linear jerk as a function of age used for statistical analysis. There was no main effect of age on the angular jerk. Error bars represent one standard deviation……....…...54 Figure 5.1. (a) Nine experimental objects given to participants one at a time in a random order. Eight objects were a combination of two different sizes, texture and shape- 10 cm hard cube, 10 cm hard ball, 5 cm hard ball, 5 cm hard cube, 10 cm soft cube, 10 cm soft ball, 5 cm soft ball and 5 cm soft cube. Ninth object was the sensor itself. (b) Protocol for study 3. There were 2 blocks per participant, 9 trials per block and 1 object per trial. Participants freely explored each object twice across two blocks……………………………………………………………..66 Figure 5.2. Average hand frequency (%) across three age groups used for statistical analysis as a function of (a) object size (b) object shape and (c) object texture. There was a significant effect of object size and shape on the hand preference of children whereas no effect of texture on their 1hand vs. 2hand reach pattern. Children used 1 hand reach more often for small and ball shaped objects and 2 hands reach for large and cube shaped objects. Error bars represent one standard deviation…………………….……………………………………………………..71 Figure 5.3. Time-point at which behaviors in a trial can be considered as initial interactions was identified by examining the behavioral category trends during different times in a trial against the overall behavioral category trends of a trial across three age groups. Behavior category frequency (%) plotted across three age groups for (a) first behavior in trial, (b) first two behaviors in a trial, (c) first three behaviors in a trial, (d) first four behaviors in a trial, (e) first five behaviors in a trial and (f) total behaviors in a trial. Behavior category trends started to align with the overall trends after the third behavior in a trial for all three age groups, so first three behaviors were considered as part of the initial interactions with an object.………...72 Figure 5.4. Average behavior category frequency (%) during initial interactions across three age groups used for statistical analysis as a function of (a) size (b) texture and (c) shape. There was a significant effect of object size and texture on the initial interactions of children with objects. Average behavior category frequency (%) during all interactions with an object across three age groups used for statistical analysis as a function of (d) size (e) texture and (f) shape. Similar to initial interactions, there was a significant effect of object size and texture on the total x exploratory behaviors of children. There was an age effect on the transportation activities during initial and total interactions where children in the 3-year group performed more throwing behaviors than older children. The average frequency (%) of finger activities was substantially higher during initial interactions compared to total exploration across all ages. The average frequency (%) of transportation activities was substantially lower during initial interactions compared to total exploration across all ages. Error bars represent one standard deviation……………………………………………………………………………………...…..77 xi CHAPTER 1. INTRODUCTION Every organism understands, adapts, and sustains itself through constant exploration of its environment. This exploratory behavior of an organism and its importance in their development had inspired pioneer biologists Darwin and psychologists G. Stanley Hall and continue to do so in the modern age of developmental psychology (Gibson, 1988). This importance of environmental exploration is not limited to developmental psychology but extends into one’s motor development. For instance, newborn babies who have the basic instinct to consider everything graspable as food begin to understand the unique purpose of objects by first year of life. This understanding of the environment and objects within it develops from their constant exploration that occurs due to different factors such as motivation (White, 1959), novelty (Hutt, 1970), and curiosity (Berlyne, 1966). One such critical exploratory behavior observed in humans is the manual exploration of objects in infants and children that help them to learn to perceive different object properties and understand object function. Manual exploration of objects is one of the most critical behaviors observed during early childhood that lay the foundation of an important motor skill- “tool use” (Goldfield, 1995; Lockman, 2000; Thelen, 1981). In order to understand the function of an object and use it efficiently, perception of object properties such as shape, size, texture, object segmentation, etc. is required. Infants do not have the innate ability to perceive such object properties (Gibson, 1988), but they learn to perceive them through different manual exploratory behaviors. After manually exploring objects for a substantial amount of time during early childhood they learn the affordances offered by different objects. Such explorations shape motor behavior from seemingly random actions with objects (banging, mouthing, etc.) to cognitively directed “functional” activities such as talking into a toy telephone (Gibson, 1988; Goldfield, 1995; Lockman, 2000). 1 Manual exploratory behaviors continue beyond infancy throughout the early childhood period. These behaviors may be perceived as purposeful play in early childhood, but they are a part of exploratory activities since the perceptual abilities are still developing (Aslin & Smith, 1988). Assessment of these behaviors in early childhood is vital given they continue to play substantial functional and clinical role beyond infancy. Functionally, these behaviors are observed in toddlers and preschoolers as part of the development of different tool use skills in them. For instance, ‘banging’ movements of the arms, a type of manual exploratory behavior observed in late infancy and toddlerhood, have been linked to the development of important tool use skill- ‘hammering’ later in development (Kahrs et al., 2012, 2013). Clinically, infants and toddlers at high risk for autism spectrum disorder (ASD) are found to have atypical manual exploratory behaviors (Kaur et al., 2015; Ozonoff et al., 2008). A thorough qualitative and quantitative assessment of these behaviors is important throughout early childhood to fully understand their functional and clinical significance in motor development. However, manual exploratory behaviors have been focused in infancy and lack characterization in early childhood. The characteristics of manual exploratory behaviors are often investigated in motor development using qualitative means of analysis such as video coding, online coding, etc. (Corbetta & Thelen, 1996; Lee et al., 2006; Newell et al., 1989; Ruff, 1984). Such qualitative analysis uses frequency, duration, or order of occurrence to characterize these behaviors. While such qualitative assessment is indicative of developmental trajectory of these behaviors, it may not quantify the underlying movements. Quantifying the movement of a behavior using kinematic variables such as velocity profile, smoothness, etc. can help to detect subtle movement signatures that are otherwise not detectable on qualitative assessment. For instance, qualitative analysis of banging an object describes its frequency/duration but quantitative analysis provides information on its speed and trajectory. Functionally, such movement characteristics provide 2 insights into the developmental trajectory of motor behaviors, while clinically they provide reliable measurements of atypical movements observed in ASD. However, with the exceptions of a few behaviors (e.g., banging) (Kahrs et al., 2012, 2013, 2014; Lockman & Kahrs, 2017), quantitative analysis of most manual exploratory behaviors is yet to be conducted. Quantifying manual exploratory behaviors involves two main challenges in the process of assessing the type of movements during these behaviors: (i) reliably classifying the type of movement, and (ii) performing this classification throughout a time series without requiring manual curation (curation is a process of labelling start and end times of a behavior on time series). Such classification of behaviors is required since a typical manual exploratory behavior study comprises of multiple different behaviors instead of a single behavior in a time series. For example, in studies assessing motor behaviors such as walking, reaching, etc., no curation is required since there is only a single behavior (walking, reaching, etc.) that needs to be analyzed; however, in manual exploratory behavior studies, there are multiple behaviors such as rotation, banging, throwing, etc. in a single trial. Therefore, the time series data needs to be classified for each of the different exploratory behaviors before calculating their quantitative properties using kinematic variables. Conventionally, such curation can be done using manual methods like video coding (Lee et al., 2011), but this might not be practical for these behaviors given their complex and variable nature. Owing to their highly variable nature wherein some behaviors can last for a very short duration, video coding them is difficult as well as require a time-consuming frame-by- frame analysis. Moreover, video coding method may also encounter the issue of subjectivity due to the complex nature of these behaviors. For the purpose of quantification, the ideal curation method is the one that can classify the entire time series data without manual intervention. Due to lack of a feasible method for such data curation, quantification of most manual exploratory behaviors is yet to be conducted. Thus, it is crucial to address the methodological challenges that 3 can facilitate the quantitative analysis of manual exploratory behaviors; thereby their systematic characterization in early childhood period. Focus of this dissertation The characteristics of manual exploratory behaviors in early childhood is a window to fully understanding their functional role in development of the tool use and clinical role in developmental delays. The first step towards conducting a thorough assessment of their characteristics is to address the methodological challenge faced in their quantitative analysis. Once a feasible method is available to curate the time series data for different manual exploratory behaviors then the movement analysis of these behaviors is possible. This in turn will facilitate a systematic characterization of the manual exploratory behaviors in early childhood under the effects of different individual, task and environmental constraints. Thus, this dissertation is aimed at understanding manual exploratory behaviors during early childhood through development of quantitative analysis methods. In study 1, we proposed a Machine Learning (ML) -based classification method to classify different manual exploratory behaviors on time series data obtained from sensors embedded in objects used for manual exploration. The primary objective of this study was to address the two challenges faced by researchers while conducting quantitative analysis of manual exploratory behaviors: (1) identify/classify type of behavior and (2) perform this classification throughout a time series data without manual curation. We trained different classifiers in order to classify three manual exploratory behaviors- rotation, throwing and fingering. We chose these behaviors as they: (1) are amongst the most commonly found behaviors in early childhood (Ruff, 1984), (2) are indicative of gross and fine motor skills in early childhood, and (3) have clinical importance in cases of ASD (Ozonoff et al., 2008). The data curated using this classifier method can then be used to conduct quantitative analysis of respective exploratory behaviors. 4 In study 2, we examined the effects of age on the quantitative characteristics of two functionally and clinically important manual exploratory behaviors- rotation and throwing. The ML -based classifier technique was also assessed in classifying these behaviors on time series data obtained from children. The primary objective of this study was to examine the effects of age on the quantitative characteristics of these behaviors as well as provide a roadmap of effectively and feasibly conduct quantitative analysis of manual exploratory behaviors in which behaviors are first classified on time series using ML -based classification method, followed by their quantitative assessment using kinematic variables. However, the ML -based classification method had low accuracy in classifying these behaviors. So, we used conventional video coding method to curate the behaviors on time series prior to their quantitative analysis. Rotation and throwing behaviors were quantitatively characterized in terms of their variability and fluency to assess age-related effects. In study 3, we qualitatively assessed the effects of age and object properties (size, shape and texture) on the characteristics of manual exploratory behaviors in early childhood. The primary objective of this study was to investigate the developmental changes in these behaviors under individual and different task constraints beyond infancy. Given that these behaviors continue beyond infancy in the form of purposeful play that eventually leads to the development of different tool use skills, it is important to assess developmental changes in them throughout early childhood. This study provided insights on how different individual and task constraints modulate manual exploratory behaviors in preschoolers. 5 CHAPTER 2. LITERATURE REVIEW Developmental course of exploration in childhood Exploration of the environment and objects within it is an important motor behavior for every organism to adapt and survive (Newell, 1986). These exploratory behaviors may often seem random and purposeless but they are accounted to have primary theoretical importance in child’s development (Piaget & Cook, 1952). Moreover, the reasons for the existence of these behaviors have been tied to different factors. From time to time, different factors such as motivation (White, 1959), novelty (Hutt, 1970), urge to know (Berlyne, 1966) have been proposed to explain what drives exploration of one’s environment and the objects within it. Such exploration of objects begins right after birth and continues to show a steady increase of exploratory behaviors until early childhood. These behaviors may exist as acts of curiosity, reactions to novelty, purposeful play or often times just as random acts of interaction. Changes are also observed in the way objects are explored wherein infants in early infancy mainly explore using mouth, waving arms or kicking movements. However, after achieving reaching and grasping motor skills, objects are actively explored using hands which is known as manual exploration of objects (Gibson, 1988). As infants grow into toddlers, preschoolers and so on, they physically grow, achieve different motor milestones, and their capabilities to explore different objects using different senses also expand (Gibson, 1988). Each exploration action leads to learning to perceive their environment and each instance of learning to perceive an object/environmental property influences their subsequent exploration action. Such dynamic interplay between exploratory actions and perception of environment aided by simultaneous individual growth and development continues throughout early childhood (operationally defined in the current context as up to 5 years of age). 6 Before discussing exploration of objects throughout early childhood, it is critical to understand how it appears in infancy. Eleanor Gibson (1988) suggested that exploration of objects have three phases in infancy. In phase 1 (1-4 months), neonates explore events in which young infants preferentially explore everything that is visually presented in moving form. Phase 2 (4-8 months) is the attention to affordances and distinctive features of objects in which infants actually engage in active exploration of the objects since they can reach and grasp objects. Lastly, the third phase (8-9 months onwards) is the ambulatory exploration in which infants attain the skill of locomotion and use this skill to optimize their manual exploration skills. This phase does not stop in infancy but continues throughout early childhood and optimizes the exploratory skills of toddlers, preschoolers in understanding their environment. Significance of manual exploratory behaviors in early childhood The dynamic interplay between manual exploratory behaviors and perception of object properties continues past infancy since the perceptual-motor system is still developing (Aslin & Smith, 1988; Libertus & Hauf, 2017). Moreover, learning to perceive different object properties emerge at different times during early childhood. For instance, perception of object size is found to guide infant’s behaviors as early as 6 months of age (Ruff, 1984) but perception of object weight appears around 11 months of age (Paulus & Hauf, 2011). As a result, the ability to perceive different object properties have different development trajectories. Thus, toddlers and preschoolers continue to exhibit manual exploratory behaviors while interacting with objects. These behaviors may seem to be purposeful play, but they are part of their exploratory actions since these actions are contributing to the maturation of the perceptual-motor system (Kahrs et al., 2012, 2013). Although manual exploratory behaviors continue throughout early childhood (upto five years of age), these behaviors are mainly investigated in infancy. Owing to their functional and 7 clinical significance, manual exploratory behaviors are often characterized for the effects of age, different object properties, different sensory information in infancy, prior sensory experiences, etc. during infancy (Corbetta et al., 2000; Corbetta & Snapp-Childs, 2009; Corbetta & Thelen, 1996; Fagard & Jacquet, 1996; Goldfield & Michel, 1986; Lee & Newell, 2013; Needham et al., 2017; Newell et al., 1989; Ruff, 1984). While these studies provide insights on how manual exploratory behaviors emerge, develop and evolve in infancy, our current knowledge on these behaviors need to be extended beyond infancy. In doing so, the current literature provides a strong framework for conducting a systematic qualitative and quantitative assessment of manual exploratory behaviors in early childhood. Therefore, in this literature review we will discuss: 1. Functional role of manual exploratory behaviors in early childhood. 2. State of the art techniques and challenges involved in conducting quantitative assessment. 3. Quantitative analysis of manual exploratory behaviors. 4. Effects of object properties on manual exploratory behaviors in infancy and the scope of assessing these effects in early childhood. 5. Clinical role of manual exploratory behaviors in early childhood. Functional significance: Tool use development in early childhood Manual exploration of objects during early childhood is an important step towards learning different tool use skills. The development of tool use from seemingly appearing random movements during infancy have been studied in motor development (Biryukova & Bril, 2012; Fitzpatrick et al., 2015; Kahrs et al., 2014; Lockman, 2000). Understanding of the manual exploratory behaviors beyond infancy provides an opportunity to understand their gradual progression into manual skilled action required for tool use. So far, there are three tool uses that have been most commonly researched with respect to manual exploratory behaviors- hammering, drumming and different grasps for picking up a tool. 8 Hammering is an important tool used in human history and the striking/banging action required to master this action is seen in infants as early as 5-6 months of age (Kahrs et al., 2013; Lockman, 2000). The striking and banging actions gradually change and emerge into the percussive tool use by the second year of life as evident from the quantitative characteristics (hand trajectory, velocity, etc.) of these behaviors (Kahrs et al., 2012, 2013). Quantitative assessment of hammering action in 1-3 year olds showed that there was an increase in use of wrist movement and decrease in shoulder and elbow movement with increase in age (Kahrs et al., 2014). Young children learn to differentiate between hammers with and without handles in the age group of 12, 18 and 24 months old (Fragaszy et al., 2016). Drumming is another important action that is studied in early childhood owing to its representation of complex self-organizing bimanual coordination. Skilled inter-limb coordination of two arms is required to accomplish this task. This first appears around 18 months, although it is not efficient or consistent (Brakke et al., 2007). Drumming action was assessed in children aged between 12–24 month olds by Brakke and colleagues (2007) using observation and quantitative analysis. The results of this study indicated that bimanual drumming was a preferred way of drumming in 2-year old children as compared to 1-year olds. Learning to pick up a tool such as a pen, spoon is an important aspect of tool use that provides insight into how children plan an action in order to use an object as a tool. Grasp strategies in picking up a spoon have been tested in infants, toddlers and children in several different situations (easy orientation of tool to body vs. difficult orientation of tool to body) (Keen et al., 2014; McCarty et al., 1999, 2001). As seen by McCarty et al. (1999, 2001), 9-month olds already attained a radial grip in an easy orientation of spoon with respect to their body. However, infants do not consistently use radial grips when a difficult orientation of spoon was provided. A study by Keen and colleagues (2014) tested different grips in 4 and 8 year olds. It 9 was found that while the 4-year olds used adult like grips certain times, there was high variability in their use of different grips. Moreover, they also showed a dominance of infantile grips such as ulnar grips. However, 8-year olds showed more mature and effective grips with almost no use of ulnar grips. The current literature provides insights on the specific aspects of tool use with their corresponding manual exploratory behaviors. However, there are other manual exploratory behaviors that do not transition directly into a specific tool use skill but rather contribute to the development of different motor skills. For example, rotation is an important manual exploratory behavior that contributes to the development of 3D perception of an object (Soska et al., 2010). This information helps children perceive the back of an object, do mental rotation of object which in turn plays important role in planning and executing different grip configurations (Jung et al., 2015). Thus, the developmental transitions of different manual exploratory behaviors are important to understand the developing perceptual-motor system which in turn is pivotal to the emergence and development of all tool use skills. State of the art techniques and challenges in conducting quantitative assessment of manual exploratory behaviors Conventionally, manual exploration of objects in infants and children is assessed using qualitative means such as video recording, online coding, observations of different behaviors (Corbetta & Thelen, 1996; Lee et al., 2006; Newell et al., 1989; Ruff, 1984). This involves analyzing them in terms of frequency, duration, order of their occurrences, etc. Qualitative assessment of these behaviors is rich in contextual information and provides information on the developmental trajectory of these behaviors. However, it generally does not involve assessing the underlying movements. For example, when a child rotates an object, qualitative analysis can assess the duration and frequency of the rotation, but it cannot measure the amount of variability 10 or smoothness in underlying rotational movements. Analyzing the characteristics of underlying movements is important to fully understand the functional and clinical significance of manual exploratory behaviors in motor development. Quantitative assessment with state-of-the-art measurement tools like inertial measurement units (IMUs), motion capture system can quantify movement characteristics using kinematic variables. Quantifying the movement of a behavior using kinematic variables such as velocity profile, smoothness, etc. can help to detect subtle movement signatures that are otherwise difficult to detect on video analysis. Researchers have started to use quantitative means of analysis for investigating some manual exploratory behaviors (Brakke et al., 2007; Jung et al., 2015; Kahrs et al., 2012, 2013, 2014; Lockman & Kahrs, 2017). However, these exploratory behaviors are the ones that are directly linked to specific tool use skills. Most manual exploratory behaviors are yet to be assessed using quantitative means of analysis. Quantitative assessment of manual exploratory behaviors is not as straightforward as it is for other motor behaviors like reaching or walking. A typical trial in manual exploration study comprises of multiple behaviors instead of one behavior per trial. Thus, before analyzing the kinematic variables of these behaviors, time series data needs to be “curated” in which the start and end times of each behavior is identified. Conventionally, such curation can be done using video coding methods (Lee, Ranganathan, & Newell, 2011), but it is not a practical option for manual exploratory behaviors. Manual exploratory behavior is an umbrella term comprising of a wide variety of different behaviors that differ in terms of their type and duration of occurrences. This complex and variable nature of these behaviors can make manual video coding difficult, time-consuming and require frame-by-frame analysis that reduces the overall scope of a study. This makes the quantitative analysis of manual exploratory behaviors difficult. 11 A possible solution to facilitate the quantitative assessment of manual exploratory behaviors is by using machine learning (ML) to automatize the curation process. In this approach, ML -based classifiers are used to classify the time series according to type of behaviors. In this approach, the ML algorithms are first trained to associate a set of unique features to respective behaviors. The trained algorithms then identify these behaviors on new time series using these unique features. Such methods are increasingly used in the classification of spontaneous movements in infants that serve as early predictors of neuro-developmental deficits in early infancy in infants (Goodfellow et al., 2018; Ihlen et al., 2019; Shin et al., 2022; B. A. Smith et al., 2015) as well as motor function in stroke patients (Kim et al., 2021). There have been similar accounts of using ML -based approach in identifying and classifying different human activities (Anguita et al., 2013; Debes et al., 2016; Ravi et al., 2005; Ronao & Cho, 2016; Xu et al., 2013) and gait analysis (Ding & Fan, 2014; Kwolek & Kepski, 2014; Mao et al., 2017; Tunca et al., 2017). Similar ML -based approach can be used to classify manual exploratory behaviors on time series which can then facilitate their quantitative assessment. Quantitative analysis of manual exploratory behaviors As described earlier, manual exploratory behaviors are mainly assessed qualitatively owing to the challenges involved in their quantitative assessment. Since quantitative assessment can quantify the characteristics of underlying movements, they provide insights on the developmental changes occurring in the movements of the exploratory behaviors with the development of perceptual-motor system. So quantitative assessment is mainly done for those manual exploratory behaviors that transition into a specific tool use skill in order to assess their developmental course. For example, transition of banging behaviors to hammering action. However, there are other manual exploratory behaviors that do not transition directly into specific skilled action but contribute to the development of the perceptual-motor system. For 12 example, rotational behaviors contribute to learning to perceive 3D nature of object, mental rotation and flipping/turning actions. Thus, a systematic qualitative and quantitative assessment is required for most manual exploratory behaviors. Two of the most commonly assessed exploratory behaviors in terms of kinematics are- banging action and reach to grasp action. Banging action in infants and toddlers is assessed in order to understand the ontogeny of hammering tool use in humans. Kahrs and colleagues (2012, 2013, 2014) assessed banging in infants and toddlers using the kinematic variables- distance traversed by hand, straightness of hand trajectory, peak and average velocity of hand during upward and downward motion and angle of impact. The traversed distance, straightness of hand trajectory and angle of impact provided spatial organization of hand movements during banging action. Velocity profile gave temporal organization of hand movements. They found developmental change in both spatial and temporal profiles of banging movements. These variables can be used to quantify the shaking behavior in infants and toddlers. Assessing the non- linear measures of the spatial and temporal profile of banging and shaking exploratory behaviors may also provide valuable information regarding exploratory behaviors in childhood. The reach to grasp behavior has been assessed in infants and children for different task constraints such as object size, shape, texture, weight etc. It is usually analyzed as two components- reach and grasp. The reach component is assessed using kinematic variables such as straightness index, smoothness, variability of end point trajectories, temporal arm-trunk coordination, movement initiation, velocity profiles (peak, mean, time to peak velocity), movement time (Konczak & Dichgans, 1997; Pryde et al., 1998; Schneiberg et al., 2002; Smyth et al., 2004; Zoia et al., 2006). The grasp component is assessed using kinematic variables such as grip aperture distance (maximum and minimum), time taken for maximum grip aperture, normalized grip aperture, peak aperture, time to and after peak aperture, percent time to peak 13 aperture (Konczak & Dichgans, 1997; Pryde et al., 1998; Schneiberg et al., 2002; Smyth et al., 2004; Zoia et al., 2006). In summary, these studies have found that the quantitative characteristics of reach and grasp components show decrease in variability and increase in fluency with increasing age. These components of exploratory behaviors have also been found to show atypical characteristics in children with autism spectrum disorder. Effect of object properties on manual exploratory behaviors Manual exploratory behaviors provide an opportunity in early childhood to actively explore the physical properties of an object such as size, shape, texture, etc. (Gibson, 1988; Lockman, 2000; Thelen, 1981) According to the direct perception theory, learning to perceive object properties directly influence the manual exploratory behaviors which in turn contribute to the perceptual abilities. Such dynamic interplay continues past infancy throughout early childhood as the perceptual-motor system is continuously developing (Aslin & Smith, 1988; Libertus & Hauf, 2017). Thus, assessing the effects of object properties on the characteristics of manual exploratory behaviors during early childhood provides insights on this developing perceptual-motor system. Assessing manual exploratory behaviors under the effects of object properties allows us to understand the developmental changes in the perception of object properties. Characteristics of manual exploratory behaviors were classified by Ruff (1984) according to object shape, texture, color and size in infants. Findings showed that infants showed an increase in fingering behaviors for the textured objects with increasing age. This not just indicated that infants were perceiving object texture but also that the object property was deemed more important since exploratory behaviors specific to object texture were exhibited more often. Using the same standardized tasks in toddlers and preschoolers, it is possible to assess the object properties that are considered more important and used for exploring objects. 14 Assessing the effects of object properties on manual exploratory behaviors also provides insights on the simultaneously developing motor system. Effects of object shape, size and texture are found to affect the prehensile grip configuration used for manually exploring objects (Lee et al., 2006; Newell et al., 1989). Both studies found that the infant’s prehension and their grip configurations changed with increasing age and were strongly guided by the object properties of size, shape and texture. Newell and colleagues (1989) also found a difference in the way infants used sensory information systems. According to the constraints model, the grip configurations and prehension activities will also change due to individual constraints such as age and physical growth. So, it is important to assess the effects of object properties on the prehension activities that precede manual exploratory behaviors throughout early childhood. These studies indicate that different individual, environmental and task constraints influence the characteristics of manual exploratory behaviors similar to other motor behaviors. By assessing the effects of different constraints on these behaviors at different time points during early childhood, it is possible to get insights on how perception of different object properties emerges, develop and evolve with age. This also provides information on the developing motor system. Together these insights on the perceptual-motor system contributes to our understanding on how children learn different tool use skills. Clinical significance: Manual exploratory behaviors in Autism Spectrum Disorder (ASD) The atypical expressions of manual exploratory behaviors are observed as early as 6 months of age in infants at high risk of ASD. Quantitative assessment techniques can provide reliable measurements of these atypical expressions which in turn can act as potential biomarkers of ASD. With a thorough quantitative and qualitative assessment of these atypical expressions in infancy, it is possible to shorten the current timeline for ASD diagnosis. With the current 15 diagnostic tools, ASD can be detected as early as 18 months of age; however, most children do not receive a final diagnosis much later in life (CDC, 2022). Atypical manual exploratory behaviors have been reported by several studies in infants and toddlers with high risk of ASD (Baranek, 1999; Bhat et al., 2011; Campione et al., 2016; Kaur et al., 2015; Landa & Garrett-Mayer, 2006; Ozonoff et al., 2008; Srinivasan & Bhat, 2019; Teitelbaum et al., 1998). Ozonoff and colleagues (2008) found atypical exploratory behaviors such as increased spinning and rotation of objects and reduced visual exploration of objects in 12-month old infants at high risk of ASD. Kaur et al. (2015) assessed exploration of distinct shape, size in infants at high risk of ASD from 6-15 months of age. They found at 6 months infants showed less grasping and mouthing, at 9 and 12 months they performed significantly lower levels of purposeful dropping of the objects and at 15 months they showed persistent mouthing of the object. Pierce and Courchesne (2001) observed that the children (age range: 3-8 years) diagnosed with ASD performed significantly less active exploration of containers that were filled with different objects. Similarly, several other studies (Kawa & Pisula, 2010; Pisula, 2003; Rodman et al., 2010) found atypical manual exploratory behaviors in children diagnosed with ASD; especially differences in the time spent looking at objects. Infants and toddlers with high risk of ASD also showed differences in their reaching and grasping movements. As reported by Yang et al. (2014), the reach to grasp movements in children with ASD had significantly longer movement times, larger normalized jerk, and more movement units than the typically developing children. Prehension movements were kinematically assessed in terms of reach and grasp components by Campione et al. (2016) in 4–5 year old children with ASD. While the grasp components (variables similar to previous study) were comparable to control participants, the reach components had atypical presentation in the 16 children with ASD. Overall, infants and toddlers with high risk of ASD showed distinct profiles of object exploration behaviors that are influenced by different task and environment constraints such as object properties. Summary In a nutshell, manual exploratory behaviors play a pivotal role during early childhood owing to their functional and clinical role in motor development of children. While these behaviors similar to other motor skills are constrained by different task, environment and organism constraints, they simultaneously contribute to learning of the perception of different object properties. Quantitative analysis of these behaviors is required to measure the subtle characteristics of underlying movements that are difficult to detect on qualitative analysis. However, their quantitative analysis is currently difficult owing to the lack of a feasible method for curation of time series data according to the type of behaviors. This methodological challenge needs to be addressed before tackling other research questions related to these behaviors. Therefore, in this dissertation, we conducted three studies that: (1) developed a feasible classification method using machine learning to identify three most commonly observed exploratory behaviors on time series data, (2) assessed kinematic characteristics of two important manual exploratory behaviors under the effect of age and, (3) assessed the effects of age and object properties (size, shape and texture) on the characteristics of manual exploratory behaviors observed in preschoolers. 17 CHAPTER 3. AUTOMATED CLASSIFICATION OF MANUAL EXPLORATORY BEHAVIORS USING SMART OBJECTS AND MACHINE LEARNING Abstract Manual exploratory behaviors that form the basis of tool use behavior in children are mostly qualitatively characterized in terms of their frequency and duration of occurrence. To fully understand their functional and clinical significance, there is a need for quantitative movement characterization in addition to qualitative analysis using traditional video coding methods. However, there are two main challenges in quantifying these exploratory behaviors - (i) reliably classifying the type of movement, and (ii) performing this classification on a time series without requiring its manual curation. Here, we propose a machine learning based classification method to address these challenges. We measured three common exploratory behaviors (rotation, fingering and throwing) in 10 college-aged adults using “smart objects” that had wireless Inertial Measurement Units (IMUs) embedded in them. We then calculated various statistical features based on the linear acceleration and angular velocity data and used them to train 22 machine learning classifiers to identify the three behaviors. All classifiers identified the behaviors on time series with a substantially higher accuracy than chance level (average accuracy = 84.95 + 4.16%, chance level = 33.33%). Of all 22 models, SVM Quadratic, SVM Medium Gaussian and Narrow Neural Network were the best models in classifying the three behaviors with fairly high average accuracy ~89% across all testing datasets. This classification method holds a promise to facilitate automated movement characterization of manual exploratory behaviors, which in turn may contribute to the early assessment of autism spectrum disorders (ASD). 18 Introduction Manual exploration of objects during early childhood is among those critical behaviors that lay the foundation of “tool use”, one of the most advanced human motor skills (Lockman & Kahrs, 2017). The act of exploration in the first year serves as raw sensory-perceptual feedback of objects that evolves into a more cognitively controlled and functionally directed play during later years of childhood (McCall, 1974). For example, infants show mouthing behavior during early infancy, but start purposeful play with the same objects by the end of their first year (Gibson, 1988; Thelen, 1981). This transition from mouthing to purposeful play emerges gradually and systematically from the innumerable active interactions with the objects that help them perceive the use of objects and means of effective interaction with them (Lockman, 2000; Lockman & Kahrs, 2017). Such active interactions with an object, especially using hands, helps the infant learn to perceive object properties such as shape, size, object segregation and individuation; thereby contributing towards tool use development (Chen et al., 2000; Lockman & Kahrs, 2017; McCarty et al., 1999; Smitsman, 1997). This dynamic interplay between exploratory actions and perception of object properties continues past infancy throughout early childhood. Moreover, atypical or limited manual exploratory behaviors are often linked with developmental motor delays (Ozonoff et al., 2008). Thus, these seemingly random looking exploratory behaviors during early childhood play an important role during development. Characterizing manual exploratory behaviors during early childhood have heavily relied on direct observation and video recording. These behaviors are assessed in relation to age (Ruff, 1984), object properties (Corbetta & Thelen, 1996; Ruff, 1984), other motor skills (Soska & Adolph, 2014) as well as their atypical manifestations in ASDs (Kaur et al., 2015; Ozonoff et al., 2008; Srinivasan & Bhat, 2019), and are mainly characterized in terms of variables such as duration, frequency, order of occurrence, etc. For instance, Ruff (1984) characterized different 19 exploratory behaviors in infants as a function of age and object physical properties by measuring their duration and frequency of occurrence using video coding methods. Similarly, Ozonoff and colleagues (2008) measured frequency and duration of exploratory behaviors in typically developing and high-risk infants, and found that infants at high risk of ASD showed significantly more rotation and spinning behaviors. Such qualitative assessments have provided insights on the developmental trajectory of these behaviors and their role as potential biomarkers. However, they do not provide information about the underlying movement characteristics during these behaviors, which is important for understanding both their theoretical and clinical significance. Movement characteristics require quantitative assessment using kinematic variables such as velocity profile, movement trajectory, smoothness, etc., to measure the movement properties (Campione et al., 2016; Kahrs et al., 2014; Konczak & Dichgans, 1997; Lee et al., 2011; Zoia et al., 2006). Quantitative assessment measures subtle characteristics of movements that often are subjective or difficult to detect using qualitative analysis. For instance, using kinematics (dimensionless jerk), Lee and colleagues (2011) quantified the developmental transitions in the object-oriented pre-reaching arm movements observed in infancy, however, their qualitative analysis could not distinguish these changes in the movements. Similarly, atypical motor behaviors such as repetitive rotational behaviors seen in ASD can be quantified in terms of their degree and nature of repetitiveness using quantitative analysis. Together with qualitative analysis, quantitative assessment can serve as reliable measurements of the stereotypical or repetitive behaviors seen in atypical motor development; thereby providing biomarkers for early detection of developmental delays and disorders. However, except for a few specific behaviors (shaking, banging, etc.) (Brakke et al., 2007; Kahrs et al., 2012), the quantitative assessment of most manual exploratory behaviors is yet to be conducted systematically. 20 Quantitative assessment of exploratory behaviors involves the “curation” of time series data according to the type of these behaviors prior to their kinematic analysis. For example, in behaviors like walking, no curation is required since there is only a single behavior that needs to be analyzed and the data is directly amenable to quantitative analysis. However, during manual exploration, there may be several behaviors that are seen within the course of a few seconds, which means that prior to their quantitative analysis the data has to be curated into individual segments, each consisting of a single behavior. Conventionally, such curation can be done using video coding methods (Lee et al., 2011), but this may not be a practical option for two reasons. Firstly, exploratory behaviors by nature are complex and variable with some behaviors lasting for a very short duration. This can make manual video coding difficult. Secondly, even if these behaviors could be identified, video coding requires a time-consuming frame-by-frame analysis that reduces the overall scope of a study. Thus, kinematic characterization of manual exploratory behaviors requires an automated curation approach that can address two of its main challenges- (i) reliably classify the type of movement and (ii) perform this classification throughout a time series with minimal to no user intervention. One possibility for such automated curation is to use the machine learning methods for behavioral classification prior to conducting their quantitative analysis. These methods are increasingly used in the classification of spontaneous movements in infants (Goodfellow et al., 2018; Ihlen et al., 2019; Shin et al., 2022; B. A. Smith et al., 2015), motor function in stroke patients (Kim et al., 2021), different human activities (Anguita et al., 2013; Debes et al., 2016; Ravi et al., 2005; Ronao & Cho, 2016; Xu et al., 2013) and gait analysis (Ding & Fan, 2014; Kwolek & Kepski, 2014; Mao et al., 2017; Tunca et al., 2017). With adequate training, classifiers can not only classify different behaviors with high accuracy but also process large amounts of data in substantially shorter amounts of time relative to conventional video coding 21 methods. As a result, the ability to analyze large datasets increases by several-fold, which often is another challenge encountered by several motor learning and development studies (Lohse et al., 2016; Ranganathan et al., 2020). This advantage of classifiers also outweighs their disadvantage resulting from data loss when they misclassify behaviors. These reasons make classifiers a feasible and practical option for behavioral classification required prior to conducting quantitative analysis of different exploratory behaviors. However, to our knowledge, machine learning techniques are mainly used in classifying spontaneous movements, motor functions in stroke patients, human activities, gait analysis (Anguita et al., 2013; Debes et al., 2016; Ding & Fan, 2014; Goodfellow et al., 2018; Ihlen et al., 2019; Kim et al., 2021; Kwolek & Kepski, 2014; Mao et al., 2017, 2017; Ravi et al., 2005; Ronao & Cho, 2016; Shin et al., 2022; B. A. Smith et al., 2015; Tunca et al., 2017), but have not been applied in the context of manual exploratory behaviors. To address this issue, we propose a machine learning (ML) -based classification method to classify manual exploratory behaviors in a time series. Manual exploratory behaviors are highly variable in nature compared to other human activities (posture, walking, etc.) wherein this concept has been used before. Thus, our first step was to test the feasibility of using ML -based classifiers for classifying these behaviors which called for a major consideration from a method’s design perspective. To test initial feasibility of the approach and gauge the upper estimate of classifier performance, data from adults was used instead of children because the data was more likely to be consistent compared to data from children. Using an adult dataset was also important from a practical perspective as it largely aided the data collections that had to be conducted remotely owing to the COVID-19 pandemic. We could collect more data per participant for training purposes, which otherwise would be difficult with children. Since this method is to be 22 eventually used on data from children, the study protocol was designed in a way that can be used with children without requiring major modifications. In this study, we classified three manual exploratory behaviors- rotation, fingering and throwing. These behaviors were chosen as they are: (1) amongst the most commonly found behaviors in early childhood (Ruff, 1984), (2) indicative of gross and fine motor skills in children, and (3) have clinical importance in cases of ASD (Ozonoff et al., 2008). Participants performed the three behaviors with “smart” sensor-equipped objects, and we evaluated the performance of our classifier algorithm in identifying these behaviors based on the sensor data. This study is a proof of concept that assesses the plausibility and scope of using ML -based classification methods in the context of manual exploratory behaviors. Method Participants Ten healthy college adults (mean age = 23.30 + 5.60 years) with no history of neurological or musculoskeletal injury participated that involved one home-based data collection. Participants received a $25 gift card for their participation. Participants provided written informed consent and procedures were approved by the Michigan State University human research protection program. Apparatus We used five experimental objects for the participants to perform the three behaviors of interest. We used multiple objects of different shapes and sizes to elicit a wide range of exploratory behaviors within participants. Four out of five experimental objects were made of firm Styrofoam in two different shapes (cube vs. ball) and two different sizes (5 cm vs. 10 cm) (refer figure 3.1a). All four objects had a slit-like cavity in the center to snugly fit a sensor in it (refer figures 3.1b, 3.1c and 3.1d). These sensors were wireless Inertial Measurement Units 23 (IMUs) (3-space mini-Bluetooth sensors, YEI Technology, Ohio USA) to measure movement data. They were 3 cm x 3 cm x 1.3 cm in size, weighed 9 grams and consisted of an accelerometer, gyroscope and magnetometer. The sensor on its own was the fifth experimental object. We covered the objects with custom made sleeves to hide the sensor from the participants during the trials so that their behaviors were not curtailed or influenced by the sensor placement but performed with a focus on the object. Further on, we used Zoom to virtually connect with the participants and TeamViewer to remotely control the computer on participant’s end (refer figure 3.2a and 3.2b). We recorded this video call and measured participants’ exploratory behaviors using sensors for data analysis purposes. Figure 3.1. (a) Four objects made of firm Styrofoam in two different shapes and two different sizes- 5 cm hard ball, 5 cm hard cube, 10 cm hard cube, 10 cm hard ball. (b) 3-space mini– Bluetooth Inertial Measurement Units (IMU) 3 cm x 3 cm x 1.3 cm in size, weighed 9 grams. (c) Every object had a slit like cavity to fit the sensor in it. (d) Sensor was securely placed in the cavity of the object and covered with a custom-made red colored sleeve to hide sensor placement from the participants. 24 Protocol Participants performed three manual exploratory behaviors (rotation, throwing and fingering) in a natural way using five experimental props (4 objects + 1 sensor itself). Each data collection session had five blocks of trials, one block for each experimental prop. Each block had 15 trials, five trials for each of the three behaviors. Participants performed one behavior per trial. Each participant performed a total of 75 trials across five blocks (refer figure 3.2c). We cued the participants about the experimental prop to use during a block and the behavior to perform during a trial along with its duration of performance. The three behaviors were performed alternatively across trials based on a randomly generated order that was unique to each block. During rotation and fingering trials, participants performed these behaviors continuously until the experimenter asked them to stop, while throwing trials ended once the participant had thrown the experimental prop. Each of the rotation and fingering trials were around 10 seconds long while throwing trials lasted for ~5 seconds. In addition, there was a test trial at the end of each block which we named as ‘sequence trial’ as participants performed all three behaviors sequentially in this trial in a random order. We measured these trials specifically to test the accuracy of the machine learning classifier since they mimic a natural situation in which exploratory behaviors are performed consecutively one after another instead of one behavior per trial. There were 5 sequence trials per participant and each sequence trial had all the three behaviors performed at least twice in a random sequence. 25 Figure 3.2. (a) One remote experimental session was conducted per participant. Experimenter communicated with the participant via video call and controlled the trials on the participant’s computer using TeamViewer. (b) Experimenter and participant’s computer screenshots demonstrating what they saw during the experimental session. Since experimenter-controlled sensors on participant’s end via TeamViewer and communicated with the participant via video call, the left side of the experimenter’s screen displays the software used to collect sensor data and right side displays the video call. Participants only saw experimenters during the session, so their screen displays video calls with the experimenter. (c) Every participant performed 5 blocks of 15 trials each. During each block, participants performed the three behaviors 5 times (one behavior per trial) in a random order. Participants also performed one sequence trial at the end of each block in which they performed all three behaviors sequentially instead of one behavior per trial. Sequence trials were used for testing purposes. Data analysis Data analysis consisted of training and testing machine learning (ML) models for classification of three behaviors- rotation, throwing and fingering. 26 Classifier training We trained the ML models for classification of three behaviors by following a four-step process (refer figure 3.3a). Train 22 classification algorithms using the features extracted from time series Figure 3.3. (a) Various classification algorithms were trained using a 4-step process: data pre- processing, segmentation, feature extraction and training classifiers. (b) Trimming of sample data from a throwing trial. The start and end of throwing trials had idle data which was collected to avoid missing the actual throw due to potential internet lag. So, the start and end of throwing trials were trimmed by applying an operational threshold of 0.3 m/s2. (c) Sample data depicting implementation of segmentation process. Segmentation is a process of dividing time series into windows of data that can be represented by a common characteristic. Sliding window approach was used in which the time series was divided into 1 second windows with 50% overlap between two consecutive windows. 1. Data pre-processing The first step was to prepare and process the data in two stages: re-sampling and filtering. Our sensors measured multiple physical quantities, but we used angular velocity (roll, pitch and yaw) and linear acceleration (x-, y- and z-axes). The data was then resampled to fix minor irregularities in data sampling rate which may have occurred due to Bluetooth connectivity. The 27 ground truth labels were also added to time series at this point. Finally, we filtered the data using a low pass Butterworth filter (5 Hz) to remove any systemic noise. For the throwing trials, we performed an additional trimming step (refer figure 3.3b). This trimming was needed to remove the idle data (~2-3 seconds long) that was recorded intentionally at the end of each throw. Throwing trials were shorter than rotation and fingering trials and consisted of a single throw. Since the experimenter remotely recorded the data on the participant’s end, these trials were more susceptible to missing the actual throw data due to internet lag. So, the experimenter continued recording the throwing trial even after the throw was complete. This idle data resembled fingering behavior and so needed to be removed for proper classification. For the purpose of trimming, we used resultant linear acceleration to identify periods of time at the beginning and end of the trial where the object was more or less stationary. We then set an operational threshold of 0.3 m/s2 based on the accelerometer sensitivity of our sensors. Using this threshold, we determined front and back cut off timestamps, which were then utilized to remove idle sensor data from both angular velocity and linear acceleration. 2. Segmentation The next step was to divide the pre-processed data into segments which can be represented by a common characteristic. This process of dividing the time series data is called segmentation which is a key to extracting meaningful information from the data later on (Sousa Lima et al., 2019). We used the overlapping time-based segmentation also known as the sliding window approach in which the time series is divided into segments/windows of a specific time period and two consecutive windows have an overlap of data points between them. This approach allowed reusing of the data; thus, optimizing the training data sample size. The datasets were divided into 1 second data windows that had 50% overlap between them (refer figure 3.3c). 28 One second was chosen as the segmentation time window since that was the shortest behavior time in our dataset. 3. Feature extraction Next, we computed features for all the windows created in step 2. A feature is a property that carries useful information about a set of data points (Sousa Lima et al., 2019). We calculated different statistical features for every 1 second data windows and chose five of them to be representative of our data based on empirical evidences and previous studies (Anderson et al., 2007; Berchtold et al., 2010; Bieber et al., 2010; Dernbach et al., 2012; Fontecha et al., 2013; Kwapisz et al., 2011; Saponas et al., 2008; Sousa Lima et al., 2019; Yang, 2009). These were mean, standard deviation, interquartile range, energy, and autocorrelation coefficients. While mean, standard deviation and autocorrelation coefficients were the main distinctive quantities between three behaviors, interquartile range and energy helped to fine tune the distinction between periods of rotation that have resemblance with fingering. Table 3.1 describes the formulae used for calculating the five statistical features. For interquartile range, we calculated the difference between the 75th and 25th percentile using the respective formula from table 3.1. For autocorrelation coefficients, we first computed autocorrelation coefficients (𝜌) and its standard error using the respective formulae from table 3.1. Next, we set 95% confidence intervals at ±2𝑠𝑒(𝜌) and calculated the total number of autocorrelation coefficients outside the confidence interval for a given window (Box et al., 2015). This was done as autocorrelation coefficients outside the confidence interval indicate repetitive rotational behaviors and so this criterion helped to distinguish between rotations from the other two behaviors. We calculated mean, standard deviation, interquartile range and energy for 1 second windows of angular velocity and linear acceleration in x, y and z direction. We also calculated 29 autocorrelation coefficients for the 1 second windows of angular velocity in the three directions. Therefore, we had 15 features extracted from angular velocity and 12 from linear acceleration, making a total of 27 features calculated for each 1 second window. Table 3.1. List of formulae used to calculate five statistical features for classifier training Statistical Feature Formula 𝑁 𝑥(𝑖) Mean (𝑥̅ ) 𝑥̅ = ∑ 𝑁 𝑖=1 𝑁 2 (𝑥(𝑖) − 𝑥̅ )2 Standard deviation (sd) 𝑠𝑑(𝑥) = √∑ 𝑁 𝑖=1 Interquartile range (iqr) 𝑖𝑞𝑟(𝑥) = 𝑄3 (𝑥) − 𝑄1 (𝑥) 𝑁 Energy (E) 𝐸(𝑥) = ∑ 𝑥(𝑖)2 𝑖=1 𝑁 ∑𝑖=𝑘+1(𝑥(𝑖) − 𝑥̅ )(𝑥(𝑖 − 𝑘) − 𝑥̅ ) 𝜌(𝑘) = ∑𝑁 𝑖=1(𝑥(𝑖) − 𝑥̅ ) 2 Autocorrelation coefficient (𝜌) and its standard error (𝑠𝑒) (1 + 2 ∑𝑘−1 2 √ 𝑖=1 𝜌(𝑘) ) 𝑠𝑒(𝜌) = 𝑁 Note. x = physical quantity whose feature is being calculated; 𝑥̅ = mean of x over a segment; N = total number of samples in a segment; Q3(x) = third quartile or 75th percentile of x; Q1(x) = first quartile or 25th percentile of x; k = sample lag 4. Training machine learning classifiers We used a total of 27 features (extracted in step 3) from the entire time series data to train different classification algorithms using the MATLAB Classification Learner application. As our data had a highly variable nature with no established precedent from prior work, it was difficult to predict apriori a particular class of classifiers that would be most suitable for classifying the data. So, we trained a total of 22 classifiers from 5 groups of algorithms that are most commonly used for classifying human activities (Sousa Lima et al., 2019). These algorithm groups were: 30 Decision Trees, Naive Bayes Classifiers, Support Vector Machine (SVM), K-Nearest Neighbor Classifiers, Neural Network Classifiers. Classifier testing and validation We conducted performance evaluation of all the 22 trained classifier models using three different methods- 5-fold cross validation, 70-30% data split and behavior classification of sequence trials. Usually, validation can be done using any one of these methods but we needed robust testing as there were no prior studies to compare our results with. Based on previous similar study (Goodfellow et al., 2018), we set a criterion to choose the best models from 22 trained classifiers that are most representative and suitable for classifying the three behaviors. According to this criterion, the three models with the highest performance on all performance indices across all testing methods were considered best representative of our data. In all the three testing methods, we measured performance of the classifiers using standard evaluation metrics- accuracy, precision, recall and f1 score (Refer Table 3.2). To classify the test data, we processed all the data using the training phase steps 1→3 Below are the details of the three testing methods: Table 3.2. List of evaluation metrics with their formulae Evaluation metrics Formula Accuracy (TP + TN / n) * 100 Precision TP / TP + FP Recall TP / TP + FN f1-score 2 x (Recall and Precision) / (Recall + Precision) Note. n = total number of samples; True Positives (TP) = correctly predicted observations as positive; True Negatives (TN) = correctly predicted observations as negative; False Positive (FP) = incorrectly predicted observations as positive; False Negative (FN) = incorrectly predicted observation as negative 1. 5-fold cross validation This method is inbuilt in the MATLAB Classification Learner app. First, it partitions data into five roughly equal subsets/folds. Four subsets are used to train a model and one subset is 31 used to validate it. This process is repeated until each of the five subsets have been used exactly once for validation. Finally, the average accuracy of all five runs is calculated. We ran a 5-fold cross validation test on all the classifier models that were trained using the entire dataset. 2. 70-30% data split For performance evaluation using the data split method, we divided the entire dataset into two: training set (70% data) and testing set (30% data). We re-trained the classifiers using 70% dataset and then tested their performance on a 30% testing set. 3. Behavior classification of sequence trials We collected sequence trials at the end of each block in which participants performed all the three behaviors consecutively in a random order instead of one behavior per trial. Thus, sequence trials were closer to what would be a typical trial in manual exploratory studies. We used the classifiers trained on the entire dataset to classify behaviors on the sequence trials and measured their performance. For this we first set the ground truth of all the sequence trials using video coding methods. We coded the recorded video call using the behavioral coding software- Datavyu for the duration and order of behaviors in the sequence trials. We then classified the behaviors on the sequence trials using the trained classifier models, matched their results with the ground truth and conducted performance evaluation. Results Using three testing methods, we evaluated performances of 22 trained models that belonged to 5 classifier groups most commonly used for the classification of human activities- Decision Trees, Naive Bayes Classifiers, Support Vector Machine (SVM), K-Nearest Neighbor Classifiers, Neural Network Classifiers. While not all the classifier groups were suitable for classifying our data, their accuracy in classifying the three behaviors was substantially above the 32 chance level (33.33%) (Refer table 3.3). According to our criterion, the three models that were most representative and suitable for classifying the three behaviors were: SVM Quadratic, SVM Medium Gaussian and Narrow Neural Network. These three models consistently showed higher performances on all evaluation metrics compared to the other trained models across three testing methods. Below, we describe the performances of these three models on all the testing methods in detail as well as in Figure 3.4. Table 3.3. Accuracies of all 22 trained models across the three different testing methods 5-fold cross 70-30% Behavior Classifier group Model name validation data-split classification of (%) (%) sequence trials (%) Fine Tree 91.90 88.72 80.71 Decision Tree Medium Tree 90.62 87.85 78.80 Coarse Tree 87.75 83.65 75.86 Gaussian Naïve Bayes 82.46 83.34 71.14 Naïve Bayes Kernel Naïve Bayes 83.0 80.96 71.70 Linear SVM 92.13 90.71 82.65 Quadratic SVM 93.80 91.40 83.20 Support Vector Cubic SVM 92.81 89.10 80.81 Machine (SVM) Fine Gaussian SVM 83.0 76.92 62.18 Medium Gaussian SVM 93.60 91.00 83.10 Coarse Gaussian SVM 85.88 83.23 79.44 Narrow Neural Network 93.90 91.10 83.0 Medium Neural Network 92.21 87.96 82.42 Neural Network Wide Neural Network 92.42 89.20 81.13 Bilayered Neural Network 93.70 90.15 82.0 Trilayered Neural Network 93.33 90.32 83.11 Fine KNN 89.55 85.30 77.56 Medium KNN 87.86 87.30 78.77 K-Nearest Coarse KNN 80.82 81.85 76.19 Neighbor (KNN) Cosine KNN 89.19 86.60 79.74 Cubic KNN 87.52 87.71 78.50 Weighted KNN 89.65 87.0 79.50 33 5-fold cross validation method Of all the trained models, Narrow Neural Network had the highest accuracy (94.0%) followed by SVM Quadratic (93.90%) and SVM Medium Gaussian (93.60%). All three models had high average precision scores across the three behaviors with a minimal difference between them (Narrow Neural Network = 93.40%, SVM Quadratic = 94.03% and SVM Medium Gaussian = 94.63%). They also had high average recall scores across the three behaviors (Narrow Neural Network = 91.73%, SVM Quadratic = 90.63% and SVM Medium Gaussian = 90.50%). Average f1 scores were quite high and similar for all the three models (Narrow Neural Network = 92.53%, SVM Quadratic = 92.19% and SVM Medium Gaussian = 92.36%). In general, all three models showcased high performance in classifying the three behaviors even though one of the behaviors (throwing) had relatively less data which could have potentially led to the problem of overfitting. 70-30% data split Here we first trained all the models using 70% training set, thereafter tested them to classify the three behaviors on the 30% testing set. SVM Quadratic had the highest accuracy (91.40%) followed by Narrow Neural Network (91.10%) and SVM Medium Gaussian (91%). In terms of precision, both the SVM models had high scores (SVM Quadratic = 91%, SVM Medium Gaussian = 91.36%) while it was slightly lower for the Narrow Neural Network (88.83%). Average recall scores were fairly high and similar across all the three models (SVM Quadratic = 89.33%, SVM Medium Gaussian = 88.43 and Narrow Neural Network = 89.93%). In terms of average f1 scores, all three models displayed fairly high scores within a difference of <1% amongst them (SVM Quadratic = 90.04%, SVM Medium Gaussian = 89.73%, Narrow Neural Network = 89.24%). Although the performance metrics were slightly lower in this 34 method compared to the 5-fold cross validation results, they were high enough to reliably predict the correct behaviors. Behavior classification of sequence trials Similar to the previous two testing methods, all 22 trained models were tested to classify behaviors on sequence trial dataset. We found that the models with the highest accuracy were the same as in the other two testing methods (SVM Quadratic = 83.20%, SVM Medium Gaussian = 83.10% and Narrow Neural Network= 83.0%). They had moderate average precision scores (SVM Quadratic = 82.67%, SVM Medium Gaussian = 83.23% and Narrow Neural Network = 82.30%). The three models did not differ in terms of their average recall scores which were also moderate in nature (SVM Quadratic = 81.83%, SVM Medium Gaussian = 81.36% and Narrow Neural Network = 81.87%). Due to moderate precision and recall scores, their average f1 scores were also moderate and in the same range (SVM Quadratic = 82.19%, SVM Medium Gaussian = 82.09% and Narrow Neural Network = 82.05%). In general, the performance of all the three models was moderately accurate but higher than the rest of the models and sensitive in classifying the behaviors on sequence trials. 35 Figure 3.4. SVM Quadratic, SVM Medium Gaussian and Narrow Neural Network were the best models that consistently showed higher performance than the other 19 models across all three testing methods They showed high accuracy, precision, recall and f1 scores on 5-fold cross validation and 70-30% data split method. Their performance metrics were moderate on the behavior classification of sequence trials. (a) Accuracy of highest three models in the three testing methods. (b) Average precision of models across the three behaviors. (c) Average recall of the models across the three behaviors. (d) Average f1 scores of the models across the three behaviors. Error bars represent one standard deviation. Discussion The purpose of this study was to develop a machine learning based automated classification system for the identification of three commonly seen manual exploratory behaviors on time series. Such ML -based automated classification systems have been previously used for classifying different human behaviors, but this study is the first attempt in using this approach in the context of manual exploratory behaviors. We trained 22 different machine learning models using data from 10 adults and measured their accuracy using 3 test methods. All the models had a performance accuracy substantially higher than the chance level (33.33%); average accuracy of 22 models across three testing methods = 84.95 + 4.16%. SVM Quadratic, SVM Medium Gaussian and Narrow Neural Network were the best models for classification of these three 36 behaviors as their performance on all the evaluation metrics was higher than the other models across all three testing methods. Moreover, the trained models classified these behaviors on time series in substantially shorter duration of time than the conventional video coding. The study findings indicate that a ML -based method is both- feasible and scalable alternative to conventional video coding for identifying the manual exploratory behaviors on time series; thus, facilitating their quantitative assessment. The first important function of the ML -based automated classification system is to reliably identify manual exploratory behaviors on a time series. This requires training ML classifiers using features that are representative of behavioral characteristics. Based on the type of data, different time and frequency domain features are used for this purpose (Sousa Lima et al., 2019), wherein human activities are commonly classified using time domain statistical features (Anjum & Ilyas, 2013; Shoaib et al., 2015). Therefore, we calculated various time domain features for the three behaviors and found five statistical features- mean, standard deviation, autocorrelation coefficients, interquartile range and energy to be most representative of their unique characteristics. Training classifiers using these features yielded moderate to high performance in many algorithms on the three testing methods; top three classifier models consistently identified the three behaviors on test datasets with high accuracy and precision. These features may not be representative of all the behaviors that fall under the umbrella term, “manual exploratory behaviors”. Such behaviors have high intra- and inter- variability which may require different features to represent their unique characteristics. However, the five features used in this study certainly provides a foundational framework to identify behavior specific features for future ML -based classification of different manual exploratory behaviors. Another important function of this automated classification system is to classify multiple behaviors occurring consecutively on a time series. We used sequence trials to test this function 37 of our classifiers since they consisted of all three behaviors performed randomly and consecutively as opposed to the training dataset that consisted of one behavior per trial. So, the sequence trials mimicked a typical manual exploratory behavior trial; thereby providing an estimate of classifier performances for use in a typical manual exploratory study. The three best models (SVM Quadratic, SVM Medium Gaussian and Narrow Neural Network) had moderately high performance but it was lower than on the other two testing methods. We attribute this reduction to: (1) presence of inter-behavioral transition data present only in the sequence trials and, (2) small testing dataset in sequence testing (~10% of the entire training dataset), which increased the weightage of every prediction on the performance metrics and so inflate the effect caused by incorrect prediction on the overall metrics. However, our classifiers still performed substantially higher on sequence trials than the chance level (33.33%) and were comparable to the classifier performances attained by prior studies classifying human activities (Goodfellow et al., 2018; Kang et al., 2018; Patel et al., 2019). We understand incorrect classification incurs data loss, but using ML -based classifiers can substantially reduce data processing time. Such an automated classification system makes manual exploratory studies highly scalable by increasing the scope of data analysis which in turn can compensate for the data loss. Although our aim is to use ML -based classification methods for classifying behaviors in children, we used an adult dataset to train and test classifiers in this study. Using adult dataset was a major consideration we made to test the plausibility of using ML techniques in the context of these behaviors. It is quite evident from our findings using the adult dataset that the ML - based approach has strong potential to handle the highly variable nature of these behaviors and classify them with high accuracy. The upper estimate of most classifier performances in this study was high enough to tolerate reasonable reductions when used on data from children. Data from children is expected to have more inconsistencies and variabilities compared to adult 38 dataset which could lead to decrease in classifier performances. However, this can be dealt with by optimizing the features used for training purposes to be more representative of the child dataset as well as training the classifiers with larger dataset. Nonetheless, there will be differences in how adults and children perform these behaviors, but this study is a proof of concept assessing the implementation of a method. In addition to the automated classification system, the smart sensor equipped objects used in this study make an important contribution to the measurement of manual exploratory behaviors in children. Conventionally, movement measurement of manual exploratory behaviors requires placing at least two sensors- one on each hand, since children may interact with objects with one or both hands. Such measurement set-up can lead to distraction in children as reported by other movement studies where children often play with sensors or remove them leading to data loss. We address this problem in our smart sensor equipped objects which is based on the concept of commercially available smart toys and place sensors within the object of interaction instead of participants. Participants cannot detect the presence of a sensor which prevents them from playing or removing it. Moreover, only one sensor is needed to track object movements owing to its placement within the object. This entire smart sensor-toy design functions like a battery-operated toy which makes it very user friendly and convenient for conducting home based studies. This smart toy design can be applied to objects of different physical properties such as size, texture, shape, color, etc.; thereby facilitating movement measurements of these behaviors under the effects of different object properties. Overall, this smart toy design together with the ML -based automated classification system opens wide avenues for movement measurement and analysis of manual exploratory behaviors. The current study proposes a proof-of-concept which requires further development in order to be used in manual exploratory studies. There were certain limitations which we plan to 39 address as future directions. First, our classifiers currently can classify only three manual exploratory behaviors, which were chose based on their clinical and functional relevance as well as frequent occurrence in early childhood. We are currently working on expanding the scope of the classifiers for classifying more manual exploratory behaviors. Second, we had to pre-process the throwing trials in order to trim additional idle data at the beginning and end of each trial. While the idle data was collected intentionally to avoid missing data due to internet lag, such pre-processing will not be required in studies that are independent of internet requirement. Moreover, the throwing data in sequence trials were not trimmed, yet the classifier performance in classifying throwing was no different than the other two behaviors. Lastly, we trained and tested classifiers with adult dataset, so this method requires validation on data from children. Our sensor-toy design will not just facilitate testing in children but simultaneously allow testing these behaviors with objects of different physical properties. In summary, we propose a ML -based automated classification system to classify three most commonly observed manual exploratory behaviors in early childhood on time-series. While this method has gained popularity amongst other human behaviors/activities, this is the first account of using it in the context of manual exploratory behaviors. Our findings indicate that the trained classifiers not just classified the three behaviors with high accuracy and precision but completed the classification in a substantially short duration of time compared to conventional video coding methods. Moreover, our smart toy design provides a practical solution to minimizing distractions and data loss by placing the sensor within the toy; thereby optimizing data collection in lab- and home- based studies. Together, the ML -based classification system and smart toy design have high potential to facilitate the quantitative assessment of manual exploratory behaviors. This in turn provides opportunity to explore the uncharted aspects of 40 manual exploratory behaviors and thoroughly understand their functional and clinical relevance in early childhood. 41 CHAPTER 4. EFFECT OF AGE ON THE QUANTITATIVE CHARACTERISTICS OF ROTATION AND THROWING BEHAVIORS IN PRESCHOOLERS Abstract Rotation and throwing are two important manual exploratory behaviors that- (1) functionally contribute to learning of perception of object properties and (2) clinically show atypical expressions in autism spectrum disorders (ASD). Although often studied qualitatively, quantitative characterization (velocity profile, smoothness, etc.) of these behaviors is needed to fully understand their role in early childhood. Here, we examine the effects of age on the quantitative characteristics of rotation and throwing behaviors in early childhood. The ML - based classification system developed in previous study (Patel et al., under preparation) was also assessed to classify behaviors in children. In a home-based remote study, rotation and throwing behaviors were measured in 30 preschoolers (3 age groups- 3-year olds, 4-year olds, 5-year olds) using 5 different objects. Data was collected by recording the video call and using wireless sensors embedded in the objects. The classifiers showed poor performance (average accuracy = 59.86 + 0.21%) in classifying the three behaviors (rotation, throwing and fingering). For rotational behaviors, there was no effect of age on the rotational variability and angular jerk, but significant effect of age on the peak resultant angular velocity. No significant effect of age was found in the kinematics (linear jerk, linear acceleration profiles) of throwing behaviors. These quantitative characteristics measured subtle movement differences in rotation and throwing behaviors that are not distinguishable via visual assessment which in turn provide insights on the developmental changes in their movements in this age group. 42 Introduction Manual exploratory behaviors observed during early childhood hold significant functional and clinical roles in motor development. Functionally these behaviors play an important role in the development of an important motor skill- tool use (Goldfield, 1995; Lockman, 2000; Thelen, 1981). Manually exploring an object in different ways allows children to learn about different object properties which in turn help them to understand the object’s function and interact with it effectively. On the other hand, clinically these behaviors show atypical expressions in autism spectrum disorders (ASD). Such atypical expressions are found in both qualitative (Kaur et al., 2015; Ozonoff et al., 2008) and quantitative characteristics of manual exploratory behaviors (Campione et al., 2016). These findings establish the importance of manual exploratory behaviors in motor development of children as well as the need to conduct systematic qualitative and quantitative assessment of these behaviors to fully understand their functional and clinical role in early childhood. The characteristics of manual exploratory behaviors are mainly assessed using qualitative means of analysis such as video recording, online coding, etc. (Corbetta & Thelen, 1996; Lee et al., 2006; Newell et al., 1989; Ruff, 1984). Qualitative analysis uses frequency, duration, or order of occurrence to characterize these behaviors. While such assessment is rich in contextual information, a quantitative analysis using kinematic variables is required to measure the underlying movements. Kinematic variables such as velocity profile, movement trajectory, smoothness, etc., can characterize the nature of a movement as well as its execution. Such characteristics are subtle and difficult to detect using qualitative analysis but required to discover the clinical and functional significance of a motor behavior. For instance, using kinematics (dimensionless jerk), Lee and colleagues (2011) quantified the developmental transitions in the object-oriented pre-reaching arm movements observed in infancy, however, their qualitative 43 analysis could not distinguish these changes in the movements. However, except for a few behaviors (banging, shaking, etc.,) (Brakke et al., 2007; Kahrs et al., 2013, 2014; Lockman & Kahrs, 2017), quantitative characteristics of most manual exploratory behaviors are yet to be assessed. Manual exploratory behaviors such as banging, striking are extensively studied for their quantitative characteristics (Kahrs et al., 2012, 2013, 2014; Lockman, 2000; Lockman & Kahrs, 2017). The developmental trajectory of these behaviors as they transition into the tool use action- “hammering” is specifically assessed using kinematic variables such as distance traversed by hand, straightness of hand trajectory, velocity profiles and angle of impact. Quantitative findings from these studies have provided a thorough understanding of how random looking banging arm movements in late infancy transition into the hammering action later in development. Together with the qualitative assessment, these findings provide information on the emergence and development of hammering action in childhood. Similar to the banging behavior, a systematic quantitative and qualitative assessment is required for other important manual exploratory behaviors in order to understand their functional and clinical significance in early childhood. Similar to banging behaviors, rotation and throwing are two other manual exploratory behaviors that have functional and clinical importance in early childhood. Qualitative assessment of these behaviors has shown how they help children learn to perceive object properties which in turn play pivotal role in the development of important tool use skills. For example, rotational behaviors help children learn about the 3D nature of objects (Soska et al., 2010). This helps them to perceive the back of an object, plan mental rotation of an object which is central to planning and executing the grip configurations during reaching and grasping of an object (Jung et al., 2015). Clinically, these behaviors show stereotypical patterns in autism spectrum disorders (ASD) (Kaur et al., 2015; Ozonoff et al., 2008). Such qualitative assessments have provided 44 insights on the developmental trajectory of these behaviors and their role as potential biomarkers. A quantitative assessment in addition to their qualitative analysis can measure the characteristics of the underlying movements which in turn can act as reliable measurements of the stereotypical or repetitive behaviors seen in infants and toddlers at high risk of ASD. However, these behaviors, similar to most other manual exploratory behaviors, are mainly assessed qualitatively and lack a quantitative assessment. A key challenge in conducting quantitative assessment of manual exploratory behaviors including rotation and throwing is to curate (identify start and end times of a behavior) the time series data according to the type of different behaviors. Since a typical manual exploratory behavior study comprises of multiple behaviors of interest, the conventional methods such as video coding are not a practical option for data curation. A potential alternative to video coding method is to use machine learning (ML) algorithms for the classification and identification of different behaviors on a time series data. Systematically training machine learning classifiers can allow such classification with high accuracy as well as process the data in substantially shorter amounts of time. We developed a machine learning based classification system to classify three commonly found manual exploratory behaviors- rotation, throwing and fingering (Patel et al., under preparation). This system currently classifies the three behaviors with high accuracy (84.95 + 4.16%) on adult data. The time series data curated using ML classifiers can then be used to conduct quantitative assessment of underlying movements in these behaviors. In this study, we aim to conduct quantitative analysis of rotation and throwing behaviors in preschoolers using kinematic variables, thereby assess the effects of age on them. Here, the efficiency of machine learning algorithm developed in our previous study (Patel et al., under preparation) was also tested for classifying these behaviors on time series data from children. By understanding how underlying movements in these two behaviors modulate with age, it is 45 possible to get insights on the developmental trajectory of rotation and throwing behaviors. The movements are characterized based on two key parameters: variability and fluency. Based on previous literature, we expect the movements to get fluent and show decrease in variability with increasing age. Method Participants Thirty-three typically developing children participated, 10 participants in 3-year old group (7 females; mean age = 3.38 + 0.24 years), 11 participants in 4-year old (5 females; mean age = 4.47 + 0.35 years) and 12 participants in 5-year old (3 females; mean age = 5.61 + 0.24 years). Data collection involved one home-based experimental session. Caregivers of the participants assisted with the data collection process and provided written informed consent for their child’s participation. Informed assent was obtained from children aged 5 years and above before their participation. All the procedures were approved by the Michigan State University human research protection program. Participants received a $25 gift card for their participation. Apparatus Five experimental objects (4 exploration toys + 1 IMU) were used for the participants to perform the three behaviors of interest. These experimental objects were made of firm Styrofoam in two different shapes (cube vs. ball) and two different sizes (5 cm vs. 10 cm) (refer figure 3.1a). Objects of different shapes and sizes were used to capture within participant variability in performing same behaviors. Each object had a slit like cavity in the center to snugly fit a sensor in it (refer figures 3.1b, 3.1c and 3.1d). These sensors were wireless inertial measurement units (IMUs) (3-space mini-Bluetooth sensors, YEI Technology, Ohio USA) to measure movement data. They were light in weight, 3 cm x 3 cm x 1.3 cm in size and comprise of accelerometer, gyroscope, and magnetometer. Additionally, objects were covered with custom made sleeves to 46 hide the sensor from the participants during the trials so that their behaviors were not curtailed or influenced by the sensor placement but performed with a focus on the object. Further on, we used Zoom to virtually provide instructions and communicate with the participants and their caregivers (refer figures 3.2a and 3.2b). We recorded this video call for data analysis purposes and used TeamViewer to remotely control the computer on the participant end to operate the sensors in the objects. Protocol Participants performed the sequence trials similar to our previous study (Patel et al., under preparation). In each sequence trial, they performed three manual exploratory behaviors (rotation, throwing and fingering) one after another in a random sequence using five experimental objects (4 experimental objects + 1 IMU) (refer figure 4.1). Since we wanted to assess the efficiency of the machine learning algorithms from our previous study in which they were trained and tested using adult dataset (Patel et al., under preparation), children in this study also performed fingering behavior in addition to rotation and throwing. Participants could perform these behaviors as many times as they wanted during each trial but experimenter made sure that each behavior was performed at least once per trial. There was one block of 5 sequence trials in each data collection session, one sequence trial per object. Each trial lasted around 1 minute. Caregivers assisted with data collection by providing and taking back the experimental object from the child at the beginning and end of each trial at the experimenter’s cue. 47 Figure 4.1. Study 2 protocol. There were 5 sequence trials performed in one block per participant. In each sequence trial, participants performed three manual exploratory behaviors (rotation, throwing and fingering) one after another in a random sequence. Data Analysis ML -based classification Before using ML classifiers to classify the three behaviors on time series, all the data was pre-processed in the same way as the training data was processed in our previous study (Patel et al., under preparation). This involved filtering the data, segmenting into 1 second windows, and calculating features for each window. Such pre-processing was required as machine learning classifiers can only be used on data that have a similar format to the data used for their training purpose. The machine learning classifiers were then used to classify time series in each sequence trial according to the type of behavior. The ground truth required for assessing classifier accuracy was established by video coding sequence trials in the recorded zoom call using the behavioral coding tool- Datavyu (www.datavyu.org). Quantitative analysis We conducted quantitative analysis on time series corresponding to rotation and throwing behaviors for all participants. Rotation was characterized in terms of variability, smoothness and speed using three kinematic variables- (1) rotational variability, (2) dimensionless angular jerk, and (3) angular velocity profiles (peak and mean) respectively. For throwing, two kinematic 48 variables were used- (1) dimensionless jerk and (2) acceleration profiles (peak and mean) that provided insights on throwing movement smoothness and speed. All data were filtered using a 4th order Butterworth filter before conducting quantitative analysis. Rotational variability. The variability in rotation along different axes was measured using principal component analysis of Euler angles which gave three eigenvectors and corresponding eigenvalues. A simple planar rotation can be represented by two eigen vectors along the plane of rotation; any rotation away from this plane results in an increase in the eigenvalue of the third vector. Using this property, we can utilize the eigenvalue of the smallest eigenvector as a measure of rotational variability. Rotational Variability closer to 1 indicates highly variable rotation, while its value closer to 0 indicates less variable rotational pattern wherein rotation is performed across a single axis. Dimensionless angular jerk. For rotation, resultant angular velocity of rotational behaviors was used to measure angular jerk. It was calculated as: 𝑀𝑇 𝐽(𝑡)2 𝑀𝑇 3 √∫ 𝑑𝑡 ∗ ( 2 ) 0 2 𝜔𝑝𝑒𝑎𝑘 where, J(t) is the derivative of the recorded angular acceleration, MT is the duration of total rotational behavior and 𝜔peak is the peak angular velocity during that rotation. 2 Dimensionless jerk was calculated by normalizing jerk using the quantity (MT3 /𝜔𝑝𝑒𝑎𝑘 ) (Hogan & Sternad, 2009), which makes the measurement of smoothness less biased with changes in the overall movement time (Lee & Newell, 2012; Rohrer et al., 2002). Angular velocity profiles. Mean and peak angular velocity were calculated using the resultant angular velocity in each rotation behavior. Dimensionless linear jerk. For throwing, resultant linear acceleration was used to calculate movement smoothness index- dimensionless linear jerk. It was calculated as: 49 𝑀𝑇 𝐽(𝑡)2 𝑀𝑇 √∫ 𝑑𝑡 ∗ ( 2 ) 0 2 𝑎𝑝𝑒𝑎𝑘 where, J(t) is the derivative of the recorded linear acceleration, MT is the duration of total movement in a throwing behavior and apeak is the peak acceleration during that throw. Since our raw data for linear movements was instantaneous acceleration, we calculated modified dimensionless jerk in which jerk was normalized by a quantity (MT/a2peak) (Patel et al., 2019). Linear acceleration profiles. Mean and peak linear acceleration were calculated using the resultant linear acceleration in each throwing behavior. Statistical analysis We analyzed the effect of age on the kinematic characteristics of rotation and throwing behaviors using a one-way ANOVA on the factor of age (3 levels – 3-, 4- and 5-year old) for the dependent variables: rotational variability, dimensionless angular jerk, angular velocity profiles (mean and peak), dimensionless linear jerk and linear acceleration profiles (mean and peak). Post-hoc comparisons were made using Tukey’s post hoc test with the significance level set as p < .05. In situations where sphericity was violated, we employed the Greenhouse – Geisser correction. Results Descriptive measures Data from 30 participants was included for data analysis, 10 per each age group (3-year group - 7 females, mean age = 3.38 + 0.24 years; 4-year group – 5 females, mean age = 4.51 + 0.34 years; 5-year group – 3 females, mean age = 5.61 + 0.23 years). Data from 3 participants was excluded due to technical issues such as corrupt video files, poor internet connection that caused unreliable synchronization of video and sensor data. A total of 150 sequence trials were 50 collected from 30 participants. Participants performed all three behaviors at least once per sequence trial. ML -based classification performance Performance of the best classifier models- Support Vector Machine (SVM) Quadratic, SVM Medium Gaussian and Narrow Neural Network, from our previous study (Patel et al., under preparation) were assessed to identify the three behaviors on time series of sequence trials collected from children. All three classifier models had poor performance on all performance metrics. Performance metrics of SVM Quadratic were: accuracy = 60.10%, average precision (%) = 55.13 + 21.43%, average recall (%) = 55.93 + 15.54%, and average f1-score (%) = 55.22 + 18.36%. Performance metrics of SVM Medium Gaussian were: accuracy = 59.80%, average precision (%) = 56.03 + 21.53%, average recall (%) = 57.32 + 12.26%, average f1-score (%) = 55.91 + 16.59%. Performance metrics of Narrow Neural Network were: accuracy = 58.84%, average precision (%) = 55.0 + 22.26%, average recall (%) = 55.73 + 14.64%, and average f1 score (%) = 54.99 + 18.57%. Analyzing the precision, recall and f1-scores for each of the three behaviors showed that the classifiers showed substantially poor performance for classifying throwing behaviors compared to the other two behaviors for which the performances were moderate. This mainly accounted for their overall performance reduction. Figure 4.2 demonstrates the precision, recall and f1-scores of all three models across the three behaviors. 51 Figure 4.2. (a) Precision (%) (b) recall (%) and (c) f1-scores of the three ML classifiers for classifying fingering, rotation and throwing behaviors. Classifiers showed substantially poor performance for classifying throwing behaviors compared to the other two behaviors for which the performances were moderate. This mainly accounted for their overall performance reduction. Quantitative analysis The kinematic variables for rotation and throwing behaviors as a function of age are summarized in Table 4.1. Since the ML classifiers showed low accuracy in classifying the three 52 behaviors, conventional video coding method was used to identify the start and end times of these behaviors on time series for conducting their quantitative analysis. 1. Rotational variability One way ANOVA confirmed that there was no significant effect of age on the rotational variability, F(2, 303) = 2.41, p = .091. However, the average rotational variability was lower in the 3-year group compared to the 4-year group and 5-year group (refer table 4.1 and figure 4.3a). The frequency distribution of the rotational variability in the three age groups indicated that children in the 5-year group performed the most variable rotations, children in the 4-year group had a mixture of variable and monotonous rotations while the 3-year olds mainly performed rotations along one axis (refer figure 4.3b). Table 4.1. Mean (SD) of quantitative characteristics of rotation and throwing behaviors as a function of age Behavior Kinematic variable Age group 3-year 4-year 5-year Rotation Rotational variability 0.26 (0.22) 0.32 (0.21) 0.33 (0.24) Dimensionless angular jerk 286.81 442.25 496.62 (579.53) (601.95) (765.28) Peak angular velocity (rad/s) 18.44 (10.79) 23.04 (12.19) 23.07 (15.67) Average angular velocity (rad/s) 5.49 (4.12) 6.53 (5.39) 6.36 (4.55) Throwing Dimensionless linear jerk 3.17 (1.43) 3.52 (1.66) 3.678 (1.64) 2 Peak linear acceleration (m/s ) 2.21 (0.09) 2.14 (0.12) 2.198 (0.12) 2 Average linear acceleration (m/s ) 0.79 (0.04) 0.69 (0.05) 0.76 (0.05) 2. Dimensionless angular jerk One way ANOVA confirmed that there was no significant effect of age on the dimensionless angular jerk, F(2, 303) = 2.86, p = .059. However, the average dimensionless angular jerk was highest in the 5-year group and lowest in the 3-year group (refer table 4.1 and figure 4.3c). This finding supports the high rotational variability observed in the older children. 53 Figure 4.3. (a) Average rotational variability as a function of age used for statistical analysis. There was no main effect of age on the rotational variability across three ages. (b) Frequency distribution of rotational variability plotted for the three age groups. Rotations with low variability were highest in the 3-year group while those with high variability were found more in the 5-year group. (c) Average dimensionless angular jerk as a function of age used for statistical analysis. There was no main effect of age on the angular jerk, but average angular jerk was highest in the 5-year group and lowest in the 3-year group. (d) Average dimensionless linear jerk as a function of age used for statistical analysis. There was no main effect of age on the angular jerk. Error bars represent one standard deviation. 3. Angular velocity profiles One way ANOVA confirmed a significant effect of age on the peak angular velocity, F(2, 303) = 4.29, p = .014, but no significant difference on the average angular velocity, F(2, 303) = 1.41, p = .245. The average angular velocity and the peak angular velocity were both lower in the 3-year group compared to the 4- and 5- year groups (refer table 4.1). Post-hoc comparisons showed that the peak angular velocity in the 3-year group was significantly lower than in the 4- year group and 5-year group. 54 4. Dimensionless linear jerk One way ANOVA confirmed that there was no significant effect of age on the dimensionless linear jerk, F(2, 197) = 2.01, p = .137. The dimensionless linear jerk was highest in the 5-year group and lowest in the 3-year group (refer table 4.1 and figure 4.3d). 5. Linear acceleration profiles One way ANOVA confirmed that there was no significant effect of age on the average linear acceleration, F(2, 197) = 1.17, p = .314 and peak linear acceleration, F(2, 197) = 0.09, p = .914. Overall, average linear acceleration and peak linear acceleration were both slightly higher in the 3-year group compared to the other two age groups (refer table 4.1). Discussion The aim of this study was to investigate two functionally and clinically important manual exploratory behaviors- rotation and throwing seen in early childhood using quantitative analysis and assess an effect of age on their characteristics. Here, we also assessed our ML classifiers from previous study (Patel et al., under preparation) to classify the three behaviors (rotation, throwing and fingering) in children. Our classifiers had moderate performance in classifying rotation and fingering behaviors, but substantially poor for throwing behaviors which resulted in an overall poor performance (average accuracy = 59.86 + 0.21%). In terms of age effect, rotation behaviors had subtle differences in the group averages of kinematic characteristics but throwing behaviors appeared very similar across three age groups. For rotation behaviors, the 3-year group had less rotational variability, angular jerk and angular velocity profiles, however a significant main effect was only observed in peak angular velocity. For throwing behaviors, characteristics across the three ages appeared very similar with linear jerk being marginally low while acceleration profiles slightly higher in the 3-year group, however, no significant main effect of 55 age was observed for any throwing characteristics. These findings indicated subtle developmental differences in the movements of 3-5 years old children. We expected the ML classifiers trained on adult dataset in our previous study (Patel et al., under preparation) to have performance reduction while classifying the three behaviors in children owing to the differences in the training and testing datasets. These classifiers were trained using dataset from adults who are expected to have differences in the way they perform these behaviors from children. Evidence exists from previous literature indicating developmental differences in movements across lifespan, especially between children and adults (Olivier et al., 2007; Ranganathan et al., 2019; Simon-Martinez et al., 2018; Yan et al., 2000). In fact, this study itself showed differences between the rotations and throwing behaviors across 3 – 5 year old. Even though most of these differences were not significant, the age modulated changes in movements are expected to increase with increase in age differences. Thus, the ML classifiers ought to have a performance reduction when tested on the data from children owing to the developmental differences in the movements of children and adults. In our view, the technical difficulty encountered during remote data collection with children was the primary reason for poor performance of ML classifiers. The video data accuracy was important in both studies since it was used to set the ground truth, however, adult behaviors appeared more controlled, clean and predictable on video data compared to children. Although the data collection protocols were the same, adults were more likely to follow instructions on when and where to perform the behaviors whereas it was difficult in case of children. A common problem encountered in child dataset was to identify the start of a behavior since children would begin performing without a clear heads-up which led to insufficient transition phase between two behaviors. This preparatory phase for a behavior is important for both- setting ground truth and ML classification. This problem was amplified in shorter duration behaviors such as throwing. 56 This mainly resulted in the substantially low performance of ML classifiers in classifying throwing behaviors compared to other two behaviors that were performed for longer durations. However, this technical difficulty is relatively manageable when the data collection is conducted in-person which is a more controlled environment than via remote video call. Our quantitative findings on rotation behaviors showed an increase in their variability with increasing age: the 3-year group was more likely to rotate objects along a single axis but older children performed more multi-axial rotations. This finding did not reach statistical significance but group averages and frequency distribution of rotational variability suggested a subtle yet salient increase in variability with increasing age. For behaviors such as reaching, banging, striking, etc., a decrease in variability is considered a sign of learning in which behaviors get efficient and fine-tuned with time (Kahrs et al., 2012; Lee et al., 2011). However, rotations are different from these behaviors wherein even planning and executing rotation in an object fitting task requires a certain skill level (bimanual coordination, mental rotation, etc.) and is found to improve with increasing age (Jung et al., 2015, 2018; Lockman et al., 2018; L. B. Smith et al., 2014; Street et al., 2011). Based on this, it can be implied that rotating objects along different axes is a function of increased skill level that requires adequate bimanual coordination which is further constrained by the individual (hand size, etc.) and task factors such as object physical properties (size, shape, etc.) (Newell, 1986; Newell et al., 1989). Such variable rotations indicate a higher skill level in which an organism has a better control of holding an object while simultaneously rotating it in different directions. This nature of rotational behaviors is in line with our findings wherein older children afforded to perform more variable rotations compared to younger ones. Similar to rotational variability, angular jerk did not show a main effect of age but the group averages showed an increase with increasing age. Angular jerk is a derivative of angular 57 acceleration, so frequent change in acceleration will result in higher jerk. Multi-axial rotations are expected to have dynamic acceleration as compared to those performed along one axis to afford object control while moving it in different directions. This explains our findings in which older children who performed more multi-axial rotations also had higher angular jerk. Our findings also suggest that angular jerk in rotational behaviors is not just a direct measure of movement smoothness but also provides insights on the rotational variability. Moreover, the angular velocity profiles for rotational behaviors also showed increase with increase in age; significant main effect of age on peak angular velocity. Similar to rotational variability, rotational speed provides information on the skill level of the performer. A fast rotation will require a good grasp of the object along with strong bimanual coordination in order to rotate without dropping it. Our findings on angular velocity profiles combined with rotational variability results indicate that older children showed faster and more variable rotations which in turn may indicate that rotations became more skillful with increasing age. Quantitative findings on throwing behaviors were inconclusive of developmental differences across the three ages with no significant main effect of age and minimal differences in their group averages. Based on previous studies (Kahrs et al., 2012; Lee et al., 2011), we expected to have an age effect specifically on the smoothness of movement (measured using dimensionless linear jerk) between 3 and 5 years of age. These studies found that the behaviors got fine-tuned and had smooth movements with increasing age. However, a key difference between the current study and previous ones is the difference in the age group and behaviors under investigation. Lee and colleagues (2011) measured the smoothness of movements in pre- reaching arm movements of infants while Kahrs and colleagues (2012, 2013, 2014) assessed banging behaviors in 6 – 15 month olds. These studies found developmental differences as the behaviors were assessed during early and late learning periods. Given that throwing behaviors 58 are first seen during late infancy (Ruff, 1984), assessing them only between 3 – 5 years of age does not capture the entire learning period. Children in 3 – 5 years of age are still learning to throw but they are not at the beginner’s level. Thus, their throwing behaviors will have developmental differences but as seen in our findings they are subtle. In conclusion, the children between 3 – 5 years of age have similar throwing patterns, so they can be grouped as one while investigating the developmental trajectory of throwing during their entire childhood. In summary, the current study is the first account where ML -based classifiers were tested for classifying three important manual exploratory behaviors in children as well as provide insights on their quantitative characteristics. The reasons for poor performance of the ML –based classification system seem to be fairly manageable and resolving them may lead to a successful implementation of this method for classifying manual exploratory behaviors and facilitate their quantitative assessment. Further on, subtle yet salient developmental differences found in the rotational movements of 3 – 5 year olds indicate that its quantitative characterization should be extended to all ages in early childhood. Such quantitative analysis can fill the literature gaps on how acts of exploration and the underlying movements transition into the development of highly skilled motor behaviors such as “tool use”. In addition, kinematic variables such as rotational variability used in this study to quantify rotational behaviors in preschoolers can be used to reliably quantify the stereotypical/ repetitive rotations seen in infants and toddlers at high risk for ASD. 59 CHAPTER 5. EFFECT OF AGE AND OBJECT PROPERTIES ON THE CHARACTERISTICS OF MANUAL EXPLORATORY BEHAVIORS IN EARLY CHILDHOOD Abstract Manual exploratory behaviors that lay the foundation of “tool use” continue beyond infancy as the perception of object properties are still maturing throughout the early childhood. Functionally, exploratory behavioral changes as a function of object properties throughout early childhood provides insights on the development of tool use, while clinically it opens up a window for potential biomarkers for early detection of ASD. Thus, we aim to conduct a qualitative characterization of manual exploratory behaviors under the effect of age and object physical properties (size, texture and shape) during early childhood. For this, manual exploratory behaviors were remotely observed in 30 preschoolers (3 – 5 years) for different objects that were a combination of two different sizes, shapes, and textures. Data was simultaneously collected by recording the video call. The qualitative findings indicated an effect of age on the exploratory behaviors of the preschoolers- 3-year olds predominantly performed throwing type behaviors while 5-year olds demonstrated more skillful behaviors (rotation). In terms of object properties, an effect of object size and shape mainly influenced participants’ hand preference during reaching while object size and texture affected type of exploratory behaviors. Our qualitative findings not just explain how children interact with objects in this age group but provide insights on the factors that primarily modulate manual exploratory behaviors in preschoolers. 60 Introduction Manual explorations of objects seen during early childhood are amongst those critical behaviors that lay the foundation of tool use, one of the most advanced human motor skills (Goldfield, 1995; Lockman, 2000; Thelen, 1981). This skill develops systematically through actively exploring objects and learning to perceive their properties (Gibson, 1988; Thelen, 1981). Infants transition from mouthing to purposeful play gradually and systematically from the innumerable active interactions with the objects that help them perceive the use of objects and how to interact with them effectively (Lockman, 2000; Lockman & Kahrs, 2017). A notable aspect of this dynamic interplay between exploratory actions and perception of object properties is that it continues past infancy, throughout early childhood. Manual exploratory behaviors similar to other motor skills show a change with increasing age. Infants that spend majority of their time mouthing on objects during early infancy, interact with the same objects in a variety of different ways by the end of their first year (Ruff, 1984). With increasing age, they perform a variety of different exploratory behaviors such as shaking, banging, rolling, etc. While some may argue these behaviors to be indicative of the purposeful play past infancy, the interplay between the perceptual skills and corresponding exploratory actions is still ongoing beyond the first year. In fact, certain perceptual skills such as object weight start to emerge at the end of the first year of life (Paulus & Hauf, 2011). As a result, behaviors that seem to be purposeful play during early childhood still count as part of the manual exploratory behaviors as they are contributing to the development of perceptual skills in early childhood. However, manual exploratory behaviors are mainly characterized during infancy (Corbetta & Snapp-Childs, 2009; Corbetta & Thelen, 1996; Lee et al., 2006; Needham et al., 2017; Newell et al., 1989; Ruff, 1984) instead of the entire early childhood (first five years of age). 61 Perception of different object properties influence the way infants interact with objects and these effects change across different times during infancy. Effects of object size, shape, texture, orientation, etc. on the manual exploratory behaviors have been assessed by a number of studies during infancy (Corbetta et al., 2000; Corbetta & Thelen, 1996; Lee et al., 2006; Newell et al., 1989; Ruff, 1984). These studies provide insights on the developmental trajectory of the perceptual skills related to object properties during infancy. However, it should be emphasized that the perceptual skills related to different object properties emerge at different times, resulting in a distinct developmental trajectory. This makes it crucial to assess the evolving perceptual skills related to different object properties that influence a child’s exploratory behaviors during entire early childhood. Such assessment can provide information on the object properties that children rely on while exploring a new object or use more often than other properties for exploration purpose. Such information is important to assess during early childhood in order to fully understand the development of perceptual-motor system which in turn is pivotal to the development of tool use in children. The effects of different object properties on manual exploratory behaviors during early childhood are particularly important from a functional perspective wherein these behaviors precede the emergence and development of different tool use skills. The dynamic interplay between manual exploratory behaviors and learning to perceive object properties is expected to play a pivotal role in how children learn different tool use skills. For instance, ‘banging’ behaviors observed in late infancy are directly linked to the development of important tool use skill- ‘hammering’ (Lockman, 2000; Lockman & Kahrs, 2017). Children are found to modulate banging actions according to different object properties such as texture of the surface, object shape, etc. (Kahrs et al., 2012, 2013, 2014). This in turn provides insights on the developmental trajectory of hammering and how it evolves and matures during early childhood. 62 Although all manual exploratory behaviors do not directly transition into a tool use skill like banging, they do play important role in understanding the function of object and interacting with them efficiently. For instance, rotation behaviors do not transition directly into a tool use skill but they help children to learn the 3D nature of an object (Soska et al., 2010) which in turn is pivotal to the development of tool use motor skill. Thus, assessing the dynamic interplay between manual exploratory behaviors and perception of object properties at different time points during early childhood provides insights on the developing perceptual-motor system. This in turn contributes to our understanding on how children learn different tool use skills. However, the effects of object properties on manual exploratory behaviors are mainly assessed during first year of life and yet to be investigated beyond infancy and throughout early childhood. In this study, we address this literature gap by assessing the manual exploratory behaviors in early childhood under the effect of different physical object properties- size, shape and texture. We investigated these behaviors in preschoolers aged 3-5 years old using objects that are a combination of different physical properties (size, shape, and texture). Based on previous literature, we expect object size to influence exploratory behaviors in children more than object shape and texture (Lee et al., 2006; Newell et al., 1989; Ruff, 1984). We also expect to see an effect of age wherein older children are likely to perform actions based on more than one object property compared to the younger ones. Method Participants Thirty-three typically developing children participated, 10 participants in 3-year old group (7 females; mean age = 3.38 + 0.24 years), 11 participants in 4-year old (5 females; mean age = 4.47 + 0.35 years) and 12 participants in 5-year old (3 females; mean age = 5.61 + 0.24 years). Data collection involved one home-based remote data collection. Caregivers of the 63 participants assisted with the data collection process and provided written informed consent for their child’s participation. Informed assent was obtained from children aged 5 years and above before their participation. All the procedures were approved by the Michigan State University human research protection program. Participants received a $25 gift card for their participation. Apparatus We used 9 experimental objects (8 exploratory toys + 1 IMU) for the participants to explore and perform various manual exploratory behaviors (refer figure 5.1a). The eight experimental objects were a combination of three object physical properties- size (5 cm vs. 10 cm), shape (ball vs. cube) and texture (soft vs. hard). These objects were- 5 cm hard ball, 5 cm hard cube, 5 cm soft ball, 5 cm soft cube, 10 cm hard ball, 10 cm hard cube, 10 cm soft ball and 10 cm soft cube. Hard objects were made of firm Styrofoam while soft objects were made of sponge Styrofoam. Each object had a cavity in the center to snugly fit a sensor in it (refer figures 3.1b, 3.1c and 3.1d). The sensors were wireless Inertial Measurement Units (IMUs) (3-space mini-Bluetooth sensors, YEI Technology, Ohio USA) to measure movement data. They were light in weight, 3 cm x 3 cm x 1.3 cm in size and comprised of accelerometer, gyroscope and magnetometer. Additionally, objects were covered with custom made red colored sleeves. We used Zoom to virtually provide instructions and simultaneously record the video call for behavioral data analysis (refer figures 3.2a and 3.2b). Protocol Participants explored 9 experimental objects (8 exploratory toys + 1 IMU) one at a time in a random order. Participant’s caregiver assisted data collection by giving and taking back the experimental objects from the child at the beginning and end of each trial. Experimenter cued caregiver about the start and end of a trial as well as which object to give to the child. 64 Each data collection session had two blocks of 9 trials each, one trial for each experimental object. Each trial lasted for around 45-60 seconds. Participants explored all the 9 experimental objects once per block and twice across two blocks (refer figure 5.1b). Objects were given in a random order using a randomly generated sequence which was different for each block and across all the participants. 65 Figure 5.1. (a) Nine experimental objects given to participants one at a time in a random order. Eight objects were a combination of two different sizes, texture and shape- 10 cm hard cube, 10 cm hard ball, 5 cm hard ball, 5 cm hard cube, 10 cm soft cube, 10 cm soft ball, 5 cm soft ball and 5 cm soft cube. Ninth object was the sensor itself. (b) Protocol for study 3. There were 2 blocks per participant, 9 trials per block and 1 object per trial. Participants freely explored each object twice across two blocks. 66 Data Analysis Qualitative analysis All the behavioral data from the recorded video call was coded using the behavioral coding tool- Datavyu (www.datavyu.org). Since the data collection was conducted remotely over a Zoom call, we expected events to occur during the video call that did not yield usable data. Thus, we discarded video data for durations in which: (1) child was not interacting with the object, (2) object was not visible, (3) caregiver was in contact or interacting with the object. Such data scanning ensured that video data was filtered for utilization of only those parts in which behaviors under investigation were guaranteed to occur and that the coder had adequate visual information of the participant’s interaction with the objects. All the filtered video data was first coded for participant’s hand preference during reaching for an object. For every reaching attempt, we coded whether participants used one hand, or two hands (Lee et al., 2006, 2011). Once participants had grasped the object, we coded for the nature of their interaction with the object that involved presence of different manual exploratory behaviors. Since there is a wide variety of manual exploratory behaviors observed in early childhood, we used three operational categories depending on previous literature (Lee et al., 2011) to categorize them for data analysis purposes. These categories were: (1) transportation activities in which the behavior was aimed to change the position of the object, (2) wrist activities in which behaviors were mainly produced by the wrist joint movements and (3) finger activities in which behaviors were mainly produced by finger movements. Manual exploratory behaviors in each of the three categories are listed in table 5.1. We coded participants' interaction with the objects for: (1) the manual exploratory behavior and (2) its category. Two independent coders- the primary coder coded 100% of all trials and the secondary coder coded 25% of all 67 trials for the frequency of hand preference (one hand vs. two hands) during every reach, manual exploratory behavior and its category (Adolph et al., 2012; Franchak & Adolph, 2012). Table 5.1. Summary of manual exploratory behaviors divided into operational behavior categories. Behavior Category Finger activities Wrist activities Transportation activities Squeezing Rotation Throwing/pushing Fingering/scratching Shaking Moving object randomly Picking surface Banging Transferring object between hands Juggling Statistical analysis We examined the effects of age and the three object properties on: (1) hand preference during reaching an object, (2) initial interactions with an object during a trial, and (3) total manual exploratory behaviors performed during a trial. Initial interactions with a new object typically result in use of highly variable manual exploratory behaviors and the change in type of these behaviors gradually reduces resulting in behaviors that are deemed more object appropriate. So, we assumed that the behavioral category trends during initial interactions will be dynamic and different compared to the overall behavioral category trends for the given duration of interaction with an object. Based on this assumption we identified the time point in a trial until which behaviors can be considered as initial interactions by examining the behavioral category trends during different times in a trial against the overall behavioral category trends of the entire trial across three age groups. We used a 3 (age group) x 2 (object size) x 2 (object shape) x 2 (object texture) repeated measures ANOVA on the following dependent variables: one hand preference during reaching, two hands preference during reaching, finger activities, wrist activities and transportation activities. All the dependent variables were measured in terms of their frequency percentage of occurrence. Post-hoc comparisons were made using Tukey’s post hoc test with the significance 68 level set as p < .05. In situations where sphericity was violated, we employed the Greenhouse – Geisser correction. Results Descriptive measures Data from 30 participants was included for data analysis, 10 per each age group (3-year group - 7 females, mean age = 3.38 + 0.24 years; 4-year group – 5 females, mean age = 4.51 + 0.34 years; 5-year group – 3 females, mean age = 5.61 + 0.23). Data from 3 out of 33 participants was excluded due to technical issues such as corrupt video file, poor internet connection that affected video data quality. All the 30 participants included in the data analysis completed the entire study protocol; data were analyzed from a total of 540 trials, 18 trials per participant. Average duration of two blocks across three age groups was: 3-year group = 692.12 + 71.14 s, 4-year group = 738.60 + 77.98 s and 5-year group = 760.42 + 52.33 s. The percentage duration of video data discarded across three age groups was: 3-year group = 22.75 + 10.95%, 4- year group = 17.56 + 8.47% and 5-year group = 17.84 + 9.56%. The inter-rater reliability between the two coders for coding the frequency of hand preference during reaching an object was high (the Kappa agreement was .98). It was generally high for the type of manual exploratory behavior (the Kappa agreement was .86) and its category (the Kappa agreement was .92). Hand preference during reaching an object: frequency percentage of 1 hand vs. 2 hands The percentage of hand preference frequency during reaching an object across three age groups as a function of object size, texture and shape is depicted in Figure 5.2. We did not find main effect of age on the participant’s use of 1 hand vs. 2 hands for reaching objects (F(2, 27) = 2.49, p = .102). We found a main effect of size (F(1, 27) = 720.15, p = <.001) in which participants reached for small objects with one hand while using two hands for large objects. We 69 also found a main effect of shape (F(1, 27) = 11.76, p = .002) in which participants were more likely to use one hand for reaching ball shaped objects but two hands for cube shaped objects. There was no main effect of texture (F(1, 27) = 0.19, p = .667) but interaction between texture x shape was significant (F(1, 27) = 4.98, p = .034). Post-hoc comparisons showed that participants significantly used 1 hand to reach soft and ball shaped objects compared to soft and cube shaped objects. The interaction between size x shape x age was also significant (F (2, 27) = 3.94, p = .032) and its post-hoc comparisons showed that participants in all three age groups significantly used one hand to reach small size cubes and balls compared to large size objects and 4-year old participants were more likely to use one hand to reach large objects as compared to 5-year old participants. Other than these two, there was no significant effect of interaction between size x age (F(2, 27) = 1.32, p = .283), texture x age (F(2, 27) = 0.70, p = .505), shape x age (F(2, 27) = 0.13, p = .877), size x texture (F(1, 27) = 1.70, p = .204), size x texture x age (F(2, 27) = 0.39, p = .680), size x shape (F(1, 27) = 2.60, p = .118), texture x shape x age (F(2, 27) = 0.72, p = .496), size x texture x shape (F(1, 27) = 0.52, p = .476), size x texture x shape x age (F(2, 27) = 0.28, p = .757) . 70 Figure 5.2. Average hand frequency (%) across three age groups used for statistical analysis as a function of (a) object size (b) object shape and (c) object texture. There was a significant effect of object size and shape on the hand preference of children whereas no effect of texture on their 1hand vs. 2hand reach pattern. Children used 1 hand reach more often for small and ball shaped objects and 2 hands reach for large and cube shaped objects. Error bars represent one standard deviation. Initial interactions with the object: frequency percentage of first three behaviors in a trial We assessed the behavior category trends starting from the first behavior up until the trends at two consecutive time points in a trial started to align with the total behavior category trends. We found that the behavior category trends started to align with the overall trends after the third behavior in a trial for all three age groups (refer figure 5.3). Thus, we used the behavior category frequency of the first three behaviors during each trial to examine the effects of age and object properties on initial interactions with an object. The average frequency percentage of 71 behavior category during these initial interactions across three age groups as a function of object size, texture and shape is depicted in Figure 5.4a, 5.4b and 5.4c. Figure 5.3. Time-point at which behaviors in a trial can be considered as initial interactions was identified by examining the behavioral category trends during different times in a trial against the overall behavioral category trends of a trial across three age groups. Behavior category frequency (%) plotted across three age groups for (a) first behavior in trial, (b) first two behaviors in a trial, (c) first three behaviors in a trial, (d) first four behaviors in a trial, (e) first five behaviors in a trial and (f) total behaviors in a trial. Behavior category trends started to align with the overall trends after the third behavior in a trial for all three age groups, so first three behaviors were considered as part of the initial interactions with an object. Finger activities showed a main effect of object size (F(1, 27) = 26.06, p = <.001) and texture (F(1, 27) = 10.12, p = .004), but no main effects of age (F(2, 27) = 0.50, p = .614) and object shape (F(1, 27) = 1.04, p = .317). Post-hoc comparisons showed finger activities occurred significantly more in small size and soft objects compared to large and hard objects. There was no statistically significant effect of interactions between size x age (F(2, 27) = 1.10, p = .347), 72 texture x age (F(2, 27) = 0.72, p = .498), shape x age (F(2, 27) = 0.89, p = .423), size x texture (F(1, 27) = 3.08, p = .090), size x texture x age (F(2, 27) = 1.02, p = .373), size x shape (F(1, 27) = 1.31, p = .262), size x shape x age (F(2, 27) = 0.94, p = .403), texture x shape (F(1, 27) = 0.25, p = .622), texture x shape x age (F(2, 27) = 0.95, p = .399), size x texture x shape (F(1, 27) = 0.09, p = .762), size x texture x shape x age (F(2, 27) = 0.35, p = .707). Wrist activities showed a main effect of object size (F(1, 27) = 25.28, p = <.001) and texture (F(1, 27) = 14.72, p = <.001), but no main effects of age (F(2, 27) = 0.84, p = .443) and object shape (F(1, 27) = 1.16, p = .290). There was a significant interaction between shape x age (F(2, 27) = 3.37, p = .049) but no significant interaction between size x age (F(2, 27) = 0.68, p = .517), texture x age (F(2, 27) = 0.40, p = .675), size x texture (F(1, 27) = 3.42, p = .075), size x texture x age (F(2, 27) = 0.67, p = .520), size x shape (F(1, 27) = 0.52, p = .479), size x shape x age (F(2, 27) = 0.37, p = .695), texture x shape (F(1, 27) = 0.15, p = .699), texture x shape x age (F(2, 27) = 0.18, p = .840), size x texture x shape (F(1, 27) = 0.89, p = .354), size x texture x shape x age (F(2, 27) = 0.01, p = .991). Post-hoc comparisons showed that participants performed wrist activities significantly more with objects that were large in size and had hard texture. Transportation activities showed a main effect of object size (F(1, 27) = 7.83, p = .009) and age (F(2, 27) = 3.49, p = .045), but no main effects of texture (F(1, 27) = 0.00, p = .962) and object shape (F(1, 27) = 0.60, p = .447). There was no statistically significant effect of interactions between size x age (F(2, 27) = 3.06, p = .064), texture x age (F(2, 27) = 2.16, p = .134), shape x age (F(2, 27) = 2.75, p = .082), size x texture (F(1, 27) = 0.10, p = .751), size x texture x age (F(2, 27) = 0.05, p = .950), size x shape (F(1, 27) = 3.07, p = .091), size x shape x age (F(2, 27) = 0.59, p = .561), texture x shape (F(1, 27) = 0.10, p = .757), texture x shape x age (F(2, 27) = 0.34, p = .716), size x texture x shape (F(1, 27) = 0.06, p = .807), size x texture x 73 shape x age (F(2, 27) = 0.10, p = .904). Post-hoc comparisons showed that participants performed transportation activities significantly more with small objects compared to large ones. Also, the 3-year old group performed transportation activities significantly more than the 5-year group. Manual exploratory behaviors performed with each object: frequency percentage of total behavior category during each trial The average frequency percentage of behavior category during each trial across three age groups as a function of object size, texture and shape is depicted in Figure 5.4d, 5.4e and 5.4f. A summary of the F and p values of all the repeated measures ANOVAs performed on the different behavior category are in Table 5.2. Table 5.2. Summary of the results of the repeated measures of analysis of variance on the three behavior categories as a function of age, object size, texture and shape Transportation Effect Finger activities Wrist activities activities Age d.f. 2 2 2 F 2.43 1.67 5* p .107 .207 .014 Size d.f. 1 1 1 * * F 17.09 31.60 16.75* p <.001 <.001 <.001 Texture d.f. 1 1 1 * * F 31.78 17.55 0.57 p <.001 <.001 .457 Shape d.f. 1 1 1 F 1.02 2.45 0.02 p .320 .129 .877 Age x size d.f. 2 2 2 F 1.10 0.71 1.26 p .344 .498 .300 Age x texture d.f. 2 2 2 * F 3.16 0.67 0.85 p .058 .517 .075 74 Table 5.2. (cont’d) Age x shape d.f. 2 2 2 F 1.14 3.70* 1.42 p .335 .038 .260 Size x Texture d.f. 1 1 1 F 0.77 3.28 0.61 p .387 .081 .440 Size x shape d.f. 1 1 1 F 5.04* 4.50e-4 2.39 p .033 .983 .134 Texture x shape d.f. 1 1 1 F 0.23 0.10 0.31 p .631 .748 .581 Size x texture x age d.f. 2 2 2 F 0.55 0.39 0.71 p .583 .675 .499 Size x shape x age d.f. 2 2 2 F 0.77 0.06 0.00 p .472 .938 .996 Texture x shape x age d.f. 2 2 2 F 0.35 0.43 1.23 p .702 .651 .308 Size x texture x shape d.f. 1 1 1 F 0.50 0.20 0.26 p .483 .655 .613 Size x texture x shape x age d.f. 2 2 2 F 1.02 0.58 1.26 p .374 .567 .299 * Note. Significant at p<0.05. Finger activities occurred significantly more in small size and soft objects compared to large and hard objects. In addition, they also occurred significantly more in small size cubes compared to large size cubes. However, their average frequency percentage as a function of object size, texture and shape was substantially lower than that observed in initial interactions 75 across all three age groups (Refer figure 5.4). Of all finger activities, fingering behaviors were more common in the 3-year group (average frequency percentage = 58.80%) and 4-year group (average frequency percentage = 55.55%), while squeezing behaviors were more common in the 5-year group (average frequency percentage = 60.79%). Wrist activities were performed significantly more with large size and hard texture objects. The 5-year group performed these activities significantly more with ball shaped objects than cubes. Their average frequency percentage as a function of object size, texture and shape were not notably different from that observed in initial interactions across all three age groups (Refer figure 5.4). Of all the wrist activities, rotation behaviors were most common across all ages: 3-year group (average frequency percentage =61.88%), 4-year group (average frequency percentage = 64.39%) and 5-year group (average frequency percentage = 60.39%). Transportation activities were performed significantly more with small objects compared to large ones. Three-year group performed transportation activities significantly more than the 5- year group. Contrary to finger activities, the average frequency percentage of transportation activities as a function of object size, texture and shape was notably higher than that observed in the initial interactions across the three age groups (Refer figure 5.4). Of all the transportation activities, throwing behaviors were most common across all ages: 3-year group (average frequency percentage = 71.26%), 4-year group (average frequency percentage = 62.37%) and 5- year group (average frequency percentage = 78.56%). 76 Figure 5.4. Average behavior category frequency (%) during initial interactions across three age groups used for statistical analysis as a function of (a) size (b) texture and (c) shape. There was a significant effect of object size and texture on the initial interactions of children with objects. Average behavior category frequency (%) during all interactions with an object across three age groups used for statistical analysis as a function of (d) size (e) texture and (f) shape. Similar to initial interactions, there was a significant effect of object size and texture on the total exploratory behaviors of children. There was an age effect on the transportation activities during initial and total interactions where children in the 3-year group performed more throwing behaviors than older children. The average frequency (%) of finger activities was substantially higher during initial interactions compared to total exploration across all ages. The average frequency (%) of transportation activities was substantially lower during initial interactions compared to total exploration across all ages. Error bars represent one standard deviation. Discussion The aim of this study was to assess the effects of age and three object properties (size, shape and texture) on the manual exploratory behaviors in children aged from 3 – 5 years of age. We found that object exploration in preschoolers was driven at different levels by age and object properties; in line with previous reaching and manual exploration studies in infants (Corbetta & Snapp-Childs, 2009; Lee et al., 2006; Lee & Newell, 2013; Newell et al., 1989; Ruff, 1984). In 77 terms of age, throwing type transportation activities were more common in the 3-year group while the 5-year group showed more skilled behaviors like rotations. Of the three object properties, object size directed participants' hand preference in reaching objects, their first interactions as well as total manual exploratory behaviors with the object. In addition, participants used object shape information for reaching an object but object texture for exploring objects. Participants specifically used object texture using squeezing and fingering behaviors during their first interactions which then influenced their further exploratory behaviors. Our findings suggest that the dynamic interplay between learning to perceive object properties and manually exploring them continues to adapt and mature beyond infancy. Our findings support the idea that children between 3 – 5 years of age reach for objects depending on both- object size and its shape. This finding is partly in line with previous studies in which infants reaching and grasping were modulated by object size (Corbetta & Snapp-Childs, 2009; Lee et al., 2006; Newell et al., 1989; Ruff, 1984) but in contrast to those where an integration between vision and haptic information was observed during late infancy to adjust grip configurations (Corbetta et al., 2000; Corbetta & Snapp-Childs, 2009; Fagard, 1998; Fagard & Pezé, 1997; Lockman et al., 1984). In this study, use of object shape instead of texture indicated that children in this age group rely mainly on all object properties perceived via vision. Children in these ages not just perceived the two object properties simultaneously but weighed between them and decided their reaching strategy based on the object property that was deemed more affordable. For instance, we found that children often reached a large ball with one hand but used two hands for the same size cube, indicating that the ball shape was deemed more important than its size since it appears to fit with their grip configuration more easily than a cube. Moreover, weighing between object shape and size was dynamic depending on other factors such as distance from the object, goal of reaching, etc. Interestingly, this pattern of using two object 78 properties remained consistent across all three ages, indicating that children in these ages not just rely on vision-based object properties for reaching but appropriately weigh between them to translate the information into efficient reaching of objects. In terms of manual exploratory behaviors, we found two object properties- size and texture to predominantly influence a child's exploration of objects. Object size is primarily perceived via vision and texture through haptic perception and a successful integration of these two perceptions have been found in late infancy by previous studies (Corbetta et al., 2000; Corbetta & Snapp-Childs, 2009; Gottfried et al., 1978; Rose et al., 1979; Stack & Tsonis, 1999). However, their integration seemed to have evolved and become complex in children between 3 – 5 years. Findings on the child's first interactions showed main effects of object size and texture, but the frequency of finger activities like fingering and squeezing was substantially higher during first interactions compared to total behaviors seen with an object. Since these finger activities are used as behaviors to explore texture (Ruff, 1984), our findings indicate that children weighed more on object texture for their first interactions with an object. This effect was specially seen after children figured out that the similar looking objects had two different textures. The integration of vision and perception appeared like a 2-step process in which children first used texture followed by size to interact with objects. This exploration pattern was subtle to be observed mainly in group averages while object size and texture predominantly influenced both- first interactions and total exploratory behaviors of children. There was also significant effect of object size on the manual exploratory behaviors in which transportation activities like throwing were more common in small objects while wrist activities like rotations were mainly observed in large objects. At first, this finding appears counterintuitive since small objects are found to be preferred exploration targets as they appear more graspable (Corbetta & Snapp-Childs, 2009; Lee et al., 2006; Newell et al., 1989; Ruff, 79 1984). However, by the same reasoning, small objects can be easily grasped with one hand, so they can afford throwing better than large ones that require two hands to throw. On the other hand, exploratory behaviors like rotation when performed with two hands require adequate bimanual coordination for holding and simultaneously rotating the object. In this case, small objects may put undue biomechanical constraints due to their smaller surface area. This explains why children in all age groups preferred small objects for throwing behaviors but larger ones to perform intricate behaviors such as rotations. This leads us to the question whether we should use large objects in daycares, schools and intervention practices to promote fine motor skills in early childhood? In terms of age, there was a developmental shift from transportation activities such as throwing to more skillful wrist activities such as rotation with increasing age. Such developmental shift is not unique to this age group but also seen in infancy and toddlerhood where infants show a shift from mouthing behaviors to throwing and fingering activities (Ruff, 1984). The change in behavior pattern can be attributed to the dynamic interplay between manual exploratory behaviors and learning to perceive object properties that continue to mature and evolve beyond infancy. With the development of the perceptual-motor system, manual exploratory behaviors are expected to become more complex and skilled. For instance, rotational behaviors were not just more common in the 5-year group but were also more complex in them than the youngest age group. Performing rotations especially with two hands should require adequate bimanual coordination which is further constrained by the individual (hand size, etc.) and task factors such as object physical properties (size, shape, etc.) (Newell, 1986). In addition to rotations, children in the 5- year group also performed juggling of objects which is yet another skillful wrist activity. This age effect on how children interact with objects not just suggests a 80 developmental shift in exploratory behaviors but are also indicative of the increased skill level in them. In summary, the current study findings extend our understanding on the role played by different factors such as age and object properties on how a child's skill to interact with objects effectively develop, adapt and mature beyond infancy. This study goes beyond the question on what object properties do children perceive but provides understanding on how their knowledge to perceive different object properties translate into their actions. Our findings suggest that children in this age group mainly use object properties to drive their interactions with objects; object size and shape to reach for an object but object size and texture for manually exploring it. Given our study found effects of age and object properties in children between 3 – 5 years, such investigation should be continued in children below 3 years to fully uncover their role in the developmental trajectories of these behaviors. A thorough assessment of manual exploratory behaviors from infancy until early childhood is required to fully understand their functional role in the emergence, development and maturation of tool use at different time points as a function of different factors. 81 CHAPTER 6. GENERAL DISCUSSION The purpose of this dissertation was to examine the characteristics of manual exploratory behaviors that have important functional and clinical roles in early childhood using qualitative (behavioral) and quantitative (kinematic) means of analysis. We also addressed two main challenges encountered in their quantitative analysis: (i) reliably classify the type of movement on time series, and (ii) perform this classification throughout a time series without requiring its manual curation. In Study 1, we addressed the quantitative analysis challenges by proposing a proof-of-concept on using Machine Learning (ML) -based automated classification method to classify three important manual exploratory behaviors (rotation, throwing and fingering) in data from adult participants. In Study 2, we assessed the accuracy of ML classifiers from study 1 using data from children and conducted quantitative analysis of two important exploratory behaviors- rotation and throwing in preschoolers under the effect of age. In Study 3, we assessed the effects of age and three object properties (size, texture and shape) on the manual exploratory behaviors in children between 3 – 5 years of age using behavioral analysis. Together, these study findings reinforce the importance of investigating the qualitative and quantitative characteristics of manual exploratory behaviors throughout early childhood to fully understand their functional and clinical role in motor development. ML -based automated classification system The results of Study 1 indicated that the ML -based classification method is a plausible solution to identify the highly variable manual exploratory behaviors on time series which is a prerequisite for conducting their quantitative analysis. All the 22 trained models had a performance accuracy substantially higher than the chance level (33.33%) in classifying the three manual exploratory behaviors (rotation, throwing and fingering). Of these, SVM Quadratic, SVM Medium Gaussian and Narrow Neural Network were chosen as the most representative 82 models based on their consistent high performance on all the evaluation metrics across all testing methods. Since these models were trained and tested using adult dataset, their performances provided an upper estimate of implementing ML -based classification methods in classifying the three behaviors. High performance of the three models also validated that the five statistical features (mean, standard deviation, autocorrelation coefficients, interquartile range and energy) used to train the classifiers are representative of the unique characteristics in rotation, throwing and fingering behaviors. These features may not be representative of other manual exploratory behaviors but they provide a foundational framework for classifying other behaviors using this method. In study 2, we tested three most representative models to classify the three behaviors (rotation, throwing and fingering behaviors) in children. There was a substantial decrease in their performances on all evaluation metrics which can be attributed to two reasons. First, there were differences in the training (adult) and testing (children) datasets owing to the developmental differences in the movements between adults and children. Second, the technical difficulties encountered during remote data collection with children led to less clean and controlled data with insufficient transition phase between two behaviors. These difficulties were amplified for short duration behaviors such as throwing which resulted in their substantially more incorrect classification by the ML classifiers. However, these technical difficulties are relatively manageable in lab-based study designs which have a more controlled environment. In summary, study 1 provides a framework required to implement ML -based methods for classifying these behaviors. In study 2, we found that the reasons for poor performance of this method seems fairly manageable and adequately resolving them may optimize ML performance. Based on these studies, the ML -based classification method appears to be a plausible and feasible method to facilitate quantitative analysis of manual exploratory behaviors. 83 Characterization of Manual exploratory behaviors Results from study 2 and 3 suggest that the characteristics of manual exploratory behaviors evolve under the effects of age and object properties. In terms of age effects, study 2 indicated kinematic changes in the rotational behaviors while study 3 showed differences in their frequency of occurrence with an increasing age. In addition, we also found effects of object properties on the qualitative characteristics of manual exploratory behaviors wherein object size and shape guided child’s hand preference during reaching while size and texture influenced type of behavior. Manual exploratory behaviors are similar to other motor behaviors that evolve and mature as a function of different individual, environmental and task constraints. In terms of developmental age effects, rotations became more variable and faster while qualitatively there was an increase in their frequency of occurrence with increasing age. Unlike other behaviors like reaching, banging, etc. (Kahrs et al., 2012, 2013, 2014; Lee et al., 2011; Lee & Newell, 2012) where reduced variability indicates better motor performance, highly variable rotations indicate high skill level. Variable rotations are performed along different axes which require adequate bimanual coordination for simultaneously rotating the object in different directions while holding it to prevent it from dropping. On the other hand, increased frequency of these behaviors indicated that children were more interested in performing them either to get better at the skill or explore different strategies of rotating objects. Increased frequency of a behavior also means that the child is able to afford performing it (Ruff, 1984). Both quantitative and qualitative findings support each other, wherein children’s increasing ability to perform complex rotations and their preference to perform them often indicated a simultaneous increase in their skill level for these types of behaviors. Furthermore, the qualitative characteristics of manual exploratory behaviors are found to have atypical expressions in infants and toddlers at high risk for ASD (Kaur et al., 2015; Ozonoff 84 et al., 2008). These atypical expressions are mainly observed in rotational and throwing/pushing behaviors in which at-high risk children perform these behaviors more often and in a repetitive pattern. However, the quantitative characteristics of these atypical rotational patterns are yet to be assessed. Just as the qualitative and quantitative findings from study 2 and 3 provided a complete picture on the development of rotations in preschoolers, a quantitative characterization is required in addition to qualitative assessment to understand the atypical expressions in infants and toddlers at high-risk of ASD. Especially, the kinematic variable- rotational variability has the potential to quantify the amount and degree of repetitiveness in atypical rotations. By comparing the rotational variability between typical and atypical expressions of rotations, it is possible to find out specific characteristics that can act as potential biomarkers for early detection of ASD. In terms of object properties, we found children between 3 – 5 years used more than one object property to guide their reaching and exploratory behaviors. Children reached for objects depending on object size and shape (perceived via vision) but explored them based on their size and texture (vision and haptic perceptions respectively). The findings on reaching behaviors are in contrast to previous literature in which infants adjusted their grip configurations by integrating vision and haptic information (Corbetta et al., 2000; Corbetta & Snapp-Childs, 2009; Gottfried et al., 1978; Rose et al., 1979; Stack & Tsonis, 1999). In study 3, children not just purely used vision- based properties (size and shape) but weighed between them to decide which object property afforded more efficient reach. For instance, children often reached a large ball with one hand because the ball shape seemed to afford one hand reach despite its large size. For exploring objects, children mainly used object size and texture in a similar fashion as seen in late infancy (Corbetta et al., 2000; Corbetta & Snapp-Childs, 2009; Lee et al., 2006; Newell et al., 1989; Ruff, 1984). However, there was a difference in how children between 3 – 5 years of age integrated vision and haptic information compared to infants. We found that once 85 children found a texture difference between similar looking objects (soft vs. hard), they started using object texture to guide their first interactions. So, their first exploratory behaviors would be squeezing objects or moving fingers on its surface to check the texture and then perform other exploratory behaviors depending on object size. This is in contrast to the Corbetta and Snapp- Childs (2009) study in which infants could not reliably utilize perceptual information from prior experiences to modify their motor behaviors. Similar to reaching behaviors, it can be deduced that children weigh between object properties for exploration purposes and perform exploratory behaviors in an order of which object property is deemed more important or affordable. The effect of object size on child’s exploratory behaviors was counter-intuitive and different from what has been observed in infants. Evidence exists that small objects are deemed as preferred exploration targets since they afford better grasp (Corbetta & Snapp-Childs, 2009; Lee et al., 2006; Newell et al., 1989; Ruff, 1984). However, children in study 3 chose large objects for skillful wrist activities while performing throwing behaviors with small objects. This can be attributed to the ongoing perceptual-motor development in children beyond infancy that help them learn new ways of exploring objects as well as optimize their existing exploring strategies. Large objects that were deemed too big to grasp in infancy were now considered to afford skilled actions owing to their larger surface area. Similarly, children by 3 years have gathered enough experience in throwing objects which helps them understand that small size of an object affords more efficient throw as it can be easily grasped with one hand. Such age specific developmental changes in the way children perceive object properties and plan their motor actions should be taken into consideration while designing activities and interventions that promote fine motor skills in early childhood. 86 Future directions In Study 1, our proposed Machine Learning -based classification system demonstrated high accuracy in classifying the three manual exploratory behaviors on data from adult participants. However, its direct implementation on data from children in Study 2 led to substantial decrease in the classifier accuracy. Further work is required to (1) identify and resolve the technical difficulties arising in data collection involving children, (2) train, optimize and test classifiers using data from children, and (3) expand the scope of classifiers to classify more manual exploratory behaviors. In Study 2, our quantitative findings showed a difference in the rotational patterns but not the throwing patterns in children between 3 – 5 years. Further investigation needs to be done to (1) quantify the kinematic characteristics of these behaviors in children under 3 years of age, (2) quantify stereotypical rotational patterns in infants and toddlers at high risk for ASD using rotational variability, and (3) conduct similar kinematic characterization of other important manual exploratory behaviors. In Study 3, we characterized the manual exploratory behaviors using qualitative analysis in children between 3- 5 years for the effects of object properties and age. However, further work is needed to (1) characterize these behaviors using quantitative analysis, and (2) conduct systematic qualitative and quantitative characterization of these behaviors throughout early childhood starting from infancy to 5 years of age. Conclusions To summarize, the results from the three studies allow for the following conclusions: 1. The machine learning -based automated classification system classified the three important manual exploratory behaviors in adult dataset with higher accuracy and in shorter duration than the conventional video coding method. Upon optimizing this 87 method to classify behaviors in children, it has strong potential to address the challenges encountered in the quantitative analysis of manual exploratory behaviors; thereby facilitating their systematic characterization throughout early childhood. 2. The kinematic variables- rotational variability, angular jerk and angular velocity profiles showed that children performed more variable and faster rotations with increasing age. Since variable rotations, that involve rotating an object on different axes, can be intricate, these findings may indicate an increase in the skill level of children with increase in age. Moreover, using the variable- rotational variability, it is possible to quantify the amount of repetitiveness in stereotypical rotations observed in infants and toddlers at high-risk of ASD. On the other hand, the kinematic variables- linear jerk and linear acceleration profiles did not find differences in the throwing patterns of children across 3 – 5 years of age. 3. Object size and shape directed reaching behaviors in children between 3 – 5 years, while object size and shape influenced the way they explored and interacted with objects. Frequency of fingering and squeezing behaviors was substantially high during the first interactions with the objects indicating that children used object texture as first means of object exploration. There was a developmental shift in the type of manual exploratory behaviors wherein throwing behaviors were more common in the 3-year group but skillful wrist activities like rotations were common in 4- and 5- year groups. Overall, findings indicate that the manual exploratory behaviors continue to mature and evolve simultaneously with the development of the perceptual-motor system beyond infancy and throughout early childhood. 88 BIBLIOGRAPHY 89 BIBLIOGRAPHY Adolph, K. E., Cole, W. G., Komati, M., Garciaguirre, J. S., Badaly, D., Lingeman, J. M., Chan, G. L. Y., & Sotsky, R. B. (2012). How Do You Learn to Walk? Thousands of Steps and Dozens of Falls per Day: Psychological Science. https://doi.org/10.1177/0956797612446346 Anderson, I., Maitland, J., Sherwood, S., Barkhuus, L., Chalmers, M., Hall, M., Brown, B., & Muller, H. (2007). Shakra: Tracking and Sharing Daily Activity Levels with Unaugmented Mobile Phones. Mobile Networks and Applications, 12(2), 185–199. https://doi.org/10.1007/s11036-007-0011-7 Anguita, D., Ghio, A., Oneto, L., Parra Perez, X., & Reyes Ortiz, J. L. (2013). A public domain dataset for human activity recognition using smartphones. Proceedings of the 21th International European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 437–442. Anjum, A., & Ilyas, M. U. (2013). Activity recognition using smartphone sensors. 2013 IEEE 10th Consumer Communications and Networking Conference (CCNC), 914–919. https://doi.org/10.1109/CCNC.2013.6488584 Aslin, R. N., & Smith, L. B. (1988). Perceptual development. Annual Review of Psychology, 39(1), 435–473. Baranek, G. T. (1999). Autism during infancy: A retrospective video analysis of sensory-motor and social behaviors at 9–12 months of age. Journal of Autism and Developmental Disorders, 29(3), 213–224. Berchtold, M., Budde, M., Gordon, D., Schmidtke, H. R., & Beigl, M. (2010). ActiServ: Activity Recognition Service for mobile phones. International Symposium on Wearable Computers (ISWC) 2010, 1–8. https://doi.org/10.1109/ISWC.2010.5665868 Berlyne, D. E. (1966). Curiosity and exploration. Science, 153(3731), 25–33. Bhat, A. N., Landa, R. J., & Galloway, J. C. (2011). Current perspectives on motor functioning in infants, children, and adults with autism spectrum disorders. Physical Therapy, 91(7), 1116–1129. Bieber, G., Koldrack, P., Sablowski, C., Peter, C., & Urban, B. (2010). Mobile physical activity recognition of stand-up and sit-down transitions for user behavior analysis. Proceedings of the 3rd International Conference on PErvasive Technologies Related to Assistive Environments, 1–5. https://doi.org/10.1145/1839294.1839354 Biryukova, E., & Bril, B. (2012). Biomechanical analysis of tool use: A return to Bernstein’s tradition. Zeitschrift Für Psychologie, 220(1), 53. 90 Box, G. E., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2015). Time series analysis: Forecasting and control. John Wiley & Sons. Brakke, K., Fragaszy, D. M., Simpson, K., Hoy, E., & Cummins-Sebree, S. (2007). The production of bimanual percussion in 12-to 24-month-old children. Infant Behavior and Development, 30(1), 2–15. Campione, G. C., Piazza, C., Villa, L., & Molteni, M. (2016). Three-dimensional kinematic analysis of prehension movements in young children with autism spectrum disorder: New insights on motor impairment. Journal of Autism and Developmental Disorders, 46(6), 1985–1999. CDC. (2022, March 31). Screening and Diagnosis | Autism Spectrum Disorder (ASD) | NCBDDD. Centers for Disease Control and Prevention. https://www.cdc.gov/ncbddd/autism/screening.html Chen, Z., Siegler, R. S., & Daehler, M. W. (2000). Across the great divide: Bridging the gap between understanding of toddlers’ and older children’s thinking. Monographs of the Society for Research in Child Development, i–105. Corbetta, D., & Snapp-Childs, W. (2009). Seeing and touching: The role of sensory-motor experience on the development of infant reaching. Infant Behavior and Development, 32(1), 44–58. Corbetta, D., & Thelen, E. (1996). The developmental origins of bimanual coordination: A dynamic perspective. Journal of Experimental Psychology: Human Perception and Performance, 22(2), 502. Corbetta, D., Thelen, E., & Johnson, K. (2000). Motor constraints on the development of perception-action matching in infant reaching. Infant Behavior and Development, 23(3– 4), 351–374. Debes, C., Merentitis, A., Sukhanov, S., Niessen, M., Frangiadakis, N., & Bauer, A. (2016). Monitoring activities of daily living in smart homes: Understanding human behavior. IEEE Signal Processing Magazine, 33(2), 81–94. Dernbach, S., Das, B., Krishnan, N. C., Thomas, B. L., & Cook, D. J. (2012). Simple and Complex Activity Recognition through Smart Phones. 2012 Eighth International Conference on Intelligent Environments, 214–221. https://doi.org/10.1109/IE.2012.39 Ding, M., & Fan, G. (2014). Multilayer joint gait-pose manifolds for human gait motion modeling. IEEE Transactions on Cybernetics, 45(11), 2413–2424. Fagard, J. (1998). Changes in grasping skills and the emergence of bimanual coordination during the first year of life. 91 Fagard, J., & Jacquet, A. Y. (1996). Changes in reaching and grasping objects of different sizes between 7 and 13 months of age. British Journal of Developmental Psychology, 14(1), 65–78. Fagard, J., & Pezé, A. (1997). Age Changes in Interlimb Coupling and the Development of Bimanual Coordination. Journal of Motor Behavior, 29(3), 199–208. https://doi.org/10.1080/00222899709600835 Fitzpatrick, P., Wagman, J. B., & Schmidt, R. C. (2015). Alterations in Movement Dynamics in a Tool-Use Task. Zeitschrift Für Psychologie. Fontecha, J., Navarro, Fco. J., Hervás, R., & Bravo, J. (2013). Elderly frailty detection by using accelerometer-enabled smartphones and clinical information records. Personal and Ubiquitous Computing, 17(6), 1073–1083. https://doi.org/10.1007/s00779-012-0559-5 Fragaszy, D., Simpson, K., Cummins-Sebree, S., & Brakke, K. (2016). Ontogeny of tool use: How do toddlers use hammers? Developmental Psychobiology, 58(6), 759–772. Franchak, J. M., & Adolph, K. E. (2012). What infants know and what they do: Perceiving possibilities for walking through openings. Developmental Psychology, 48(5), 1254– 1261. https://doi.org/10.1037/a0027530 Gibson, E. J. (1988). Exploratory behavior in the development of perceiving, acting, and the acquiring of knowledge. Annual Review of Psychology, 39(1), 1–42. Goldfield, E. C. (1995). Emergent forms: Origins and early development of human action and perception. Oxford University Press on Demand. Goldfield, E. C., & Michel, G. F. (1986). The ontogeny of infant bimanual reaching during the first year. Infant Behavior and Development, 9(1), 81–89. Goodfellow, D., Zhi, R., Funke, R., Pulido, J. C., Mataric, M., & Smith, B. A. (2018). Predicting infant motor development status using day long movement data from wearable sensors. ArXiv Preprint ArXiv:1807.02617. Gottfried, A. W., Rose, S. A., & Bridger, W. H. (1978). Effects of visual, haptic, and manipulatory experiences on infants’ visual recognition memory of objects. Developmental Psychology, 14(3), 305–312. https://doi.org/10.1037/0012-1649.14.3.305 Hogan, N., & Sternad, D. (2009). Sensitivity of Smoothness Measures to Movement Duration, Amplitude, and Arrests. Journal of Motor Behavior, 41(6), 529–534. https://doi.org/10.3200/35-09-004-RC Hutt, C. (1970). Specific and diversive exploration. Advances in Child Development and Behavior, 5, 119–180. Ihlen, E. A. F., Støen, R., Boswell, L., Regnier, R.-A. de, Fjørtoft, T., Gaebler-Spira, D., Labori, C., Loennecken, M. C., Msall, M. E., Möinichen, U. I., Peyton, C., Schreiber, M. D., 92 Silberg, I. E., Songstad, N. T., Vågen, R. T., Øberg, G. K., & Adde, L. (2019). Machine Learning of Infant Spontaneous Movements for the Early Prediction of Cerebral Palsy: A Multi-Site Cohort Study. Journal of Clinical Medicine, 9(1), E5. https://doi.org/10.3390/jcm9010005 Jung, W. P., Kahrs, B. A., & Lockman, J. J. (2015). Manual action, fitting, and spatial planning: Relating objects by young children. Cognition, 134, 128–139. https://doi.org/10.1016/j.cognition.2014.09.004 Jung, W. P., Kahrs, B. A., & Lockman, J. J. (2018). Fitting handled objects into apertures by 17- to 36-month-old children: The dynamics of spatial coordination. Developmental Psychology, 54(2), 228–239. https://doi.org/10.1037/dev0000420 Kahrs, B. A., Jung, W. P., & Lockman, J. J. (2012). What is the role of infant banging in the development of tool use? Experimental Brain Research, 218(2), 315–320. Kahrs, B. A., Jung, W. P., & Lockman, J. J. (2013). Motor origins of tool use. Child Development, 84(3), 810–816. Kahrs, B. A., Jung, W. P., & Lockman, J. J. (2014). When does tool use become distinctively human? Hammering in young children. Child Development, 85(3), 1050–1061. Kang, X., Huang, B., & Qi, G. (2018). A novel walking detection and step counting algorithm using unconstrained smartphones. Sensors, 18(1), 297. Kaur, M., Srinivasan, S. M., & Bhat, A. N. (2015). Atypical object exploration in infants at-risk for autism during the first year of life. Frontiers in Psychology, 6, 798. Kawa, R., & Pisula, E. (2010). Locomotor activity, object exploration and space preference in children with autism and Down syndrome. Acta Neurobiol Exp, 70(2), 131–140. Keen, R., Lee, M.-H., & Adolph, K. (2014). Planning an action: A developmental progression in tool use. Ecological Psychology, 26(1–2), 98–108. Kim, J. K., Choo, Y. J., & Chang, M. C. (2021). Prediction of motor function in stroke patients using machine learning algorithm: Development of practical models. Journal of Stroke and Cerebrovascular Diseases, 30(8), 105856. Konczak, J., & Dichgans, J. (1997). The development toward stereotypic arm kinematics during reaching in the first 3 years of life. Experimental Brain Research, 117(2), 346–354. Kwapisz, J. R., Weiss, G. M., & Moore, S. A. (2011). Activity recognition using cell phone accelerometers. ACM SIGKDD Explorations Newsletter, 12(2), 74–82. https://doi.org/10.1145/1964897.1964918 Kwolek, B., & Kepski, M. (2014). Human fall detection on embedded platform using depth maps and wireless accelerometer. Computer Methods and Programs in Biomedicine, 117(3), 489–501. 93 Landa, R., & Garrett-Mayer, E. (2006). Development in infants with autism spectrum disorders: A prospective study. Journal of Child Psychology and Psychiatry, 47(6), 629–638. Lee, M.-H., Liu, Y.-T., & Newell, K. (2006). Longitudinal expressions of infant’s prehension as a function of object properties. Infant Behavior and Development, 29(4), 481–493. Lee, M.-H., & Newell, K. (2012). Visual feedback of hand trajectory and the development of infant prehension. Infant Behavior and Development, 35(2), 273–279. Lee, M.-H., & Newell, K. (2013). Contingent auditory feedback of arm movement facilitates reaching behavior in infancy. Infant Behavior and Development, 36(4), 817–824. Lee, M.-H., Ranganathan, R., & Newell, K. (2011). Changes in object-oriented arm movements that precede the transition to goal-directed reaching in infancy. Developmental Psychobiology, 53(7), 685–693. Libertus, K., & Hauf, P. (2017). Motor skills and their foundational role for perceptual, social, and cognitive development. Frontiers in Psychology, 8, 301. Lockman, J. J. (2000). A perception–action perspective on tool use development. Child Development, 71(1), 137–144. Lockman, J. J., Ashmead, D. H., & Bushnell, E. W. (1984). The development of anticipatory hand orientation during infancy. Journal of Experimental Child Psychology, 37(1), 176– 186. https://doi.org/10.1016/0022-0965(84)90065-1 Lockman, J. J., Fears, N. E., & Jung, W. P. (2018). Chapter Two - The Development of Object Fitting: The Dynamics of Spatial Coordination. In J. M. Plumert (Ed.), Advances in Child Development and Behavior (Vol. 55, pp. 31–72). JAI. https://doi.org/10.1016/bs.acdb.2018.05.001 Lockman, J. J., & Kahrs, B. A. (2017). New insights into the development of human tool use. Current Directions in Psychological Science, 26(4), 330–334. Lohse, K., Buchanan, T., & Miller, M. (2016). Underpowered and overworked: Problems with data analysis in motor learning studies. Journal of Motor Learning and Development, 4(1), 37–58. https://doi.org/10.1123/jmld.2015-0010 Mao, A., Ma, X., He, Y., & Luo, J. (2017). Highly portable, sensor-based system for human fall monitoring. Sensors, 17(9), 2096. McCall, R. B. (1974). Exploratory manipulation and play in the human infant. Monographs of the Society for Research in Child Development, 1–88. McCarty, M. E., Clifton, R. K., & Collard, R. R. (1999). Problem solving in infancy: The emergence of an action plan. Developmental Psychology, 35(4), 1091. 94 McCarty, M. E., Clifton, R. K., & Collard, R. R. (2001). The beginnings of tool use by infants and toddlers. Infancy, 2(2), 233–256. Needham, A. W., Wiesen, S. E., Hejazi, J. N., Libertus, K., & Christopher, C. (2017). Characteristics of brief sticky mittens training that lead to increases in object exploration. Journal of Experimental Child Psychology, 164, 209–224. Newell, K. (1986). Constraints on the development of coordination. Motor Development in Children: Aspects of Coordination and Control. Newell, K., Scully, D. M., McDonald, P. V., & Baillargeon, R. (1989). Task constraints and infant grip configurations. Developmental Psychobiology: The Journal of the International Society for Developmental Psychobiology, 22(8), 817–831. Olivier, I., Hay, L., Bard, C., & Fleury, M. (2007). Age-related differences in the reaching and grasping coordination in children: Unimanual and bimanual tasks. Experimental Brain Research, 179(1), 17–27. https://doi.org/10.1007/s00221-006-0762-6 Ozonoff, S., Macari, S., Young, G. S., Goldring, S., Thompson, M., & Rogers, S. J. (2008). Atypical object exploration at 12 months of age is associated with autism in a prospective sample. Autism, 12(5), 457–472. Patel, P., Shi, Y., Hajiaghajani, F., Biswas, S., & Lee, M.-H. (2019). A novel two-body sensor system to study spontaneous movements in infants during caregiver physical contact. Infant Behavior and Development, 57, 101383. https://doi.org/10.1016/j.infbeh.2019.101383 Paulus, M., & Hauf, P. (2011). Infants’ use of material properties to guide their actions with differently weighted objects. Infant and Child Development, 20(4), 423–436. Piaget, J., & Cook, M. (1952). The origins of intelligence in children (Vol. 8). International Universities Press New York. Pierce, K., & Courchesne, E. (2001). Evidence for a cerebellar role in reduced exploration and stereotyped behavior in autism. Biological Psychiatry, 49(8), 655–664. Pisula, E. (2003). Behavior of children with autism in a new environment. Polish Psychological Bulletin, 34(4), 217–224. Pryde, K. M., Roy, E. A., & Campbell, K. (1998). Prehension in children and adults: The effects of object size. Human Movement Science, 17(6), 743–752. Ranganathan, R., Lee, M.-H., Padmanabhan, M. R., Aspelund, S., Kagerer, F. A., & Mukherjee, R. (2019). Age-dependent differences in learning to control a robot arm using a body- machine interface. Scientific Reports, 9(1), 1960. https://doi.org/10.1038/s41598-018- 38092-3 95 Ranganathan, R., Tomlinson, A. D., Lokesh, R., Lin, T.-H., & Patel, P. (2020). A tale of too many tasks: Task fragmentation in motor learning and a call for model task paradigms. Experimental Brain Research. https://doi.org/10.1007/s00221-020-05908-6 Ravi, N., Dandekar, N., Mysore, P., & Littman, M. L. (2005). Activity recognition from accelerometer data. Aaai, 5(2005), 1541–1546. Rodman, J. L., Gilbert, K. A., Grove, A. B., Cunningham, M., Levenson, S., & Wajsblat, L. (2010). Efficacy of brief quantitative measures of play for screening for autism spectrum disorders. Journal of Autism and Developmental Disorders, 40(3), 325–333. Rohrer, B., Fasoli, S., Krebs, H. I., Hughes, R., Volpe, B., Frontera, W. R., Stein, J., & Hogan, N. (2002). Movement Smoothness Changes during Stroke Recovery. Journal of Neuroscience, 22(18), 8297–8304. https://doi.org/10.1523/JNEUROSCI.22-18- 08297.2002 Ronao, C. A., & Cho, S.-B. (2016). Human activity recognition with smartphone sensors using deep learning neural networks. Expert Systems with Applications, 59, 235–244. Rose, S. A., Gottfried, A. W., & Bridger, W. H. (1979). Effects of haptic cues on visual recognition memory in fullterm and preterm infants. Infant Behavior and Development, 2, 55–67. https://doi.org/10.1016/S0163-6383(79)80008-9 Ruff, H. A. (1984). Infants’ manipulative exploration of objects: Effects of age and object characteristics. Developmental Psychology, 20(1), 9. Saponas, T. S., Lester, J., Froehlich, J., Fogarty, J., & L, J. (2008). iLearn on the iPhone: Real- Time Human Activity Classification on Commodity Mobile Phones. Schneiberg, S., Sveistrup, H., McFadyen, B., McKinley, P., & Levin, M. F. (2002). The development of coordination for reach-to-grasp movements in children. Experimental Brain Research, 146(2), 142–154. Shin, H. I., Shin, H.-I., Bang, M. S., Kim, D.-K., Shin, S. H., Kim, E.-K., Kim, Y.-J., Lee, E. S., Park, S. G., Ji, H. M., & Lee, W. H. (2022). Deep learning-based quantitative analyses of spontaneous movements and their association with early neurological development in preterm infants. Scientific Reports, 12(1), 3138. https://doi.org/10.1038/s41598-022- 07139-x Shoaib, M., Bosch, S., Incel, O. D., Scholten, H., & Havinga, P. J. M. (2015). A Survey of Online Activity Recognition Using Mobile Phones. Sensors, 15(1), 2059–2085. https://doi.org/10.3390/s150102059 Simon-Martinez, C., Santos, G. L. dos, Jaspers, E., Vanderschueren, R., Mailleux, L., Klingels, K., Ortibus, E., Desloovere, K., & Feys, H. (2018). Age-related changes in upper limb motion during typical development. PLOS ONE, 13(6), e0198524. https://doi.org/10.1371/journal.pone.0198524 96 Smith, B. A., Trujillo-Priego, I. A., Lane, C. J., Finley, J. M., & Horak, F. B. (2015). Daily quantity of infant leg movement: Wearable sensor algorithm and relationship to walking onset. Sensors, 15(8), 19006–19020. Smith, L. B., Street, S., Jones, S. S., & James, K. H. (2014). Using the axis of elongation to align shapes: Developmental changes between 18 and 24months of age. Journal of Experimental Child Psychology, 123, 15–35. https://doi.org/10.1016/j.jecp.2014.01.009 Smitsman, A. W. (1997). The development of tool use: Changing boundaries between organism and environment. Smyth, M. M., Katamba, J., & Peacock, K. A. (2004). Development of prehension between 5 and 10 years of age: Distance scaling, grip aperture, and sight of the hand. Journal of Motor Behavior, 36(1), 91–103. Soska, K. C., & Adolph, K. E. (2014). Postural position constrains multimodal object exploration in infants. Infancy, 19(2), 138–161. Soska, K. C., Adolph, K. E., & Johnson, S. P. (2010). Systems in development: Motor skill acquisition facilitates three-dimensional object completion. Developmental Psychology, 46(1), 129. Sousa Lima, W., Souto, E., El-Khatib, K., Jalali, R., & Gama, J. (2019). Human activity recognition using inertial sensors in a smartphone: An overview. Sensors, 19(14), 3213. Srinivasan, S. M., & Bhat, A. N. (2019). Differences in means-end exploration between infants at risk for autism and typically developing infants in the first 15 months of life. Developmental Psychobiology, 61(2), 203–215. Stack, D. M., & Tsonis, M. (1999). Infants’ haptic perception of texture in the presence and absence of visual cues. British Journal of Developmental Psychology, 17(1), 97–110. https://doi.org/10.1348/026151099165177 Street, S. Y., James, K. H., Jones, S. S., & Smith, L. B. (2011). Vision for Action in Toddlers: The Posting Task. Child Development, 82(6), 2083–2094. https://doi.org/10.1111/j.1467- 8624.2011.01655.x Teitelbaum, P., Teitelbaum, O., Nye, J., Fryman, J., & Maurer, R. G. (1998). Movement analysis in infancy may be useful for early diagnosis of autism. Proceedings of the National Academy of Sciences, 95(23), 13982–13987. Thelen, E. (1981). Rhythmical behavior in infancy: An ethological perspective. Developmental Psychology, 17(3), 237. Tunca, C., Pehlivan, N., Ak, N., Arnrich, B., Salur, G., & Ersoy, C. (2017). Inertial sensor-based robust gait analysis in non-hospital settings for neurological disorders. Sensors, 17(4), 825. 97 White, R. W. (1959). Motivation reconsidered: The concept of competence. Psychological Review, 66(5), 297. Xu, J. Y., Chang, H.-I., Chien, C., Kaiser, W. J., & Pottie, G. J. (2013). Context-driven, prescription-based personal activity classification: Methodology, architecture, and end- to-end implementation. IEEE Journal of Biomedical and Health Informatics, 18(3), 1015–1025. Yan, J. H., Thomas, J. R., Stelmach, G. E., & Thomas, K. T. (2000). Developmental Features of Rapid Aiming Arm Movements Across the Lifespan. Journal of Motor Behavior, 32(2), 121–140. https://doi.org/10.1080/00222890009601365 Yang, H.-C., Lee, I.-C., & Lee, I.-C. (2014). Visual feedback and target size effects on reach-to- grasp tasks in children with autism. Journal of Autism and Developmental Disorders, 44(12), 3129–3139. Yang, J. (2009). Toward physical activity diary: Motion recognition using simple acceleration features with mobile phones. Proceedings of the 1st International Workshop on Interactive Multimedia for Consumer Electronics, 1–10. https://doi.org/10.1145/1631040.1631042 Zoia, S., Pezzetta, E., Blason, L., Scabar, A., Carrozzi, M., Bulgheroni, M., & Castiello, U. (2006). A comparison of the reach-to-grasp movement between children and adults: A kinematic study. Developmental Neuropsychology, 30(2), 719–738. 98