Abstract
The researchers report on the reliability and validity of six tests of vestibular and proprioceptive functions of the Evaluation in Ayres Sensory Integration® (EASI).
Ayres Sensory Integration® (ASI) is a frame of reference that guides an understanding of how sensory and motor functions contribute to behavior, learning, and occupational performance (Ayres, 2005; Parham & Cosbey, 2020; Parham & Mailloux, 2020; Schaaf & Mailloux, 2015). The vestibular and proprioceptive systems are foundational to ASI theory. Their contributions to postural–ocular and bilateral motor skills are essential to skill development and lay a foundation for play and participation, independence in self-care skills, and academic achievement (Ayres, 1972a, 1975, 1978; Bundy & Lane, 2020; Parham & Cosbey, 2020; Parham & Mailloux, 2020). Vestibular- and proprioceptive-related functions are important assessment considerations during an occupational therapy evaluation (Accreditation Council for Occupational Therapy Education, 2018; Parham et al., 2011; Reynolds et al., 2012). Vestibular–proprioceptive functions are strongly associated with ocular, postural, and bilateral integration, suggesting that these sensory and motor functions are functionally interdependent (Ayres, 1965, 1972a; Lane & Reynolds, 2020; Mailloux et al., 2011; Mulligan, 1998).
Vestibular Sensory System
Vestibular receptors in the inner ear are specialized to detect gravitational pull arising from static head position and acceleration or deceleration of the head through space. Detection of the pull of gravity or movement of the head results in neural activation that travels along the vestibular portion of the eighth cranial nerve to the brain stem, cerebellum, and cortex (Tascioglu, 2005).
As vestibular sensation is processed and integrated with proprioceptive and visual information in the brain stem, cerebellum, and cortex, reflexive and adaptive motor responses arise (e.g., extensor muscle tone, protective extension, righting and equilibrium reactions; Lane & Reynolds, 2020 ; Macpherson & Horak, 2013). On the basis of neuroscience knowledge, and her own research examining patterns of sensory integration function and dysfunction, Ayres (1972b, 1978, 1989) hypothesized that the vestibular system contributes to several important functions, such as coordination of the two body sides, spatial orientation and directionality, and regulation of arousal levels. These basic sensory and motor functions underlie many daily life activities (e.g., maintaining postural control while balancing on one leg during dressing, coordinating head and eye movements for copying from the board, or using bilateral actions for riding a bicycle). Ayres (1972a, 1978) also found that vestibular function contributes to academic learning.
Proprioceptive Sensory System
Proprioceptive receptors within muscles, tendons, and joint capsules detect active and passive movements of the limbs, neck, and trunk. Incoming proprioceptive sensation is relayed to the cerebellum for precise gradation of movement and to the parietal lobes for perception of body position and awareness. Proprioception thus allows adjustments needed to perform movements that are part of everyday occupational activities (e.g., bringing a fork accurately to the mouth, adjusting the position of a pen for writing, or petting an animal with appropriate force; Ashton-Miller et al., 2001; Bundy & Lane, 2020; Ferrell et al., 2004).
Measures of Vestibular and Proprioceptive Function
Although few clinically available tests directly measure vestibular and proprioceptive function in children, consideration of these important sensory systems and their related functions is important to occupational therapy practice, given their close relationship to the foundations for successful participation in daily life. As early as 1965, Ayres identified a relationship between vestibular and proprioceptive perception and postural–ocular control. She found patterns linking right–left discrimination, crossing the midline of the body, difficulty with rhythmic activities, and poor postural–ocular control (Ayres, 1965, 1972a, 1972c). Ayres (1972a, 1972b) first called this pattern “postural bilateral integration,” noting that the relationship between these functions might “lie in their common relationship with the vestibular system” (Ayres, 1972b, p. 137). She then later confirmed her hypothesis when similar functions grouped in a factor that she named “vestibular bilateral integration disorder” (Ayres, 1978, 2005, p. 73). She found groupings of similar tests in factor analyses with the Sensory Integration and Praxis Tests (SIPT; Ayres, 1989), and similar associations also emerged in several subsequent factor analyses (Mailloux et al., 2011; Mulligan, 1998; Van Jaarsveld et al., 2015). This pattern has been found in studies of children with cochlear implants (Koester et al., 2014), autism (Roley et al., 2015; Siaperas et al., 2012), and other learning and developmental concerns (Mailloux et al., 2011).
Although some tests of the SIPT (Ayres, 1989) measure functions that are included on other tests of balance and bilateral coordination (Bruininks & Bruininks, 2005; Henderson et al., 2007), direct measures of vestibular function (i.e., the Postrotary Nystagmus Test) and proprioception (i.e., the Kinesthesia Test) for children are unique to the SIPT. However, therapists commonly report issues that limit use of the SIPT, such as cost, age of normative data, and concerns related to applicability for groups outside the United States (Mailloux et al., 2018; Schaaf et al., 2014). To address these difficulties, Mailloux et al. (2018) developed the Evaluation in Ayres Sensory Integration® (EASI), which includes measures of vestibular–proprioceptive functions within a set of 20 tests of sensory perception, ocular and postural control, bilateral integration, praxis, and sensory reactivity.
An important aspect of standardized testing is the degree to which therapists feel confident in the value of a test. Therapists seek tests that yield data with strong psychometric properties, including knowing that tests have good validity and reliability. The purpose of this study was to evaluate the validity and reliability of the following six EASI tests of vestibular- and proprioceptive-related functions: Ocular Motor & Praxis (O: M&Pr), Bilateral Integration (BI), Postural Control (PC), Balance (Bal), Proprioceptive: Force (Prop: F), and Proprioceptive: Joint Position (Prop: JP).
Method
Approval for the study, which included data collection on all 20 EASI tests, was obtained through the Thomas Jefferson University institutional review board (IRB).
Participants
We recruited a convenience sample from the continental United States. In total, 234 children ages 3 to 12 yr participated: 150 children in the typically developing (TYP) group (n male = 55, n female = 95; n ages 3–6 = 73, n ages 7–9 = 42, n ages 10–12 = 35) and 84 children in the sensory integration concerns (SI) group (n male = 49, n female = 35; n ages 3–6 = 22, n ages 7–9 = 40, n ages 10–12 = 22). The mean age of the SI group was significantly older than the TYP group (8.60 yr vs. 7.46 yr), t(233) = −3.27, p = .001; thus, we included age as a covariate in our analyses. We did not record race, ethnicity, or socioeconomic status for this pilot study.
Inclusion criteria for both groups were chronological age between 3 yr, 0 mo and 12 yr, 11 mo and English as a primary language. We included children in the TYP group who were developing and performing within age expectations, via parent report and tester observation. We excluded children from the TYP group if they had reported medical or sensory integrative concerns or if they had received any prior intervention using the ASI approach. Specifically, we asked parents to verify that none of the conditions or concerns listed in the exclusion criteria applied to children in the TYP group.
We included children in the SI group if they had been identified as having sensory integration difficulties by an occupational therapist, physical therapist, or speech-language pathologist through testing on the SIPT (Ayres, 1989), the Sensory Profile (Dunn, 1999), the Sensory Processing Measure (SPM; Parham et al., 2007), or evaluative clinical observations of their performance. We prioritized children with known or suspected problems in vestibular and proprioceptive functions for inclusion by asking the testers to try to include children with low scores or signs of problems in these areas, because the tests in this study were intended to assess these functions. We excluded children with physical disabilities (e.g., cerebral palsy, spina bifida, or spinal cord injuries), visual or hearing impairments, and significant cognitive deficits (defined as IQ score less than 70 or diagnosed with a significant developmental delay or cognitive disability) to reduce the likelihood that test performance was affected by inability to understand the test instructions; we also excluded children for whom English was not a primary language.
Procedures
We recruited 36 testers via social media and word of mouth. Although most testers were occupational therapists, a few were occupational therapy students, physical therapists, or speech-language pathologists. Testers completed online training modules, submitted sample test scores to check adherence to scoring format, recruited participants, and obtained parental consent and child assent per IRB requirements. In general, testers recruited children from local schools and programs for the TYP group and from therapy practices for the SI group. Most children completed the six tests in one 1-hr session at these clinics (16 children required two or more sessions to complete testing).
Instruments
We administered six EASI tests of vestibular- and proprioceptive-related functions: O: M&Pr, BI, PC, Bal, Prop: F, and Prop: JP. Table 1 contains a detailed description of each test. The original instruments contained many items (more than would be clinically feasible). In this study, we integrated statistical information with feedback from testers and theoretical knowledge to create more parsimonious item sets.
Description of Tests
Note. Bal = Balance; BI = Bilateral Integration; N/A = not applicable; O: M&Pr = Ocular Motor & Praxis; PC = Postural Control; Prop: F = Proprioceptive: Force; Prop: JP = Proprioceptive: Joint Position.
We reverse-coded these items for all analyses so that higher numbers represent better scores.
Data Analysis
We used Winsteps (Version 4.4.6; https://www.winsteps.com) to conduct iterative Rasch analyses to first revise the items and then evaluate the construct validity and internal reliability of the revised tests. The Rasch model is a latent-trait psychometric model that constructs linear measures from ordinal raw data. The Rasch model generates item-difficulty and person-ability measures along a single, unidimensional, true-interval scale. A detailed description of the Rasch model can be found elsewhere (e.g., Bond & Fox, 2015). The analyses aimed to first select the strongest items on the tests and then to evaluate reliability and validity of the revised tests. We complemented Rasch procedures with classical statistical methods (e.g., Cohen’s d, Cronbach’s α). We did not correct p values for multiple tests because this pilot study had a relatively small sample size. In this article, we report the psychometric properties of the data collected on the revised tests.
Analysis of Construct Validity
Rasch analysis generates several indicators of construct validity: Point–measure correlations indicate that items align in the same direction as the latent variable (i.e., a higher score on an item correlates with a higher score on the overall measure). We expected all items to demonstrate positive correlations. Goodness-of-fit statistics are expressed as (a) mean-square (MnSq) values that indicate the degree of randomness in the measurement system and (b) Zstd
values that indicate the improbability of the data if the data fit the measure perfectly (Linacre, 2002). Both MnSq and Zstd
statistics are reported as infit (weighted or inlier sensitive) and outfit (unweighted) statistics. We expected 95% of items to demonstrate MnSq values within the acceptable range of 0.5 to 1.5 and Zstd
values within the acceptable range of −2 to 2. If both criteria were violated for either infit or outfit for any item, we considered that the item failed to conform to Rasch expectations. All rating scale categories should be used at least 10 times and progress logically, such that lower categories correlate with lower average measure scores than higher categories. For this study, if either assumption was violated, we considered collapsing rating scales. Wright maps allowed us to visually examine the match between sample ability and item difficulty.
We used known-groups analyses to evaluate the six tests for further evidence of construct validity. We expected children in the TYP group to score significantly higher on all tests than children in the SI group. Because the SI group was significantly older than the TYP group (M = 8.60 yr and M = 7.46 yr, respectively), we computed group differences using analysis of covariance with age as a covariate; we then calculated effect sizes for group comparisons using Cohen’s d and unadjusted pooled standard deviation. We considered d values ≥ 0.8 to represent large effects, values ranging from 0.5 to 0.8 to represent medium effects, and values ranging from 0.2 to 0.5 to represent small effects (Cohen, 1988).
Analysis of Internal Reliability
Winsteps generates two indicators of internal reliability: person reliability and person separation. Person-reliability indices suggest the reproducibility of relative measure locations (Linacre, 2017). We considered person-reliability indices ≥ 0.8 to indicate adequate internal reliability. We also evaluated person-separation indices, which can be translated to strata using the following formula:
We calculated Cronbach’s α for each test and subscale, excluding subscales that comprised only 1 item. We considered α ≥ .70 to indicate adequate internal consistency and α ≥ .80 to indicate strong internal consistency (Gliner et al., 2017).
Results
We eliminated 8 items from O: M&Pr, 9 items from both PC and BI, 12 items from Prop: JP, and 11 items from Prop: F; we also collapsed rating scales and established maximum scores for Bal, Prop: F, and Prop: JP. Table 1 shows finalized scoring procedures. We found that Bal did not have enough difficult items; therefore, we added 4 items requiring more balance ability (e.g., balancing on tiptoes). The following results reflect finalized test versions, excluding added items on Bal.
Construct Validity
Table 2 contains summary statistics for the six finalized Rasch models. All revised tests had uniformly positive point–measure correlations. On Prop: F, Prop: JP, and PC, all items fit the Rasch expectations. O: M&Pr, Bal, and BI had 95.5%, 91.7%, and 86.7% item fit, respectively. Overall, >96% of items fit the expectations of the Rasch model.
Construct Validity: Rasch Analyses
Note. Bal = Balance; BI = Bilateral Integration; O: M&Pr = Ocular Motor & Praxis; PC = Postural Control; Prop: F = Proprioceptive: Force; Prop: JP = Proprioceptive: Joint Position.
Table 3 shows the known-groups analyses. The TYP group scored statistically significantly better on five total test scores and on all but three subscales. We did not compute total score analyses for Prop: F because of divergent scoring mechanisms in the subscales. Each of the three subscales that did not demonstrate significant group differences trended toward the TYP group scoring more favorably: Rolling With Hands and Rolling With Feet subscales of Prop: F (F[1, 207] = 3.62, p = .059, and F[1, 212] = 2.33, p = .128, respectively) and the Two Hands subscale of Prop: JP (F[1, 206] = 3.66, p = .057). For included total scores, effect sizes ranged from d = 0.31 to 1.20. For subscale scores, effect sizes ranged from d = 0.20 to 1.31.
Construct Validity: Known-Groups Analyses
Note. Not all participants were able to complete all items on the tests (because of age or difficulty completing some test items), so sample sizes differ across analyses. Bal = Balance; BI = Bilateral Integration; O: M&Pr = Ocular Motor & Praxis; PC = Postural Control; Prop: F = Proprioceptive: Force; Prop: JP = Proprioceptive: Joint Position; SI = sensory integration concerns group; TYP = typically developing group.
Analyses on original eight items without additional difficult items.
Internal Reliability
Note. Bal = Balance; BI = Bilateral Integration; O: M&Pr = Ocular Motor & Praxis; PC = Postural Control; Prop: F = Proprioceptive: Force; Prop: JP = Proprioceptive: Joint Position.
Single-item scales; Cronbach’s α not calculated.
Analyses on original eight items without additional difficult items.
Prop: F test involves divergent scoring strategies; total Cronbach’s α not conducted.
Internal Reliability
Table 4 provides the results of final reliability analyses. Five tests (O: M&Pr, BI, PC, Bal, Prop: JP) demonstrated strong evidence for internal consistency on the basis of Cronbach’s α ≥ .80. Eight of 16 subscale scores with more than one item also demonstrated strong evidence for internal consistency, whereas 4 demonstrated adequate internal consistency (α ≥ .70). Two subscale scores in Prop: JP approached our criteria (One Hand, α = .68; One Foot, α = .68). Both multiple-item subscale scores in Prop: F fell below criteria (Rolling [Hands], α = .33; Crayon, α = .58).
O: M&Pr, BI, Bal, and Prop: JP demonstrated strong evidence for internal reliability on the basis of Rasch person-reliability indices ≥ 0.80 and strata > 3. PC demonstrated moderate internal reliability (person-reliability index = 0.67; strata = 2.29). Prop: F demonstrated inadequate evidence for internal reliability (person-reliability index = 0.51; strata = 1.68).
Discussion
Psychometrically sound assessments are important to occupational therapy practice, providing valuable insight into the factors that contribute to participation challenges. The EASI adds to occupational therapy practitioners’ assessment toolbox by providing valid and reliable measures of sensory and motor factors that often affect participation. Our results indicate that the EASI vestibular- and proprioceptive-related tests show good to excellent construct validity. Rasch analysis revealed uniformly positive point–measure correlations and fit statistics from >96% of items within the desired range. Among all the tests, only 4 of 101 items failed to conform to Rasch expectations; this small number could reflect chance. Nonetheless, additional data collection and further refinement of these tests are ongoing.
Total raw scores from five of six tests (O: M&Pr, BI, PC, Bal, Prop: JP) yielded statistically significant differences between groups (p < .001) with moderate to strong effect sizes (ds = 0.50–1.20). Although Prop: F did not allow for a total score comparison because of variations in scoring among items, both validity and reliability data were less strong compared with the other tests. The lower validity and reliability may be related to the lack of a total test score because subscores consist of fewer items. In addition, we suspect some confusion about scoring may have occurred during this pilot study; thus, clarifications regarding scoring were made on the basis of the feedback from testers. Future research will reexamine the validity of this test.
Regarding reliability, total scores for five EASI tests measuring vestibular- and proprioceptive-related functions demonstrated strong internal consistency (Cronbach’s αs ranging from .87 to .96) and moderate to strong person reliability per Rasch analysis (.67–.83). Rasch strata values met our criteria of at least 3.0 on four of six tests, indicating that those tests separated participants into three levels of ability. Subscale scores on Prop: F fell below the desired criteria for Rasch reliability indicators and Cronbach’s α. Bal could not be evaluated at this time because additional difficult items were added after data collection, which the analyses had indicated were needed. Further revision of these tests may contribute to stronger test reliability.
This study indicates that the EASI vestibular and proprioceptive tests provide valid and reliable ways to measure often-hidden challenges. Past research has suggested that vestibular–proprioceptive challenges are common in numerous populations (Ayres, 1972a; Isaac et al., 2017; Koester et al., 2014; Mailloux et al., 2011; Roley et al., 2015) and that these sensory integration difficulties affect performance and participation. Occupational therapy practitioners may find these tests useful for identifying people who could benefit from intervention that is designed to optimize vestibular–proprioceptive function as a foundation for successful participation.
Limitations and Future Directions
A limitation of this study is that participating children were from a convenience sample. In future studies, we will collect data from representative samples of children ages 3 to 12 yr from the United States and from an international sample. Additionally, we did not stratify results by age, race, diagnostic group, or other demographic variables for this pilot evaluation. In future studies and normative data collection, we will evaluate the external validity of data collected using these instruments. Additional studies related to test–retest and interrater reliability, as well as congruent validity comparing the EASI to tests such as the SIPT and the SPM, are currently underway.
In addition to strong psychometric properties, occupational therapy assessments must be clinically feasible. Before our revisions, these six instruments took approximately 1 hr to complete. Because an important step in instrument development is the selection of the strongest items, the tests administered in this study included more items than will be included in the final version of the tests; therefore, the tests required more time than the final tests will take to complete. In addition, because we do not yet have normative data that are extensive enough to determine basal and ceiling levels by age, we administered all items with all participants in this study. As we continue developing the instrument, we anticipate that administration time will decrease because some items will not be administered to younger age groups if data indicate that the items are too difficult (or to older age groups if too easy).
International normative data collection on all 20 tests of the EASI is underway with plans for the final version of the tests to be available in the future. An important goal of the EASI is to reduce social and economic disparities related to access to assessment. The EASI will be an “open access” test, meaning that it will not be owned by a publisher. The test manual, test sheets and forms, test materials, and scoring program will be made available to appropriately trained professionals at low or no cost. Ultimately, the goal of EASI test development is to make comprehensive assessment of sensory integration functions accessible so that children will benefit from appropriate and effective intervention planning.
Implications for Occupational Therapy Practice
The results of this study have the following implications for occupational therapy practice: The six EASI vestibular and proprioceptive tests are feasible to administer to children with and without sensorimotor concerns. Four of the six EASI vestibular and proprioceptive tests demonstrate strong reliability and validity; thus, they can be used with confidence to identify children who have vestibular–proprioceptive difficulties that interfere with occupations such as play, self-care, and academic work. Ongoing research on the other two tests may demonstrate stronger reliability and validity after further revision. Difficulties in vestibular and proprioceptive functions often affect participation in daily life for children and their families. Psychometrically sound assessment tools that assess these functions will assist occupational therapy practitioners in implementing effective, individually tailored interventions (Mailloux & Miller-Kuhaneck, 2014).
Conclusion
The EASI vestibular and proprioceptive tests are appropriate measures for assessing these functions in children ages 3 to 12 years. Because vestibular and proprioceptive functions are associated with many skills needed for successful participation and engagement in daily life, including performance at school, these tests provide essential information for intervention planning for children with sensory integrative and other developmental disorders.
Footnotes
Acknowledgments
We thank the testers, families, and children who participated in this study. We also acknowledge statistical assistance from Ben Leibly, Thomas Jefferson University, and Steven Paul, University of California, San Francisco. We have no conflicts of interest to disclose.
