Identification of Sensitive Outcome Measures of Participation for Children With Autism

PURPOSE: We sought to identify and test performance-based outcome measures of participation—specifically, daily living skills and socialization—that are feasible to implement in a larger comparative effectiveness study of children with autism spectrum disorder (ASD) ages 6–9 yr. The feasibility and discriminant validity for the chosen measures are also explored.

DESIGN: This study used mixed methods including a systematic literature review, expert rating using a quality indicator scale, and a nominal group process with expert consensus methods to identify and rate measures. A convenience sample of expert raters participated in the review and rating of the instruments. Inclusion criteria for instruments were that the measure (1) evaluates daily living skills and/or socialization, (2) is appropriate for use with children with ASD ages 6–9 yr, (3) is performance based, and (4) is norm referenced for use with children with ASD. In Phase 2 of the study, the top instruments were evaluated for feasibility using a feasibility rating scale. Discriminant validity was explored by examining whether the chosen instruments discriminated between two known groups within the sample: children with high ASD severity versus those with low severity (severity is based on Autism Diagnostic Observation Schedule severity score). Means and standard deviations for the chosen instruments were calculated for each of the two groups, and t tests will be used to determine if mean scores differ between the two groups.

METHOD: A systematic review of the literature was completed to identify currently available performance-based assessments of activities of daily living and socialization for children with ASD that met the inclusion criteria. The search was initiated using a PICO format to determine key search terms for the following electronic databases: PubMed, CINAHL, Google, OTsearch, and American Occupational Therapy Association resources. We also referred to an assessment resource (Asher, 2014) and inquired with professional colleagues to identify instruments.

Next, a quality indicator scale was developed based on criteria described by Law and MacDermid (2013) and Coster (2013), and this scale was used by the experts to rate instruments. The expert panel then participated in a consensus meeting to confirm the top two instruments. For Phase 2 of the study, the instruments were administered to 20 ethnically diverse children with ASD. Feasibility was rated on a scale of 1 (lowest) to 3 (highest) and weighted according to level of importance. Qualitative data from the assessors regarding their impressions were also collected. Discriminant validity was assessed.

RESULTS: Seven performance-based outcome measures of daily living skills or socialization were identified from the literature search. Quality indicator ratings showed that the top-rated instruments that met most criteria were the Assessment of Motor and Process Skills (AMPS) for daily living skills outcomes (Fisher & Jones, 2012) and the Evaluation of Social Interaction (ESI) for socialization outcomes (Fisher & Griswold, 2010). In the consensus meeting, the experts agreed on use of these measures for the future study. Feasibility data revealed that the instruments were acceptable to participants, feasible to implement, and had strong interpretability. One hundred percent of the children were able to complete the assessments. Comparison of scores between those with high and low autism severity showed that the tests had moderate strength to detect differences between the groups.

CONCLUSION: Seven performance-based measures met the study’s inclusion criteria. The AMPS and the ESI were top rated by experts who also reached consensus for use in a larger comparative effectiveness study of children with ASD ages 6–9 yr. The measures showed adequate feasibility and discriminant validity.

References

Asher, I. E. (2014). Asher’s occupational therapy assessment tools (4th ed.). Bethesda, MD: AOTA Press.

Coster, W. J. (2013). Making the best match: Selecting outcome measures for clinical trials and outcome studies. American Journal of Occupational Therapy, 67, 162–170. https://doi.org/10.5014/ajot.2013.006015

Fisher, A. G., & Griswold, L. A. (2010). Evaluation of Social Interaction (3rd ed., rev.). Fort Collins, CO: Third Star Press.

Fisher, A. G., & Jones, K. B. (2012). Assessment of Motor and Process Skills: User’s manual (8th ed.). Fort Collins, CO: Three Star Press.

Law, M., & MacDermid, J. (2013). Evidence-based rehabilitation: A guide to practice (3rd ed.). Thorofare, NJ: Slack.

Schaaf, R. (2015). Creating evidence for practice using data-driven decision making. American Journal of Occupational Therapy, 69, 6902360010. https://doi.org/10.5014/ajot.2015.010561