Abstract
The results of this study can serve as a framework to improve education, practice, and clinical research in outpatient occupational therapy for children on the autism spectrum.
Occupational therapy is an integral component of comprehensive interventions designed for children on the autism spectrum (Monz et al., 2019). Autism is widely accepted as a heterogeneous condition with multiple subtypes and developmental trajectories (Masi et al., 2017). Given the remarkable heterogeneity in autism, therapists often use a myriad of treatment approaches tailored to each child’s unique profile. Occupational therapy interventions primarily address family-centered goals, including functional skills, performance and participation in activities of daily living, and quality of life (Tomchek & Koenig, 2016). Analyses of rehabilitative interventions, including occupational therapy, have been categorized on two levels: macro, which conceptually identifies the type of intervention, and micro, which defines the treatment components and active ingredients of the intervention (Whyte & Hart, 2003). Types of evidence-based occupational therapy interventions for autism include ▪ sensory integration and sensory-based, ▪ interactive relationship–based, ▪ developmental skill–based, ▪ social-cognitive skill–based, ▪ parent-directed or parent-mediated, and ▪ intensive behavioral-based interventions (Case-Smith & Arbesman, 2008).
The Rehabilitation Treatment Specification System (RTSS) is a novel theoretical framework that provides a universal system for characterizing rehabilitation intervention through a process of explicitly stating the hypothesized active ingredients, treatment targets, and how an ingredient directly affects the treatment targets (Van Stan et al., 2021a). According to the RTSS, a treatment component is the smallest unit of treatment linked to a treatment theory. Each treatment component has a singular treatment target and can include one or more active ingredients (Van Stan et al., 2021a). A treatment target is a specific patient function that is observable and measurable, whereas active ingredients are specific elements of an intervention that are thought to facilitate change (Van Stan et al., 2021a). In rehabilitation, research has made significant advancements in the macro analysis of interventions; however, there has been less progress in the micro analysis, such as the characterization of effective treatment components and ingredients that produce outcomes (Van Stan et al., 2021a; Whyte & Hart, 2003).
Routine or usual care in rehabilitation often involves the simultaneous use of multiple different treatment types, leading to the notion of a “black box” of treatment components and active ingredients (Whyte & Hart, 2003). In the context of rehabilitation, usual care typically refers to the standard of care that clients would typically receive in routine clinical practice without any additional experimental or specialized interventions (Arienti et al., 2022). Usual care serves as the baseline against which the efficacy of new interventions or treatments is often compared. In real-world practice, therapists often pick and choose specific components of evidence-based treatments that are most appropriate for their client and setting rather than implementing a single treatment type with fidelity (Stahmer et al., 2005). This practice can lead to difficulties in describing interventions and result in unintended variability in intervention delivery. Thus, an accurate and effective characterization of usual-care services that children on the autism spectrum receive in their communities is important for rigorous definition of treatment components that are central to tenets of occupational therapy intervention and that span multiple treatment frameworks. Clearly defined treatment components of usual-care occupational therapy intervention for children on the autism spectrum will enable consistent and accurate descriptions of interventions, link therapy to outcomes, and evaluate the relationship between treatment approach and autism subtypes. Although interventions are individualized for each child, an empirically validated process to guide intervention decisions on the basis of a child’s person–environment–occupation factors is critical to ensure use of the highest quality and most appropriate interventions.
In the present study, we sought to define key components of occupational therapy intervention for children on the autism spectrum to enable accurate documentation of the smallest unit of treatment employed during therapy, based on the RTSS framework. The RTSS framework of defining key treatment components is particularly applicable in occupational therapy for autism, given the heterogeneity within the autism spectrum (Reynolds, 2023). A recent scoping review using the RTSS framework to map the level of specificity of research on Ayres Sensory Integration® intervention noted the usefulness of RTSS in describing interventions and increasing research translation (Choudhury et al., 2024). Defining key treatment components helps to avoid the pitfall of different words and terms that describe the same component and helps to facilitate the systematic characterization of usual care. In addition, consensus on definitions of key components will enable the investigation of mechanisms of action and which components are associated with positive outcomes (Stahmer et al., 2016), particularly with regard to how individual treatment components are implemented in real-world practice; and which components are most often used together. Ultimately, systematic characterization of usual care will allow refining therapy, increase implementation of evidence-based practice, and improve the service delivery infrastructure. Taken together, there is a critical need to develop an operationalized list of key treatment components of usual-care occupational therapy for children on the autism spectrum. This study used the Delphi process to create a comprehensive list of treatment components used in usual-care occupational therapy services for children ages 6 to 13 yr on the autism spectrum and establish standard names, definitions, and examples for each component. This age range was chosen to focus on standard outpatient and community-based occupational therapy services for school-age children because early intervention (for children from birth to age 5 yr) and adolescent–adult services may involve different treatment components. Additionally, this age range was chosen as part of a larger ongoing study focusing on middle childhood.
Method
Developing an Initial List of Treatment Components
On the basis of emerging research using the RTSS (Hart et al., 2014; Van Stan et al., 2021b) and prior work characterizing evidence-based autism interventions (Stahmer et al., 2019), an initial list of key treatment components with definitions and examples was developed. We used multiple strategies to identify relevant prior work, including systematic analyses of reviews of occupational therapy interventions for autism, occupational therapy practice guidelines for people on the autism spectrum (American Occupational Therapy Association [AOTA], 2022; Tomchek & Koenig, 2016), published treatment protocols, and documentation of clinical care and qualitative methods. Identifying distinct treatment components within occupational therapy could result in theoretically infinite possibilities, given considerations of factors such as different client characteristics and settings; hence, we developed this initial list to broadly cover the occupational therapy components that are “common” within and across evidence-based treatment frameworks (Van Stan et al., 2021a).
Delphi Method
The Delphi method is a systematic interactive process of consensus building wherein a panel of experts provides feedback through one or more rounds of a structured questionnaire (Diamond et al., 2014; Hsu & Sandford, 2007). Consensus was defined as 80% agreement across all panelists. Four methodological characteristics of the Delphi process contribute to the rigor of this method to develop consensus on treatment components (Jünger et al., 2017). First, panelists consisted of a group of experts with diverse backgrounds. Second, the anonymous nature of the feedback allowed each panelist to contribute without undue influence from other members. Third, each round used a standardized format to collect responses, and the product was revised on the basis of the panelists’ feedback at each round. Last, there was a structured feedback loop between investigators and panelists. The feedback loop began with the Round 1 review of a structured criteria evaluation questionnaire by the panelists that was used to evaluate the product. Next, the investigators consolidated and analyzed the panelists’ ratings and comments on the product with the revised criteria evaluation questionnaire and revised the product accordingly. Investigators then provided panelists with a summary of their individual- and group-level comments and described revisions made to the product. Round 2 used the revised product and followed a similar procedure, and subsequent rounds continued until consensus (at least 80% agreement on ratings) was achieved. To establish face and content validity, experts were iteratively asked if the treatment components accurately and exhaustively covered usual-care occupational therapy in autism. Participant identities were kept concealed from each other, and anonymity during the feedback was maintained by synthesizing and summarizing the responses without attributing them to individual participants.
Delphi Panel Participants
Expert occupational therapy practitioners were identified through the AOTA’s Board Certification in Pediatrics list, research expertise in occupational therapy interventions, and through word of mouth. Inclusion criteria included at least 3 yr of experience working with children on the autism spectrum. A total of 17 expert panelists agreed to participate in the study. One panelist only completed Rounds 1 and 2, and 16 panelists completed all four rounds. The panelists’ demographics are presented in Table 1. The Delphi method has been successfully used to identify treatment components with the RTSS framework (Van Stan et al., 2021a) and in behavioral interventions for autism (Stahmer et al., 2019). Study procedures were approved by Ohio State University’s Institutional Review Board. Participants received $50 in compensation for participating in the study. Surveys were administered by means of Research Electronic Data Capture (REDCap).
Demographic Information of Delphi Panelists
Note. N = 17. AOTA = American Occupational Therapy Association; EdD = doctor of education; OTD = occupational therapy doctorate; PhD = doctor of philosophy; ScD = doctor of science.
Panelists endorsed more than one category or range.
Developing the Criteria Rating Form
In Round 1 of the Delphi review, panelists provided feedback on the criteria rating form that would be used to evaluate the definition and example of treatment components in the initial list. The initial criteria rating form included five questions, and for each criterion question, panelists responded whether this criterion was necessary to achieve project goals (yes–no) and provided comments in a free-text response. On the basis of feedback from panelists, the criteria rating form was revised and included four questions scored on a 4-point Likert scale ranging from 1 (strongly disagree) to 4 (strongly agree): Does the definition fully encapsulate the treatment component? Is the language of the definition unambiguous? Does the example fully encapsulate the treatment component? Is this component relevant to occupational therapy practice in children (ages 6–13 yr) on the autism spectrum?
To view the form, see the Supplemental Material, Appendix A (available online with this article at https://research.aota.org/ajot).
Developing Consensus on Definitions and Examples of Treatment Components
The second Delphi round included the initial list of 25 treatment components with definitions and examples. The initial and subsequent lists are presented in the Supplemental Material, Appendix B. Panelists were asked to conceptualize treatment components that are common to usual-care services at outpatient and community-based occupational therapy delivery, as well as evidence-based treatments and frames of reference, regardless of the evidence level for a specific component. Panelists were asked to review and rate each treatment component with the criteria rating form. If they disagreed or had comments, they were asked to provide specific feedback or alternative wording in a free-text response. At each round, panelists were also asked for suggestions for treatment components that were not included in the list with a free-text response. Each survey provided a description of the project goals with a definition of the terms treatment component and active ingredient.
Rounds continued until a supramajority consensus of above 80% agreement—scores of 3 (agree) and 4 (strongly agree) on the criteria rating form—on a treatment component was reached. After each round, the research team reviewed and analyzed responses and provided the panelists with a summary of their individual responses and the group’s results, stating the percentages of consensus on each criterion per treatment component, common themes derived from comments, and adjustments that were made to the list. Panelists were then asked to complete the criteria rating form for the revised list. Panelists were given 2 to 3 wk to complete each round of evaluation. For Round 4, only the treatment components and questions that did not achieve consensus were sent for further review. After this round, 3 panelists who disagreed with a specific component were invited to a virtual meeting to discuss their feedback. Meetings began with an explanation of the goal of the study, and then panelists were invited to clarify their ratings and comments and necessary revisions were discussed. The individual meetings allowed for comprehensive and interactive dialogue. Survey Rounds 1 to 3 took about 1 to 2 hr to complete, and the final round took about 15 to 30 min to complete. The individual meetings lasted approximately 30 min.
Results
Criteria Rating Form
The five proposed evaluation criteria were endorsed by 80% to 100% of 17 responding panelists. We used ratings and qualitative feedback to revise the criteria. For example, “Does this component avoid unnecessary overlap with other components?” received the lowest level of endorsement (80%), with comments noting that overlap is a natural factor of occupational therapy strategies and, hence, this criterion was dropped from later rounds (see the Supplemental Material, Appendix A).
Rating the Treatment Components
Through four Delphi rounds of consensus building, we developed an operationalized list of 20 treatment components with definitions and examples (see Table 2). Through the iterative feedback process with expert occupational therapy pediatric practitioners, we concluded that the individual treatment components were unique and common to evidence-based treatment frameworks. After revising the list, definitions, and examples using panelist feedback, we removed nine components and added four components. We obtained consensus (>80% agreement) on the definitions and examples of all treatment components. For the treatment component, Reflex Integration, although we obtained consensus on the definition and example, only 75% of the respondents agreed that this component was relevant to occupational therapy practice, whereas 25% continued to disagree about its relevance (see the Supplemental Material, Appendix C, for the final list of treatment components with definitions and examples).
List of Active Ingredients and Consensus on Each Delphi Round
Note. Percentages in italics represent components that did not reach 80% consensus. Percentages in bold type represent components presented to panelists in Round 4, which included items that did not achieve 80% consensus in prior rounds. Dashes indicate that components were not part of Round 2. OT = occupational therapy.
Discussion
Through four rounds of feedback from expert panelists utilizing the Delphi method, we developed an operationalized and comprehensive list of 20 relevant treatment components, definitions, and examples used in usual-care occupational therapy with children on the autism spectrum. Evidence-based occupational therapy practice comprises these treatment components that identify therapist behavior; that is, what the therapist actually does to facilitate function or adaptive behavior in the client. Agreement on the makeup, definition, and examples of a comprehensive set of treatment components is critical and significant, because treatment terms are often used variably across settings and disciplines. The resultant list of treatment components can serve as a foundation for developing standardized documentation practices in community-based service delivery.
As with most complex rehabilitation interventions with heterogenous clinical conditions, therapists in pediatric occupational therapy for children on the autism spectrum often use a wide variety of treatment components and pieces of approaches rather than systematic whole interventions (Van Stan et al., 2021a; Watling & Daughton, 2023; Watling et al., 1999). It is often challenging for practitioners to implement a specific manualized intervention with fidelity in community settings (Stahmer, 2007; Stahmer et al., 2005). Without an evidence-based operationalized list of treatment components, researchers and therapists lack an objective way to identify and measure real-world practice patterns. Furthermore, without being able to accurately characterize and track the specific treatment components used in each therapeutic session, it is difficult to study the most effective approaches among children on the autism spectrum. This standard operationalized list can reduce semantic ambiguity in occupational therapy terminology. Researchers can use the treatment components in this list (with their operationalized definitions and examples) to describe new or existing evidence-based or manualized interventions in their research protocols. Researchers can then characterize the similarities and differences between usual-care practice which often serves as a control group in clinical trials and their experimental intervention. Using this standard framework could help establish better protocols to improve the empirical evidence supporting interventions for children on the autism spectrum (Whyte et al., 2021).
Results from this study provide a framework to examine practice patterns in usual-care occupational therapy intervention, which will also enable empirical investigation of the relationship between child characteristics and treatment components that are most effective. Prior Delphi studies on treatment of children on the autism spectrum have identified key strategies that maximize child engagement (Stahmer et al., 2019). Similar studies in the realm of usual-care occupational therapy using the framework developed through our Delphi study will provide critical and much-needed insight into the most effective treatment strategies for children on the autism spectrum and, ultimately, build the evidence base for occupational therapy.
Evidence-based occupational therapy interventions for autism have been categorized as ▪ sensory integration and sensory–based; ▪ interactive relationship–based; ▪ developmental skill–based; ▪ social cognitive skill–based; ▪ parent-directed or parent-mediated; and ▪ intensive behavioral–based (Case-Smith & Arbesman, 2008).
The micro treatment components operationalized in our study can be categorized within these six macro areas. Sensory integration and sensory-based interventions would include the treatment component sensory activities or sensory equipment; this component is used in autism interventions to provide sensory-rich experiences in which the child learns to integrate sensory stimulation and develops an adaptive response (Schaaf & Case-Smith, 2014). Interactive relationship–based interventions would include the treatment components of adult imitation of child’s actions, child-led or -directed, and therapeutic use of self, where the goal is to facilitate the child’s social engagement and joint attention skills to improve co-occupation participation (Case-Smith & Arbesman, 2008). Developmental skill-based interventions would include gross and fine motor skills, motor planning and praxis, activities of daily living, and reflex integration. Social cognitive skill–based interventions would include cognitive-based strategies, self-regulation strategies, and social narratives (Cheung et al., 2018). Parent- directed or parent-mediated interventions would include parent education and parent-mediated intervention coaching, where the therapist educates and coaches the parent or caregiver on how to modify the home context to best support their child and expand the therapeutic environment to the home (Foster et al., 2013). Intensive behavioral–based interventions would include the key treatment components of visual support, reinforcement-based strategies, and repetitions (Case-Smith & Arbesman, 2008). The treatment components of assistive technology and environmental arrangement are also supported under the treatment scope of performance and participation in activities of daily living (Tomchek & Koenig, 2016). The treatment components that our expert panel identified and deemed relevant to current usual- care occupational therapy practice for children on the autism spectrum align with current evidence-based treatment approaches and components (Althoff et al., 2019; Kuhaneck et al., 2020; Watroba et al., 2023).
Our study highlights the use of the Delphi method to probe key treatment components used in usual-care occupational therapy practice across diagnoses and settings. Consistent with our findings here, this methodology has been successfully used in rehabilitation interventions to yield consensus labels for unique treatment components and operationalize definitions and examples while preserving the flexibility of tailoring the component (Stahmer et al., 2019; Van Stan et al., 2021a). More important, this work emphasizes the need for careful investigation into practice patterns of multicomponent and complex occupational therapy intervention. Developing standardized and operationalized treatment components that are tailored for specific diagnoses is critical for advancing rehabilitation approaches as we seek to systematically evaluate real-world practice patterns and identify key components of effective usual-care interventions. Our study adds to the growing research aiming to develop consensus around terminology relevant to occupational therapy practice (Backman et al., 2021; Miller et al., 2007).
Limitations and Strengths
Because of the nature of the Delphi method, a relatively small sample size of panelists is required to synthesize feedback and reach consensus. The sample size of 17 pediatric occupational therapy practitioners allowed for a thorough analysis of detailed feedback and discussion; however, our sample was limited in quantity and ethnic diversity. Furthermore, the majority of our panelists (70%) had more than 15 yr of experience in pediatric occupational therapy working with children on the autism spectrum, and none of them had less than 6 yr of experience in the field. Although this length of experience is beneficial for the expert feedback, we may be missing the valuable experience of practitioners who recently graduated and learned new or refined approaches for this population. A strength of the present study is its focus on children between the ages of 6 and 13 yr; establishing consensus across all of childhood may have proven difficult and would have been inappropriate, given the developmental milestones that are associated with different periods of childhood. Further research is required to explore the treatment components for occupational therapy for people on the autism spectrum in early childhood, adolescence, and adulthood. Although the initial list of treatment components included evidence-based strategies, which were based on feedback from the panelists, other treatment components with weaker levels of evidence were included because they were deemed part of usual-care occupational therapy practice by the panelists.
Future Directions
Future research should involve diverse expert feedback, including that from occupational therapy practitioners with additional ethnic backgrounds, in different settings, and with a wide range of years of experience with pediatrics. Researchers should also explore the treatment components occupational therapy practitioners report using with children on the autism spectrum outside of the age range of 6 to 13 yr and in different settings. This information will offer insight into patterns of occupational therapy intervention for people of various ages on the autism spectrum and can lead to a holistic understanding of the continuum of care throughout the lifespan. An operationalized list of occupational therapy treatment components would allow the development of standard documentation practices across settings. The incorporation of operationalized definitions of usual-care occupational therapy practice for autism into real-world clinical care systems such as electronic health records (EHRs) will enable us to evaluate practice patterns beyond currently used self-report surveys and questionnaires that rely on inconsistent use of terminology to describe the most basic ingredients of treatment. Currently, occupational therapy EHR data are unstructured, fragmented, are variable, and lack standard treatment components (Bertagnolli et al., 2020). EHR databases could use this operationalized list as a template for documenting usual-care therapy services, which will allow researchers to leverage EHR data to evaluate occupational therapy practice patterns across multiple health systems and settings. Researchers who use this list to describe usual-care or experimental interventions must also develop further guidelines to ensure treatment fidelity. Additionally, future research warrants mapping the operationalized list of treatment components to the RTSS to further probe mechanisms of action and treatment targets for each treatment component. Finally, further research is needed to test the validity of the treatment components across a wide range of community-based practice settings across geographic regions.
Implications for Occupational Therapy Practice
The results of this Delphi study have the following implications for occupational therapy practice: ▪ Occupational therapy practitioners can use this operationalized list of definitions and examples in everyday usual-care clinical practice with children on the autism spectrum (ages 6–13 yr) as a foundational framework to describe, document, and track key treatment components used across sessions. ▪ In clinical settings, the use of key treatment components as defined in this list can enhance communication with interdisciplinary teams. ▪ Occupational therapy practitioners can use this framework to document key treatment components that are used across different types of evidence-based interventions.
Conclusions
This study represents a first step toward characterizing key treatment components of usual-care occupational therapy practice for children on the autism spectrum. This operationalized list of treatment components can serve as a foundational framework to improve education, practice, and research of occupational therapy intervention for children on the autism spectrum. Identification of the key treatment components in usual-care occupational therapy for children on the autism spectrum will allow us to systematically and empirically evaluate the efficacy of treatment protocols. In addition, examination of key treatment component use in real-world practice and its association with outcomes may lead toward personalized care and improved outcomes for children on the autism spectrum.
Supplemental Material
Supplementary material for Characterizing Occupational Therapy Intervention for Children on the Autism Spectrum
Supplementary material, sj-pdf-1-aot-10.5014_ajot.2024.050734.pdf for Characterizing Occupational Therapy Intervention for Children on the Autism Spectrum by Jewel E. Crasta, Joseph Martis, Meredith Kromalic, Shannon Jarrott, Lauren Wengerd and Amy Darragh in The American Journal of Occupational Therapy
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
