Fostering Autonomy: A Technology-Aided Check-In/Check-Out Intervention for Online High School Students With Autism

Self-Management and ASD

Self-management is a common evidence-based practice (EBP) used with students with ASD in the school and home settings. The research base for self-management for students with ASD is extensive (Carr, 2016; Carr et al., 2014). Self-management procedures have been used to address several skills including academic (Smith et al., 2013), daily living (Munsell & Coster, 2018), social and communication (Koegel et al., 2014), and on-task behavior (Stasolla et al., 2014). Several literature reviews and meta-analyses have reported on the general effectiveness of self-management interventions for students with ASD. Carr (2016) reviewed published studies focusing on using self-management interventions to address challenging behaviors. Results of this review demonstrated self-management to be effective at reducing such behaviors as aggression, tantrums, elopement, inappropriate vocalizations, and self-injury.

Students with ASD can face difficulties in self-management during online learning. For example, students may not have the organizational skills to complete assignments or the ability to focus on the lesson being taught for more than a few minutes at a time. The lack of structure or predictability can create higher states of anxiety for students with ASD, often resulting in problem behaviors (Hume et al., 2014). Many difficulties students with ASD encounter in their learning can lead to a reliance on external agents such as teachers and caregivers to provide direction and supervision in daily living and academic activities; this in turn can lead to prompt dependency (Chia et al., 2018). The lack of independent functioning and personal autonomy contributes to poorer overall outcomes for these students and the amount of support they need to successfully participate in virtual learning environments is often not provided. One possible solution to support students with self-management is to implement a Check-In/Check-Out (CICO) intervention.

Check-In/Check-Out

CICO is one of the most widely used and extensively researched secondary interventions for students who do not respond to universal support (Drevon et al., 2019). CICO is typically composed of the following elements (Maggin et al., 2015); First, the student participating in CICO meets with a mentor each morning to check in. The mentor reviews behavioral expectations for the student, sets an attainable goal, and provides the student with a daily progress report (DPR). Second, the student solicits feedback from their teachers throughout the day using the DPR. The teacher records the student’s progress and may give corrective feedback or verbal praise depending on how the student performed. Third, the student checks out with the mentor at the end of the day. During the end-of-day check-out, the mentor reviews the DPR, recognizes the student’s accomplishments, and delivers reinforcements if the student met their goal. Last, the student takes the DPR home for their parents to review with the student and sign. The student brings the signed DPR back to the mentor the next day and the process starts again for the new day.

CICO has been shown to be an effective intervention at the elementary, middle, and high school levels (e.g., Kittelman et al., 2018) as well as for targeting various functions of behavior when used in tandem with a function-based reinforcement system (Turtura et al., 2014). CICO has also been used in non-school environments such as residential and juvenile correction settings (Swoszowski et al., 2012). Despite the abundance of evidence supporting the effectiveness of CICO, there is a lack of research that investigates the effects of implementing CICO with students engaged in online learning.

There are several potential reasons CICO has not been explored in online learning environments. CICO works because it adds increased positive interactions with adults throughout the day (Weber et al., 2019); these interactions provide instruction and practice in self-monitoring of behavior. The lack of a physically present teacher in the home environment shifts the role of the teacher to a parent or other caregiver. However, it may be the case that a parent is simply too busy to learn to implement an intervention that requires increased interaction with their student. They may have to work away from home or may have other children that require their attention. In addition to limited time, parents often lack the skills or resources to effectively support their students in online learning environments (Efstratopoulou et al., 2021). As a result, parents may be either unaware of or inadequately trained to implement common support strategies employed by teachers in physical classrooms. As with any evidence-based practice, for CICO to be successfully adapted for online education, educators and researchers need to consider student and family needs, capacity to assist the student, and the ease of the intervention to be implemented in a dynamic learning and living environment (The IRIS Center, 2014).

One adaptation to assist in implementing CICO for students in online learning environments is the use of technology-aided interventions and instruction (TAII; Odom et al., 2015). TAII are those interventions in which technology is the central feature of an intervention. TAII incorporates a broad range of devices, such as speech-generating devices, smart phones, tablets, and computer-assisted instructional programs. In recent years, many studies have shown that TAII used in the classroom can help improve students’ academic and behavioral outcomes (Chia et al., 2018; Hong et al., 2017; Odom et al., 2015). During learning and instruction, electronic devices such as smartphones, tablets, and computers provide ample use of visual cues and present clearly defined tasks, which can benefit students with ASD (Grynszpan et al., 2014). In addition to having a strong evidence base supporting its effectiveness (Hong et al., 2017), TAII tends to demonstrate high levels of implementation fidelity (e.g., Bruhn, McDaniel, & Kreigh, 2015) and high levels of social validity among students using the support (e.g., Lee et al., 2015). Considering TAIIs can be implemented with fidelity and are preferred by students, it is possible that using TAII with students enrolled in online education may be beneficial for both the students and parents.

Parent Coaching to Support Students

The importance of having parents implement interventions for children with ASD was explored as early as the 1970s by (Lovaas et al., 1973). Significant disruptive behavior problems are commonly targeted for intervention using parent-mediated interventions (Bearss et al., 2015). In their review of literature, Kaat and Lecavalier (2013) reported several studies that have shown parents capable of implementing interventions that reduced behaviors such as tantrums, aggression, noncompliance with routine demands, self-injury, property destruction, and hyperactivity. Other studies have demonstrated effectiveness of parent-implemented interventions focused on functional life skills such as food refusal (Muldoon & Cosbey, 2018), sleep disturbance (Sanberg et al., 2018), and toileting problems (Unlu, 2019). As parents learn to effectively implement an intervention with their child with ASD, they can continue to use this practice throughout their child’s development (Ingersoll et al., 2020).

It is common practice for parents and professionals to work together through on-going parent coaching to determine how to best facilitate the child’s skill acquisition and address the family’s concerns and priorities. Parent coaching promotes the application of intervention techniques from the therapist or professional to the parents (Bearss et al., 2015) and supports parents in maintaining fidelity of the intervention (Wolery, 2011). Parents are uniquely situated to know their children better than outside professionals, however they often need on-going training and support to effectively deliver an intervention. Whether in-person or through telehealth, coaching provides parents with ongoing support to implement interventions effectively. To date, there is a lack of research on parent-implemented interventions in online education settings. As more and more students with ASD enroll in online education, it is becoming increasingly important that parents are given the skills to support their students, such as implementing self-management strategies like CICO.

Purpose of the Current Study

While there are numerous EBPs and supports available for students with ASD in a traditional classroom setting, few interventions have been researched in virtual classroom settings. Identifying those interventions that are effective in online environments are essential for students with ASD to access support necessary for their success. However, there are also significant gaps in the literature this study attempts to address. First, there is limited research pertaining to behavioral interventions for students enrolled in online learning. Second, there is limited research on implementing a CICO intervention with students with ASD. Third, the number of empirical studies that have tested the effectiveness of a technology-aided CICO intervention is virtually nonexistent.

Only two studies have looked at the implementation of CICO with students with ASD specifically. In the first study, Mallory and Hampshire (2022) piloted a technology-aided CICO intervention with a high school student with ASD enrolled in online schooling during the COVID-19 pandemic. The intervention consisted of a mobile app to facilitate CICO, initial parent training on implementing the intervention, and on-going parent coaching. Results showed that student engagement increased after the intervention was implemented; however, the AB single-case design of the study prevented a causal effect from being determined.

The second study, by Carpenter and colleagues (2023), explored the effects of a traditional and adapted CICO on the behavior of students with ASD who have extensive support needs. Four elementary-aged students with ASD received hybrid instruction that included in-person and live remote instruction during school closures during the COVID-19 pandemic. The researchers employed a multiple baseline across participants design that included the implementation of a traditional CICO intervention followed by an adapted CICO. The adapted CICO included individualized modifications to the CICO process, the DPRs, and point goals (e.g. visual of target behavior on the DPR, function-based rewards, additional check-ins). Overall, the effects of traditional and adapted CICO on the adherence to school-wide expectations and challenging behavior showed no functional relation. Two of the four students demonstrated some level of decreased challenging behavior and increased adherence to school-wide expectations with overall increased stability when adapted CICO was introduced. However, all four participants needed further adaptations or additional, more intensive support beyond traditional CICO.

The current study expands the literature on CICO for students with ASD by implementing a modified CICO intervention for students with ASD participating in online learning. The purpose of this study was to examine the effectiveness of a technology-aided, modified CICO intervention in conjunction with a parent coaching program to improve the on-task behavior of high school students with ASD enrolled in full-time online school. The following research questions guided the study: (1) Will the implementation of a technology-aided CICO intervention increase on-task behavior for high school students with ASD enrolled in online high school programs? and (2) Given structured parent coaching sessions, will the parents implement the intervention as intended, as measured by a standardized implementation fidelity checklist?

Participants and Setting

After the study’s proposal was cleared through the university Institutional Review Board (IRB), the researcher contacted appropriate school personnel (e.g., special education directors, special education teachers, or school principals) from area online high schools and asked for nominations of students who would be eligible to participate in the study. The researcher provided school contacts the qualifying criteria for the student to participate in the study and asked to forward an informational handout about the study to the parents. Families electing to participate in the study were directed to contact the researcher using the contact information provided on the handout.

The participants for this study included three students who met the following criteria: (a) currently enrolled in an online high school program, (b) received special education services under the classification of ASD, (c) demonstrated challenges with staying on task, (d) had access to an electronic device (i.e. phone, tablet, or laptop) to access the app used to deliver the intervention, and (e) were able to demonstrate basic technology skills that include downloading and installing a mobile app, sending and receiving text messages, and navigating between pages on a mobile app or website.

In addition to the students, one parent or caregiver for each student was asked to participate and agree to be the interventionist for the study. The parents/caregivers met the following criteria: (a) were able to participate in an initial training with the researcher to learn to implement the intervention being tested, (b) were open to on-going coaching sessions with the researcher as needed for the duration of the study, (c) agreed to participate in three to five interviews with the researcher throughout the study, (d) had access to electronic device (i.e. phone, tablet, or laptop) to access app used to deliver the intervention, and (e) were able to demonstrate basic technology skills that included downloading and installing a mobile app, sending and receiving text messages, and navigating between pages on a mobile app or website.

This study took place in each students’ home. During the study, the researcher conducted remote observations of the students and provided ongoing coaching sessions for the parents via phone call or Zoom video conferencing. Observations were only conducted during the times in which the students were engaging in math coursework to control for the extraneous variable of coursework influencing on-task behavior.

Research Design

A multiple baseline design across participants was used to evaluate a functional relation (Kazdin, 2020; Riley-Tillman et al., 2020) between the intervention and the percentage of on-task behavior for the students. The experimental conditions consisted of a baseline phase and an intervention phase. Baseline procedures continued until an observed pattern of responding was sufficiently consistent to allow for prediction of future responding (Kazdin, 2020). Baselines were considered stable when the data demonstrated little to no variability and trend over at least 3 consecutive sessions (Riley-Tillman et al., 2020). Students participated in the intervention in a staggered timeframe to demonstrate experimental control of the intervention over the dependent variable. After the first student moved into the intervention phase, the following students moved into the intervention phase after the previous student received the intervention for at least 3 sessions. This is in addition to the stable baseline data requirement.

Intervention Procedures

Data Analysis

Implementation Fidelity

To measure fidelity of implementation, a modified CICO Implementation Fidelity Checklist was used (See Table 1). The checklist was based on similar CICO fidelity checklists used by Horner et al. (2004), Hawken and Horner (2003), and March and Horner (2002). The checklists outlined the key components of CICO that could be scored according to whether that component had been carried out completely. To verify implementation fidelity of the intervention, the researcher and the parent collected fidelity data using the CICO Implementation Fidelity Checklist once per week during live observation sessions. For items that included steps that were typically completed outside of the observation session, such as item 9, the researcher accessed the student’s Bloomz profile to identify if the step was in fact completed. A parent coaching booster session pertaining to the items of concern was provided if the parent fell below 88% for two consecutive sessions or if the parent requested a booster session. Booster sessions included identifying areas that are going well, the steps of the routine that the parent was not consistently demonstrating, the researcher modeling the routine, and the parent practicing the routine.

OPEN IN VIEWER

Table 1.

Implementation Fidelity Checklist.

Fidelity Component	Completed?
1. The student checked in with the parent at the beginning of class	Yes No N/A
2. The parent positively acknowledged the student at check in, making sure the student had access to the Bloomz app	Yes No N/A
3. The parent had access to the Bloomz app at the beginning of class	Yes No N/A
4. The parent reviewed the student’s goals with the student at the beginning of class	Yes No N/A
5. The parent provided contingent feedback on Bloomz at least once during the class period	Yes No N/A
6. The student and parent checked-out at the end of the class period	Yes No N/A
7. The parent recorded points on the Bloomz app at the end of the class period	Yes No N/A
8. The parent provided a comment with each point awarded	Yes No N/A
9. The parent provided feedback to the student when all work for the day was completed	Yes No N/A
10. If the student’s point goal was met, the student received reinforcement	Yes No N/A
Percentage (# of Yes/# of items) × 100
Comments & observations

On-Task Behavior

Effect Size

The Tau-U scores for Jose, Brandon, and Rachel were 0.8762 (p = .001), 0.7909 (p = .002) and 0.6058 (p = .02) respectively. The omnibus Tau-U score was 0.75 (p < .001). These data indicate a moderate effect size of the intervention for each student.

Implementation Fidelity

Implementation fidelity for Jose’s mother averaged 88% accuracy throughout the intervention condition, with a range of 75%–100%. Two booster sessions were requested by Jose’s mother after the first two fidelity checks were completed. Her average score of these 2 sessions was 82%. Following the booster sessions, her average fidelity score was 94%. Reliability between the researcher and Jose’s mother across all cases averaged 97% with a range of 88%–100%. Brandon’s father averaged 93% accuracy throughout the intervention condition, with a range of 78%–100%. One booster session was requested by Brandon’s father following the second fidelity check. His average fidelity score prior to the booster session was 89%. Following the booster session, his average fidelity score was 100%. Reliability between the researcher and Brandon’s father for all cases was 100%. Rachel’s mother averaged 86% accuracy throughout the intervention condition, with a range of 78%–100%. One booster session was requested by the researcher following the second fidelity check, as both checks demonstrated 78% fidelity. Following the booster session, her average fidelity score was 100%. Reliability between the researcher and Rachel’s mother for all cases was 89%. Jose’s mother received two booster sessions during the intervention phase, and both Rachel’s mother and Brandon’s father received one booster session.

Social Validity

At the conclusion of the study, all of the students and parents were asked to participate in a brief interview to assess their overall satisfaction with the intervention. The students all reported overall positive perceptions of the study. The aspects they liked the most were the use of the Bloomz app to communicate with their parents and earning reinforcers that they selected. When asked to rate the usefulness of the intervention for supporting them in completing their schoolwork on a scale of 1–10 (1 meaning not at all useful and 10 meaning extremely useful), the responses were varied. Jose reported a 10, Brandon a 6, and Rachel an 8.5.

All of the parents indicated that they thought the intervention was very helpful for themselves and their students. They stated that the intervention was easy to use, the app helped them stay on top of checking in with the students regularly, and the ongoing coaching helped them feel more confident in their ability to support their students. Finally, when asked the same question as the students to rate the usefulness of the intervention for supporting their students in completing their schoolwork, all three parents said 9.

CICO in Online Education

The data collected in this study suggests that implementing a CICO intervention in an online learning environment is feasible and at least moderately effective at improving on-task behavior for students with ASD. CICO is a highly adaptable intervention that has been effectively used in conjunction with other interventions such as peer mentoring (Collins et al., 2016) and self-monitoring (Miller et al., 2015). This study suggests that CICO may also be implemented successfully with parent-implemented intervention and technology-aided intervention. Much of the flexibility in CICO comes from the individualized goals, frequent and consistent feedback on performance, and a choice of reinforcers that are motivating to the student. These areas of adaptation allowed for CICO to be an effective tool for the students in this study by ensuring contextual fit between the intervention and each student’s learning environment (e.g., online school).

The uniqueness of online learning environments compared to traditional classroom settings cannot be overlooked. Traditional classrooms are arranged to provide structure, organization, and stability that promote student learning. A family’s home often lacks this type of structure. There is often a lack of transition cues, unclear expectations for students’ behavior, and numerous distractions that can interfere with the students’ abilities to focus on their schoolwork (Ferri et al., 2020). In fact, research shows that many parents of students enrolled in online schooling lack the skills or resources to adequately support their students’ learning (Borup et al., 2019). The CICO intervention used in this study provided the families support by creating a routine for the students and parents to follow, giving the students clear expectations for their behavior, setting achievable goals for the students to reach, and scheduling frequent and routine times for feedback and the delivery of reinforcement to the students.

The Importance of Parent Coaching

Parents and teachers working to support each other is vital to creating an inclusive learning environment for students with disabilities. The intervention used in this study highlighted this through the use of ongoing parent coaching in conjunction with the CICO intervention for the students. One of the main reasons for the successful implementation of the intervention is that a parent coaching manual was developed to ensure consistent teaching of the CICO intervention to the parents. Johnson and colleagues (2007) suggest that “an essential prerequisite for a multisite study of a behavior therapy intervention is the development of a manual that can be delivered uniformly by competent therapists and is acceptable to parents” (p. 215). By creating a standardized manual to provide coaching to the parents in this study, consistent delivery of the CICO intervention across families was ensured while still allowing for flexibility to be responsive to individual needs of the parents.

The parent coaching procedures used in this study alleviated many of these obstacles for the families who participated. Parents reported feeling that the coaching provided them with a plan to support their students. In addition, the use of technology by teleconferencing and using the Bloomz app allowed coaching to be more accessible for families as meeting with the researcher virtually allowed for greater flexibility in scheduling meetings around the families’ routines. Prior to the start of the intervention, the parents discussed that they mainly relied on reactive approaches to redirect their students when they got off-task rather than preventative approaches to encourage positive self-management behaviors from the start. Frequent coaching presented parents with emotional and practical support that contributed to the parents gaining both the confidence and competence needed to deliver the intervention effectively and efficiently even in the absence of the researcher (Raulston et al., 2019). As parents acquired strategies for helping the students with their self-management skills, parent involvement became less frequent and less intrusive, reducing the likelihood that parents were inadvertently contributing to students’ prompt dependency (Hampshire & Allred, 2018).

Students as Stakeholders

This study also highlights the importance of motivation in self-management interventions for high school students with ASD enrolled in online schooling. The findings suggest that being involved in the development of their own intervention was motivating. The students reported enjoying setting their own goals and choosing their own reinforcers, both of which promoted the development of their self-management skills (Howard et al., 2020). The students were also motivated by the technological component of the intervention. They enjoyed using the Bloomz app to track their progress on their goals and to exchange messages with their parents. These findings add to the growing literature on the motivating nature of technology for students with ASD (e.g., Grynszpan et al., 2014; Vélez-Coto et al., 2017).

Additionally, the inclusion of students as collaborators in the design of the intervention promoted ownership of the intervention. When treated as valued individuals in developing their own interventions, students are more likely to buy-in to the intervention (Mallory et al., 2021). Interventions designed to aid students in online learning must be designed to not only support students’ use of positive behaviors but must also be motivating for the students to want to participate.

Limitations and Future Directions

Trend!

This study has several limitations that require mentioning. First and foremost, the increasing trend during baseline sessions for all three students presents a major limitation of this study. Without a clear and stable baseline, it is more difficult to attribute any subsequent changes to the intervention rather than ongoing trends. To mitigate this limitation, we used visual analysis to observe a variety of elements (e.g. level, average, variability, immediacy of change) and incorporated multiple statistical methods to consider the larger context of the data. Despite this, we cannot rule out every alternative explanation for the observed trend in the intervention phase.

Another significant limitation is a lack of generalizability of obtained effects. Interventions shown to be effective for a single individual may not be effective with other individuals (Kazdin, 2020). Using multiple participants helps to address limitations in that replicating an effect across multiple individuals at various points in time helps to reduce the plausibility of a claim that some external influence resulted in the change. Additionally, this study looks specifically at high school students with ASD enrolled in online school, which limits the generalizability of this study to this population of students. Therefore, to determine if the intervention used in this study would be beneficial for other populations of students, additional studies would need to be conducted using students that belong to different populations.

The implications of this study’s findings are also worth mentioning. One implication pertains to the use of manualized interventions in supporting students with disabilities in online learning. Intervention manuals have been seen as essential for the dissemination and replication of evidence-based practices for decades (e.g., Wilson, 1996). Manuals offer a potentially helpful way to bridge the gap between research and practice and deliver effective interventions in real-world settings. In addition, manualized interventions address the issue of maintaining high levels of implementation fidelity across providers (Sipila-Thomas et al., 2021), which is particularly important for schools who rely on multiple teachers and staff to implement an intervention with students. One of the main benefits of using a manualized Tier 2 intervention such as CICO is that schools typically already have the systems in place needed to implement the intervention with relatively short turnaround. This study explored the effectiveness of a manualized CICO intervention adapted for students enrolled in online learning, but replication of this study is greatly needed to determine if its findings can be reproduced.

An additional implication of this study is the need for a greater understanding of the complexities of providing parent coaching. Many parents are unfamiliar with behavior interventions used in schools, so in order for parents to successfully implement an intervention, there must be a support system in place for the parents to implement any intervention. Parent coaching was essential for the successful implementation of the intervention used in this study. The remote nature of coaching sessions provided greater accessibility to support for the parents who may have otherwise been unable to participate in-person. This may be a useful consideration to support more families, including families who live in rural settings or have other barriers preventing them from participating in ongoing coaching. Future research should consider whether remote parent coaching can be successfully implemented across a variety of families with different backgrounds and demographics.

Finally, further information is needed that identifies what population of students and what types of families would be most successful with this intervention. Cultural differences, perceived social significance, and the relation between the family and coach can all present challenges that either inhibit or enhance the family’s ability or desire to participate in intervention (Chung et al., 2020; Parra-Cardona et al., 2017). The family is typically the main support system for an individual with a disability throughout most of their lives, so providing a family with the knowledge and resources to provide effective support to the individual extends far beyond the family’s current context. Future research should seek to learn more about which families would benefit from coaching and which ones may not.