A Well Walked Path to Program Efficacy

About a month ago I attended an Individualized Education Program (IEP) meeting for the 3-year-old son of a friend. I was mystified to see a calculator by the preschool teacher’s side, and as service recommendations were made, she added numbers on the calculator. I started to get more concerned when the team focused on assigning different staff (by discipline) to “work” with little Joey, instead of designing outcomes and interventions to meet his developmental and behavioral needs. When I asked whether there were outcomes assigned to each discipline, or whether the team developed collaborative outcomes, I was told that they had to get to 20 hr of service first, as recommended by the out-of-district neuropsychologist and then they would assign outcomes. I was also told that this was the more difficult task, as the district usually offered preschool children with disabilities a 4-day per week program for 3 hr a day, which totaled 12 hr. To make up the difference, the special education director was going to assign Joey an additional paraprofessional (besides the one he would have in the preschool special education classroom), so he could attend a kindergarten class 2 afternoons a week (6 hr), he could also come in to school early to have speech therapy for half an hour 2 times a week, and he could have lunch with the occupational therapist on the 2 days that he stayed for kindergarten (for half an hour, 2 times a week). They all breathed sighs of relief when the preschool teacher said they got to 20, and I understood why they needed the calculator! I was told how difficult it was to get these program hours in place for the children who qualified for them. When I then asked if we could now talk about outcomes, I was told they didn’t need to, as Joey was going to have Applied Behavior Analysis (ABA) discrete trial training from a certified behavior specialist. I asked if my friend could talk about her priorities. We were told sure, but it was not really appropriate to talk about home issues, as they were only responsible for the 20 hr of in-school time.

Why the focus on hours and methodology only? Joey has a diagnosis of autism, and because of this, he was given additional preschool service delivery hours, and ABA is often used as the intervention of choice for children with autism. As it has been 35 years since I taught my first preschool student diagnosed with autism, I sat in the meeting wondering why a preschool service delivery team would be prioritizing contact hours and discrete trial learning for Joey’s special education program, as opposed to intervention content, learning paradigms focused on the use and generalization of functional behaviors, and the collection of data to document measurable outcomes. What became clear to me was that this team, like many nationwide, was struggling to design interventions for a growing population of children being diagnosed with Autism Spectrum Disorder (ASD). It is within the context of this struggle that I was so pleased to review the excellent article by Phil Strain and Ted Bovey titled “Randomized Controlled Trial of the Leap Model of Early Intervention for Young Children With Autism Spectrum Disorder.”

The article presents a large-scale experimental study that is both credible and reliable, and as such makes significant contributions to the literature in early childhood intervention. These contributions go well beyond the main focus of the article, which is a detailed description of the effectiveness of the Learning Experiences and Alternative Programs for Preschoolers and Parents (LEAP) preschool service delivery model for children with ASD. However, I would be remiss if I moved beyond this focus without acknowledging the significance of the study and the outcomes it achieved across children with ASD, families, and service providers. Indeed, the main thrust of the study was an evaluation of a detailed set of intervention practices that, when implemented with fidelity, resulted in statistically significant improvements across a range of measures with the children in the experimental group when compared with a randomly assigned control group, which received a less systematic exposure to the intervention model.

The outcomes that were achieved through this scientifically sound and well-documented design are indeed notable; yet, the study had many laudable implementation features that should be highlighted. For example, the challenges presented by the large numbers of participants across geographically and contextually different classrooms must be acknowledged. This is especially noteworthy as these classrooms were all community- and school-based sites, having the “noise” and complexity associated with such authentic research settings. In addition, the implementation and coordination of model components within each study site deserves both recognition and commendation, as does the fact that the maintenance of program integrity was thoroughly documented throughout study implementation.

Almost 15 years ago, Michael Guralnick (1997) challenged the field of early childhood intervention to move beyond research designs that were focused on singular cause and effect relationships to second-generation designs that were specific and replicable across child and family characteristics, program features and practices, and specified and multiple outcomes. Phil and Ted must be applauded for providing us the first such second-generation design for young children with ASD. This study presents a well-described intervention and evaluation model for programs serving this growing population of young children. As such, I hope it will be recognized as the benchmark for exemplary and evidence-based intervention practices for young children with ASD from this point forward.

However, beyond the significant implications for young children with ASD, the detail provided in the article on the breadth and depth of the model development process also contributes to the knowledge base for the broader field of early childhood intervention. There are many in the field today who may not remember, or have been around, when the Handicapped Children’s Early Education Program guided us through a service delivery demonstration phase to a outreach training phase of model development. During this same time frame, Paine, Bellamy, and Wilcox (1984) proposed a similar, if not more precise, model development process based on their examination of innovative programs in human services. The first phase in this process was the determination of an experimentally proven relationship between one or more independent (intervention) and dependent variables (outcome). These interventions were then applied collectively to demonstrate a solution to a service problem (e.g., the need for effective and efficient classroom models for young children with ASD). If successful as a demonstration, the group of interventions was packaged as program model and replicated and evaluated across a wider audience and settings. The test of model effectiveness and subsequent dissemination was the fidelity with which the model was implemented and the reliability of the results across the wider universe. These authors’ stressed the importance of a model development process that used operational definitions, measurable outcomes, and well-documented interventions that could be replicated with fidelity across variations in program variables. They concluded that the success and long-term sustainability of solutions to service delivery challenges were dependent on a deliberate and thoughtful process that stressed both fidelity to the independent variable and reliability of the dependent variable across persons, conditions, and settings. Only recently has the field of early childhood intervention attended to this important process when scaling up interventions for use across multiple dimensions of service delivery (Dunst & Trivette, 2009b; Fixsen & Blase, 2009; Fixsen, Naoom, Blase, Friedman, & Wallace, 2005; Odom, 2009).

In this article, Phil and Ted have provided a detailed illustration of the model development process in early childhood intervention, beginning with the evolution and validation of the features of the model. Since 1982, Phil and his colleagues have implemented 28 separate studies on separate intervention features of the LEAP model for young children with ASD. These features once represented innovations in early childhood service delivery and through the findings of these and other studies are now considered evidence-based and accepted practices. The innovations include placing children with disabilities into inclusive classrooms representing natural ratios of children with and without disabilities, using peer-mediated interventions, embedding IEP learning objectives and interventions into classroom routines and activities, teaching parents to improve their child’s development and behavior in the home and community, and using applied behavioral analysis as a tool to guide naturalistic interventions and data-based decisions, to name but a few. The evidence supporting these features solidified the foundation of the LEAP model as well as service delivery for any young child with developmental or learning challenges.

Rarely have I seen the elegance of this model development process as well described or implemented as is in this article. Even rarer is the inclusion of such detail on the independent variable or intervention. Rarer still is the attention to such a wide range of outcome variables to establish both child change and also social validity across those who implemented the interventions. By providing such a rich description on the evolution of the LEAP model through the series of research studies on component features, Phil has reminded us of the importance of using a careful and well-documented process when establishing a relationship between interventions and child and family outcomes.

I am well aware (through frequent feedback from my graduate students!) that the younger generations of researchers and service providers may perceive that this process to develop effective intervention models is not applicable to the current world of instant, easy, and oftentimes short-sighted solutions to child, family, and program needs. Indeed, at the risk of offending current users of Internet search engines (of which I am one, if only to augment my aging memory!), it seems that the complexity of challenges presented by children with disabilities and their families are easily reduced to instant and simplistic solutions as presented in a first citation/descriptor provided by a Google search. These solutions are then applied to any child and family presenting even a minor similarity to what was found through the search. Likewise, the use of social networks such as Facebook and Twitter has also generated a problem-solving methodology that relies on stream of consciousness reflections that are given the same weight as scientifically based solutions. Finally, there is also the trend of celebrity experts who present interventions that are given credibility without any scientific merit. It is because of these current practices (each of which have been used this year by students in a leadership course I teach) that I am grateful for the study that is presented in this article. It provides a rationale and guidance to document a comprehensive and scientifically based model development process to address the multitude of issues confronting early childhood intervention today (e.g., intervention integrity, service delivery cost, training and supervision of service providers, etc.).

In addition to the sequence of the model development process, the article also demonstrates the need for detailed and deliberate strategies when teaching multiple audiences to implement successful intervention practices across service delivery sites. The comprehensive detail provided about the type and intensity of the training that occurred to those service providers who participated in both the experimental and control group (Table 1 and appendix) provides invaluable information to those involved in personnel preparation and/or professional development. Although it is not surprising that the intensity of training provided to the experimental group resulted in the fidelity with which the LEAP program features were implemented, of most interest should be the lack of intervention effects demonstrated by the control group. It must be noted that those in the control group were given a level of training intensity that was even beyond standard practice in early childhood intervention (see http://www.uconnucedd.org/projects/per_prep/per_prep.html). These teachers and staff were provided with detailed information on components of the LEAP model such as family training, social skills training, and classroom design and management through intervention manuals, videos, and training presentation materials. These materials were concrete, explanatory, and included multiple exemplars. Yet, those in the control group were not able to replicate LEAP program practices consistently, nor effectively, as demonstrated by both the fidelity data and the child outcome data that were collected throughout the study. The findings are indisputable: Child outcomes were negatively impacted by an undertrained workforce. The significance of this cannot be overestimated!

In contrast, the experimental group that was able to effectively replicate the LEAP model practices received consistent and more intensive contact with an on-site program trainer over the 2 years of study implementation. The LEAP trainer used strategies that have been proven effective in teacher training (Joyce & Showers, 1982), and these included discussions, the modeling of practices, and feedback to the trainee as they demonstrated a practice with a child or group of children. On-site support to the trainee was continued by the on-site supervisor, and follow-up implementation and maintenance checks were also provided periodically by the LEAP trainer.

It is not surprising that these learning activities were effective as they were based on the research on adult learning as summarized by the National Research Council (Donovan, Bransford, & Pellgrino, 2000) and as first described by Knowles (1978). In particular, adults learn best when (a) activities are learner centered; (b) attention is given to what is taught (information), why it is taught (understanding), and what competence or mastery looks like; (c) formative assessments occur frequently; and (d) learning is contextually referenced and occurs in the context it is needed. Although most would agree to the relevance of these strategies, only recently have they been systematically examined and applied as a professional development process in early childhood intervention (Dunst & Trivette, 2009a). The recommended process begins with the introduction and illustration of a new intervention or teaching behavior, continues to the application of the behavior through practice and evaluation, and culminates with the understanding of the behavior through reflection and mastery. Each of these phases were evident in the description of the training provided on the LEAP model, and the providers’ self-efficacy reports and child outcomes demonstrated the effectiveness of using such evidence-based adult learning strategies to insure the fidelity to the model’s program features.

For 6 years, I served as the codirector of the Center to Inform Personnel Preparation Policy and Practice in Early Intervention and Preschool Education. The center was charged with identifying personnel standards and training opportunities available to those serving infants and young children across the country. Unfortunately, our findings documented a lack of key training features, including a lack of available and appropriate content, methods and intensity in university preservice programs (Bruder & Dunst, 2006), and state in- service systems (Bruder, Mogro-Wilson, Stayton, & Dietrich, 2009) as well as a lack of consistent state personnel guidelines and standards (Stayton et al., 2009). These findings were also reflected through the self-reported competence and confidence of 400 early childhood interventionist when implementing intervention practices with infants and young children with disabilities (Bruder, 2010).

In comparison to these findings, it was refreshing to see the adult learning and behavior change that occurred through the training methods used in this study. This rigorous and effective training model gives me cause for optimism about future workforce development in early childhood intervention, an area I am passionate about. The implications from the study are clear: Intense, systematic, relevant, and well-documented training opportunities using evidence-based adult learning strategies resulted in measurable benefits to service providers and the children they served, whereas limited and nonsystematic use of training materials had no measurable effects on service providers or on the children they served. To see improved outcomes in children and families as a result of receiving early childhood intervention, we must provide systematic and effective training to those implementing the interventions. This is a call to action we cannot ignore.

Let me conclude with the expectation that the federal, state, and local policy makers, researchers, service providers, and those in personnel preparation will recognize the substantial contributions this study provides to a field that has been struggling to identify the right variables to address when designing interventions for young children with ASD. Indeed, as there are no studies that provide credible and reliable guidelines for community-based preschool service provision for this population, this article provides a comprehensive description of evidence-based procedures, intervention practices, and outcome measures that should not be limited to children with ASD. I would like to see all preschool classrooms serving young children under Individuals With Disabilities Education Act implement the described interventions and consistently measure outcomes accordingly. I only wish my four daughters, each of whom attended inclusionary public school preschool classrooms, had been able to participate in such an effective model of early childhood education.

A quote by Maya Angelou sums up this article and the contribution it will make to the field of early childhood intervention: “When you know better you do better.” As a result of this study, Joey and his family should have the opportunity to receive an intervention program that is not based on hours of contact in school. To achieve this, his preschool intervention team will have to go beyond a calculator and learn to consistently and reliably implement and evaluate individualized interventions that may be new to their intervention repertoire. This may not be easy, but it is the responsibility of our field to see that children such as Joey receive the best of what we can offer so that he may reach his full potential, and this is what early childhood intervention is all about, isn’t it? In fact, to honor the illustrious career of Phil Strain who has spent his life on the development, implementation, refinement, and sustainability of the LEAP model, I hope those in the field recognize and act on the ethical dilemma he has presented to us. As a result of this study, we now know better. From this point forward, it would be unconscionable to ignore the methodology and findings presented in this article as we design and deliver interventions for young children with ASD or other related disabilities. And we will do better.