Program Monitoring Practices for Teachers of the Deaf and Hard of Hearing in Early Intervention

Abstract

Program monitoring is an important and necessary assessment practice within the field of early childhood deaf education. Effective program monitoring requires a focus on both the consistent implementation of intervention strategies (fidelity) and the assessment of children’s ongoing progress in response to interventions (progress monitoring). Teachers of the deaf and hard of hearing (TODs) who provide early intervention services need to conduct regular program monitoring to evaluate the merit of their efforts. However, progress monitoring is a practice often overlooked by practitioners within the field of early intervention. It is recommended that TODs monitor children’s progress “regularly,” but evidence of such practices by TODs is as yet unavailable. In this article, we describe how TODs can use both progress monitoring and fidelity checks to achieve the goal of effective program monitoring for young children who are deaf and hard of hearing.

Keywords

birth to 3 years early intervention deaf/hard of hearing exceptionalities Individualized Family Service Plan (IFSP)intervention strategies assessment

Teachers of the deaf and hard of hearing (TODs) play an important and specialized role in providing early intervention services to infants and toddlers who are deaf and hard of hearing (DHH) and their families. Children who are DHH constitute a unique subgroup of individuals whose language and communication needs often differ from those of hearing children, as well as hearing children with disabilities (American Speech-Language-Hearing Association [ASHA], 2008). Given the high percentage of children who are DHH and also are diagnosed with an additional disability or developmental delay such as vision loss, cerebral palsy, or an intellectual disability (Bhasin, Brocksen, Avchen, & Van Naarden Braun, 2006), TODs need to be well-prepared to address needs beyond those only associated with deafness. In addition to having knowledge of children’s hearing loss, use of amplification and/or augmentation, and preferred mode of communication, TODs need to appreciate how hearing loss influences other developmental skills. Specialized services and interventions are needed to target the social, linguistic, and non-linguistic communication skills of these children and should be sensitive to developmental patterns, auditory amplification options, and mode of communication chosen by parents.

To achieve these goals, TODs need to carry out effective program monitoring, which includes both a focus on the consistent implementation of intervention strategies (fidelity) and the ongoing assessment of children’s progress in response to those interventions (progress monitoring). Identified as a recommended practice by the Division for Early Childhood (DEC) of the Council for Exceptional Children (CEC, A–9; DEC, 2014) and a key component to improving outcomes for children with disabilities, progress monitoring is an assessment practice that can provide frequent and important information about young children’s ongoing advancement toward desired outcomes on Individualized Family Service Plans (IFSPs) and developmental milestones (Walker, Carta, Greenwood, & Buzhardt, 2008). However, limited research exists regarding the use of child progress monitoring among TODs who provide services to infants and toddlers who are DHH. A search of publications pertaining to the assessment practices of TODs in early intervention revealed a paucity of information. Within the broader fields of early intervention and early childhood special education, the literature revealed challenges for practitioners in collecting and using data to make informed decisions. In 2004, Sandall, Schwartz, and Lacroix investigated early interventionists’ and early childhood special educators’ perspectives toward data collection. Although the monitoring of children’s progress through the use of systematic data collection was considered an important and useful practice among practitioners in the study, many practitioners reported using such methods sparingly or not at all in daily practice. A primary reason for this was due to lack of knowledge and skill in systematically collecting data and engaging in data-based decision making. In a similar study, Banerjee and Luckner (2013) examined the assessment practices and training needs of early childhood professionals (e.g., early childhood special educators, early interventionists, administrators, related service providers, general educators). In this study, practitioners reported that lack of time was the greatest challenge in the assessment of young children, followed by lack of developmentally appropriate tools for assessing infants and young children with disabilities. Such studies indicate that more needs to be done to provide early interventionists, including TODs, with the training, tools, and resources they need to engage in effective and consistent child progress monitoring.

In this article, we describe how TODs can use both progress monitoring and intervention fidelity checks to increase the effectiveness of early intervention services for infants and toddlers who are DHH. We begin by addressing the role of TODs in early intervention by examining some of the resources currently available to TODs who provide early intervention services and the importance of recommended and evidence-based practices in early intervention. We then describe the purpose of progress monitoring in early intervention, provide examples of both formal and informal progress-monitoring tools that TODs can use in practice, and present a set of guidelines for collecting progress-monitoring data. We also address the importance of intervention fidelity, its role in program monitoring, and provide an example of a fidelity checklist. We conclude with an explanation of why a combination of progress monitoring and intervention fidelity checks is necessary for TODs to achieve the goal of effective programming in early intervention services for young children who are DHH.

Teachers of the Deaf in Early Intervention

TODs who provide early intervention services to infants and toddlers who are DHH have access to a variety of resources and professional development opportunities that can aid them in understanding recommended practices for their work with this specific population of children. Membership in professional organizations such as the CEC’s DEC and Division for Communicative Disabilities and Deafness (DCDD), the Alexander Graham (AG) Bell Association, and the Council of American Instructors of the Deaf (CAID) offer access to conferences and publications to advance TODs’ knowledge and skills. In addition, TODs can access local and regional professional development workshops and trainings related to curricula (e.g., SKI-HI®, Learn to Talk Around The Clock®, The Creative Curriculum®, HighScope®, the Hanen Centre’s It Takes Two To Talk®) and visual communication systems (e.g., ASL, Cued Speech, Manually Coded English) to support learning in young children who are DHH. These resources should prompt TODs who provide early intervention services to reflect on recommended practices for young children with disabilities (DEC, 2014) and the use of evidence-based practices for targeted IFSP outcomes (goals) for infants and toddlers who are DHH.

In this age of accountability and call for evidence-based practices (Rous & Hyson, 2007), it is the responsibility of TODs to design and evaluate the effectiveness of interventions that promote young children’s development of needed cognitive, communicative, linguistic, and social behaviors when hearing loss is evident. These interventions will most likely include strategies that require adults to interact with these children in special ways to prompt, model, and reinforce targeted communication behaviors. However, itinerant and consulting TODs who travel from site to site (home or center-based) have limited amounts of time to spend with children, family members, and childcare providers and may find it challenging to collect data necessary for evaluating the effectiveness of an intervention plan. Therefore, within an inter-disciplinary team of speech-language pathologists, parents, and other IFSP team members, TODs need to design focused intervention programs that can be easily understood and implemented by family and other team members during spontaneous teaching and learning opportunities that occur between TODs’ visits and which can be monitored efficiently for their effectiveness.

Program Monitoring

When planning targeted outcomes for infants and toddlers who are DHH, TODs need to take into account a number of important factors related to both children and families. First and foremost, TODs need to be able to assess and identify the strengths and needs of the children they serve and select outcomes that build on those strengths while appropriately targeting areas of concern. In addition, TODs are encouraged to consider families’ strengths and needs, cultural values and practices, and children’s engagement and independence in daily routines with care providers (Jennings, Hanline, & Woods, 2012; McWilliam, 1996, 2010). This attention to contextual factors for learning will complement TODs’ knowledge of hearing loss and intervention strategies specifically designed to target the social and linguistic needs of infants and toddlers who are DHH. Finally, TODs need to consider how the success of an intervention plan will be evaluated. The monitoring of children’s progress toward targeted outcomes is an important component to any intervention program as it provides TODs with the information they need to make timely and effective data-based decisions.

Strategies and interventions highlighted for young children, including children who are DHH, often include activity-based interventions, natural learning opportunities, naturalistic prompts and reinforcers, and coaching of parent–child interactions (Carotta, in press; Dunst, Bruder, Trivette, Raab, & McLean, 2001). Therefore, TODs need to select progress-monitoring procedures that fit these natural settings and interactions for the functional goals targeted for the children they serve. To make appropriate decisions regarding children’s progress or possible changes to their services and interventions, TODs, in collaboration with other members of the IFSP team, need to collect, analyze, and interpret relevant data that accurately reflect children’s targeted IFSP outcomes and the fidelity of designed interventions. Given the possible itinerant nature of their services, TODs might partner with teachers and/or families to design and collect data for monitoring the fidelity and effectiveness of their efforts at home or in childcare settings. In the next two sections, we will describe how both child progress monitoring and intervention fidelity checks can be used to achieve this goal of effective program monitoring.

Child Progress Monitoring

Quality assessment practices and use of data-based decision making have a long history within the fields of early childhood special education and early intervention (McConnell, 2000). With the current national emphasis on accountability, the attention toward these practices has intensified (Snyder, Wixson, Talapatra, & Roach, 2008). For young children with disabilities and developmental delays, assessment serves a variety of purposes, including screening, diagnosis, determining eligibility for specialized services, program planning and evaluation, and progress monitoring. However, program evaluation and progress monitoring are two practices often overlooked by practitioners (Sandall, Hemmeter, Smith, & McLean, 2005) and sometimes only occur annually or when children exit early intervention services or are assigned new IFSP goals (Rous & Hyson, 2007).

Child progress monitoring is defined as the frequent and ongoing assessment of children’s short-term progress toward identified outcomes (Walker et al., 2008). According to Part C of the Individuals With Disabilities Education Improvement Act (IDEA, 2004), progress monitoring should be conducted at least every 6 months, or earlier if requested by the parents, and in conjunction with all IFSPs. However, DEC recommended that progress monitoring be conducted every 90 days, or earlier if requested by the family (Sandall et al., 2005). In addition, the Consensus on Infants and Young Children With Hearing Loss (Marge & Marge, 2005) recommended that progress monitoring of child/family outcomes occur at regular intervals and include a combination of formal and informal measures.

Although these requirements and recommendations are important to ensure progress monitoring is conducted consistently, the frequency of progress monitoring should be influenced by the specific outcomes being targeted and the need for making decisions, rather than minimum timeline recommendations. Some situations may prompt the need for progress-monitoring data to be collected monthly, bi-monthly, or even weekly. For example, suppose a child age 15 months has a targeted outcome to use actions, gestures, and/or vocalizations to initiate interactions with caregivers and siblings. The TOD should not wait 3 or 6 months before collecting any data to assess whether the child is making progress toward this outcome. Instead, weekly or bi-weekly data should be collected to assess whether progress is being made in the use of vocalizations and/or gestures, and whether this progress is evidenced only with familiar adults or with siblings as well. Furthermore, data regarding the contexts for such progress may provide the TOD and family with information about how to adjust the location and partners for teaching and learning new skills, aiming for the most generalizable skills possible. Therefore, it is the responsibility of TODs to take into consideration important factors associated with proposed interventions (e.g., proposed target, current skill level, desired timeline, contexts) to determine the type and amount of information (i.e., data) needed to adequately and reliably assess child progress and to determine whether adjustments are needed in the frequency and quality of strategies being used.

Formal Progress Monitoring

Until recently, very few formal progress-monitoring tools existed that were appropriate for use with infants and toddlers with disabilities (Buzhardt et al., 2010). Today, formal tools such as the Assessment, Evaluation, and Programming System for Infants and Children (AEPS; Bricker, 2002) and general outcome measurements such as the Infant and Toddler Individual Growth and Development Indicators (IGDIs; Buzhardt et al., 2010; Carta, Greenwood, Walker, & Buzhardt, 2010) are specifically designed for use with infants and toddlers with disabilities, and are used by some states for annual reporting of early childhood outcomes for children exiting Part C services (Early Childhood Technical Assistance Center [ECTA] Center, 2014).

The AEPS organizes ongoing observations of children’s behaviors and skills across six areas of development (i.e., fine motor, gross motor, adaptive, cognitive, social-communication, and social) over time with a 3-point rating system to reflect criteria relevant to absent (0 points), emerging/inconsistent (1 point), and established functional behaviors (2 points). Items on the AEPS can be adapted so that signs, gestures, and/or pictures can be used in place of spoken language to indicate the emergence or establishment of skills. The AEPS also encourages quarterly graphing of the percentage of behaviors demonstrated per domain that are expected of children by age 3. These graphs provide teams with a visual profile of children’s growth over time and direct attention to domains needing continued intervention while prompting review of specific developmental behaviors that should be targeted next.

The Infant and Toddler IGDIs are progress-monitoring measures that provide quick and efficient documentation of children’s skill development in early communication, social development, movement, problem solving skills, and parent–child interactions (Buzhardt et al., 2010; Greenwood, Carta, & McConnell, 2011). As with the AEPS, the Infant and Toddlers IGDIs allow for scoring of language and communication other than spoken English, including gestures, sign language, and simultaneous communication (speech and sign language together). However, such coding is only to be done by individuals who are fluent in the language and/or communication mode being scored (Carta et al., 2010). During brief 6-min semi-structured play situations, the frequency of the targeted skills or behaviors are recorded (either through direct observation or video-recording), scored, and entered into the IGDI online data system. The totals of these target skills are then converted to a rate per minute score, aggregated, and graphed. The Early Communication Indicator and Early Social Indicator IGDIs, as well as the social and communication components of AEPS, are examples of formal progress-monitoring tools that TODs can use for collecting ongoing information about the social-communication development of infants and toddlers who are DHH.

Informal Progress Monitoring

Informal progress-monitoring procedures allow practitioners to collect reliable data that are specifically designed to match targeted behaviors of interest. Such data collection procedures are ideal for TODs who have specific questions they seek to answer about the development and behaviors of the children they serve. Informal progress-monitoring procedures permit TODs to answer broad questions such as Is progress occurring? (occurrence/non-occurrence), as well as more specific questions such as With whom is he communicating? (context) and How long does she engage in interactions with parent? (duration). In the next section, we describe a process for establishing a progress-monitoring system that assures collection of data that are useful to TODs in determining child progress.

Progress-Monitoring Procedures

The collection of progress-monitoring data need not be difficult, complex, or time-consuming. In fact, the collection of data should be easy and efficient, fitting seamlessly into home or center-based activities. The following six-step process can help TODs design an informal progress-monitoring system that results in information that can be used for making decisions about continuing, revising, or discontinuing current efforts relevant to children’s targeted social, linguistic, and communication outcomes.

Step 1: Select a Functional Target Behavior

First and foremost, the progress-monitoring process must begin with the selection of target behaviors which are the focus of the intervention (Hojnoski, Gischlar, & Missall, 2009). For TODs working in early intervention programs, target behaviors will most likely fall within the communication and social domains of development, but this process can also be used to address other domains (e.g., cognition). Target behaviors must be observable, measurable, and meaningful to families, care providers, and TODs; they should clearly serve an important function in the daily lives of the children and/or families for whom they are designed. Targeted behaviors that are vaguely worded, such as “will improve communication” or “use words to communicate,” may be more difficult to monitor as the behaviors of interest are not clearly specified. Therefore, it is also important to use precise and explicit wording that reflects functional needs. Below are three examples of functional targets appropriate for children who are DHH. (Examples of abbreviated target behaviors are listed in Table 1, column 1.)

Sophie will get the attention of family members during meals, daily routines, and shared play times to protest and/or to request repetition of desired actions/objects by using relevant and recognizable actions, gestures, and/or signs.

Dan will request “more,” “help,” and/or an object/food by using word/sign approximations during his interactions and daily routines with siblings and parents.

Arron will take turns imitating the actions and vocal behaviors of others when engaged in songs and games with parents and siblings.

Table 1.

Examples of Functional Targets, Questions, Procedures, and Data Collection for Progress Monitoring.

Child target	IF you want to measure . . . (type of behavior/question)	THEN collect data using a . . . (data collection procedure)	Examples of data collected
C initiates interactions with parents and sibling at least 3 times a day	Occurrence/non-occurrence Did the behavior occur?	Yes/no or +/− documentation Behavior tallies/checklist	9/11: No, No, No 9/15: No, Yes, No 9/19: Yes, Yes, Yes 9/13: √√ (mom, brother) 9/18: √√√√ (dad, mom, mom, brother) 9/24: √√√ (mom, dad, mom)
D uses words/signs to communicate his wants/needs	Frequency How often does the behavior occur?	Frequency count (use when you want to count each and every occurrence of behavior) Time interval recording (use when you want to count the occurrence of behavior within a selected time frame/interval)	10/8: √√ (more, cracker) 10/15: √√√ (more, water, teddy) 10/22: √√√√√ (nana, more, milk, book, car) 11/5 (8:15 a.m.–8:20 a.m.): √√√√ 11/11 (11:20 a.m.–11:30 a.m.): √√ 11/17 (3:30 a.m.–3:40 p.m.): √√√
G speaks clearly so that others are able to understand her	Quality How well does the child (or adult) perform the behavior?	Behavior rating scale that describes the quality of the behavior (e.g., 1 = poor, 2 = ok, 3 = good, 4 = great) Anecdotal notes that are coded and quantified	10/18: 2 10/23: 2 10/29: 3; brother elaborated on topic to her “car blue” in car
K engages in prolonged (extended) interactions and turn-taking with parents and siblings	Duration How long does the behavior last?	Start/stop documentation (using timer/stop watch)	11/2: 10 s, 5 s, 9 s (patty-cake) 11/6 : 16 s (peek-a-boo) 11/10: 30 s (patty-cake)
M will follow adults one-step directions within 3 s.	Latency How long does it take for the child (adult) to exhibit the behavior when given a prompt or presented a stimulus?	Start/stop documentation (using timer/stop watch)	11/12: 12 s (direction needed to be repeated) 11/16: 8 s 11/22: 5 s
H will imitate actions/sounds of parents and siblings during songs and social interactions/games.	Description What did the behavior look like?	Anecdotal notes reflecting on observations Running record of in-the-moment observations	10/1: clapped hands 2 times during patty-cake 10/6: made moo sound for song 10/11: waved bye to grandma 10/17, 8:10 a.m.—waved “hi” when mom prompted 10/23, 9:20 a.m.—repeated “EI” during Old McDonald song 10/27, 2:15 p.m.—imitated (covered eyes) during peek-a-boo with sister
S uses words or signs to indicate when wants more	Contexts When, where, and with whom was the behavior observed?	Anecdotal notes reflecting on observations	11/2: with Charley at dinner 11/10: with mom at breakfast 11/16: in the car with mom; during bath time with dad

Note. √ = occurrence/frequency.

Step 2: Determine the Questions to be Answered and the Type of Behaviors to be Monitored

As there are several ways TODs could view “progress” of targeted behaviors, it is important for them to consider what questions they have about the behaviors. Column 2 of Table 1 lists seven possible questions TODs could consider when deciding what type of data will provide them with essential and important information for monitoring child progress. These questions address the occurrence, frequency, quality, duration, latency, and description of targeted behaviors. Context variables can also be of interest to TODs when describing whether targeted behaviors have generalized across partners and settings.

Step 3: Determine the Method of Data Collection

One commonly used method of data collection is natural observation of children at play or while engaged in daily routines with siblings, peers, or attentive adults; these observations can be live or video/audio-recorded. Targeted behaviors can also be measured through direct testing, a process in which children are asked to demonstrate targeted behaviors and are assigned a score or percentage of success based on a number of trials within a set period of time (Hojnoski et al., 2009). Existing documents, such as recent family photos or video clips, and permanent products, such as word checklists that reflect children’s demonstration of targeted words, are also valuable methods for engaging parents and caregivers in the progress-monitoring process. Furthermore, parent- or caregiver-report is a reliable method of data collection (Libertus & Landa, 2013) and can provide useful information about the generalization of targeted behaviors while children are at home engaging in everyday routines and activities with family members.

Step 4: Determine the Procedure for Data Collection

There are various ways progress-monitoring data can be collected and documented. Some formal measures for collecting progress-monitoring data (e.g., the AEPS and IGDIs) were described earlier in this article. Next, we are going to describe some informal data collection procedures TODs can use in daily practice.

When choosing an informal data collection procedure, it is important for TODs to select procedures that are easy to implement and ones that can accurately capture the types of behaviors to be monitored and the questions to be answered. For example, suppose a TOD wanted to get a sense of how often a child’s targeted behavior is occurring (e.g., using words and/or signs to communicate his or her thoughts, wants, and/or needs during daily routines at home), the TOD could count the occurrence of the child’s targeted behavior within a selected time frame or interval (e.g., 5 min) using a time interval recording procedure. This data collection procedure will provide the TOD a “snapshot” or approximation of the frequency at which the behavior occurs (in rate per minute) and help him or her determine whether the intervention strategies are working. Another example is documenting the start/stop time of a behavior for which the latency of the behavior is sought (e.g., how long does it take for the child to follow a one-step direction?). Each time the parent or TOD gives a directive, the adult simply counts or uses a timer to determine how long it takes for the child to begin and complete the directive. Column 3 of Table 1 provides a listing of various data collection procedures corresponding to specific questions and target behaviors. Examples for each procedure are provided as they might apply to child progress monitoring for children who are DHH.

Step 5: Determine the “When, Where, and Who” of Data Collection

To gather the most useful and informative data, TODs need to think about the three Ws: when the data should be collected (i.e., time of day and frequency), where the data collection should happen (i.e., the setting and routines), and who should collect the data (Hojnoski et al., 2009). As targeted behaviors for children who are DHH often focus on the development of social-linguistic communication, TODs will want to consider the times of day and settings that prompt interactions with new or established communication partners. For instance, if a child’s goal is to use words when requesting items, the TOD should reflect with the child’s parent or childcare provider on the times of day and settings when this behavior can occur most naturally. Although children may request things throughout the day, meal time and structured or spontaneous play times may be particularly good times to teach and practice this skill as they can provide repeated opportunities for children to request foods and beverages, utensils with which to eat, specific toys of interest, actions desired with objects, or repetition of enjoyable movements or hand actions.

Finally, the frequency of data collection deserves discussion among team members. Specifically, the team needs to ask When will the data be needed to make a decision? In 1 week? . . . 1 month? . . . 6 months? TODs need to ensure that the team will have ample data to make decisions and answer specific questions. More frequent data collection will be necessary when the decisions are to be made in a short period of time, as at least three to five data points are needed to provide stable information about trends. For example, if the team wants to know which contexts or situations elicit the most communication from a child for reporting on generalization across daily routines at an upcoming IFSP meeting in 2 weeks, they may choose to collect this data bi-weekly for the next 2 weeks. However, if the targeted outcome for a child is to establish a vocabulary of at least 50 signs by the time this child transitions to preschool, an inventory of parent-reported signs may be collected bi-monthly. It is important to note that the collection of unnecessary or excessive data can exhaust team members, distract them from intervention efforts, and tax their patience when the utility of the data is not evident in a timely manner. Therefore, agreement regarding the frequency of data collection and intended timeline for its review are important to establish among team members prior to the actual collection of any data.

When deciding who should be responsible for collecting data, TODs need to consider the following three factors: (a) the opportunity for individuals to observe targeted behaviors (Hojnoski et al., 2009), (b) the competence and confidence of individuals collecting the data, and (c) the needed frequency of data collection. To collect reliable and useful data, the persons collecting the data need to have ample opportunity to observe the targeted behaviors “in action.” In addition, the data need to be collected consistently and accurately. Sometimes, the TOD may be the best person to do this. Other times, it may be a parent, care provider, or another member of the IFSP team; and in some instances, it may be a combination of individuals, especially when multiple data are needed in a short period of time. It is important that the IFSP team, including the parents, have a thorough discussion about who would be the best candidate(s) for collecting the data and when data collection needs to occur. The individuals chosen should be comfortable with the task and have a firm understanding of what they are to observe and document.

Step 6: Use Data to Make Decisions About Next Steps

Once collected, TODs will need to organize the data so they can be analyzed and interpreted. This may involve something as simple as counting up tallies and determining the frequency of targeted behaviors observed for that past week. In other cases, it may require entering the data into an Excel® spreadsheet and graphing it to visually inspect data trends over days or weeks (Barton & Reichow, 2012). This decision will depend on the type and amount of data collected and the questions to be answered. Not all situations require graphing data to discern the amount of progress (or lack thereof) being made or whether the intervention is effective. However, TODs need to be sure that the data collected are organized in a manner that is interpretable (i.e., people can look at it and accurately determine whether progress is/is not being made) and meaningful (i.e., people can use the information to make informed decisions about programming). Graphed data can be particularly useful in providing objective information about targeted behaviors. Line graphs are helpful for noting rate of progress for continuous variables such as the frequency or duration of a behavior. Bar graphs are useful for showing similarities, differences, or changes in categorical data, such as a child’s mean number of communication attempts while at home versus a child care center over a 3-week period. Graphed data can also be used as a tool for communicating with parents, administrators, and other members of the IFSP team about children’s progress, the effectiveness of services provided, and the steps that should be taken next (Hojnoski et al., 2009).

These six steps outline a process for establishing an informal progress-monitoring system that can be tailored to the specific questions TODs have about a child. Although ongoing progress-monitoring data are necessary for providing TODs with timely and useful information about the effectiveness of their intervention efforts, they do not tell the whole story, particularly in situations where little progress is being made. In the next section, we will describe intervention fidelity checks as another important component of effective program monitoring and outline the process for establishing the collection of reliable intervention fidelity data.

Intervention Fidelity

Intervention plans typically include two key components: desired outcomes and strategies to reach those outcomes. Although both are necessary, the first can sometimes receive the majority of attention when teams meet to discuss and develop IFSPs (McWilliam, 2010). The desired outcomes for children and/or families are key to intervention planning in that they direct a team’s attention to goals for which to aim. However, the strategies selected to promote progress toward those desired outcomes are equally important to intervention plans and should also be clearly addressed on all IFSPs. Typically, the intervention strategies listed on IFSPs may describe frequency of contacts between professionals and family members, location of services, methods of service delivery, individuals responsible for implementing the plan, and children’s daily opportunities for learning the targeted outcomes. However, intervention plans should also be discussed and include brief descriptions of the proposed adult behaviors to be used (e.g., prompts, reinforcers), environmental accommodations to be made to help children or parents practice and acquire the new skills, and methods for assessing the fidelity of these interventions. These more specific details may not always be written into an IFSP document, but TODs should be prepared to discuss and/or explain these strategies with/to family and team members.

Intervention fidelity, or the degree to which an intervention or instructional strategy is implemented as intended, relies heavily on the actions of the IFSP team members who are responsible for implementing specific intervention strategies accurately and consistently (Gomez, Walis, & Baird, 2007; Gresham, MacMillan, Beebe-Frankenberger, & Bocian, 2000; McKenna, Flower, & Ciullo, 2014). It is important for team members to assess the fidelity of their interventions to determine whether limited child progress is due to poor implementation of the intervention (i.e., failing to carry out the intervention as it is intended) or poor choice of intervention (i.e., failing to select an intervention that appropriately addresses the child’s needs; McKenna et al., 2014). Without assessing fidelity, team members really have no way of knowing whether their interventions are truly effective, particularly in situations where children are making slow or minimal progress toward targeted outcomes.

The success of any intervention, therefore, often depends on its implementation fidelity. Interventions that are complex, involve multiple steps, or require the assistance of additional staff members or materials are often harder to implement with fidelity than interventions that are simple and involve only a few steps (Gomez et al., 2007; Gresham et al., 2000). Fidelity may also be influenced by team member factors, such as knowledge of, experience with, and perceived acceptability of the intervention (Gomez et al., 2007). Luze and Peterson (2004) argued that interventions are more likely to be implemented with fidelity if the team member(s) responsible for implementing the intervention view it is as acceptable. Therefore, it is very important for TODs to engage in a discussion with team members, especially parents, as they are often the ones responsible for implementing the intervention during natural learning opportunities, regarding the acceptability and feasibility of potential interventions.

Unless TODs are using interventions that include fidelity measures, most interventions will require that TODs create a fidelity checklist that clearly defines the components or steps of the intervention. The process for designing a fidelity checklist is much like the process for designing an informal progress-monitoring system, discussed earlier in this article. However, the focus is now on monitoring the quality of the adult’s behavior for the intervention. Once a desired outcome for a child is identified and a strategy plan is determined, the team then needs to write-out the specific components or steps of the intervention—this is the “heart and soul” of the fidelity checklist, as it outlines for team members what it is they should be doing or saying (as an antecedent or consequence to a targeted behavior) when implementing the intervention. Although all essential components should be listed, moderate levels of specificity can be used when defining the components of the intervention strategy, as to not overwhelm the individuals responsible for implementing the intervention (Gomez et al., 2007; Gresham et al., 2000). An example of an intervention fidelity checklist is provided in Figure 1.

After a fidelity checklist has been developed, the team then needs to determine a method for measuring the fidelity of adults’ efforts. Intervention fidelity can be monitored effectively using a variety of methods, including direct observation, self-monitoring, and/or analysis of permanent products (McKenna et al., 2014). Video-recorded interactions of TODs with children and live observations of parent–child interactions can be used to note presence or absence of key intervention components as outlined on the fidelity checklist. As with progress-monitoring data, fidelity data provide practitioners with important information for data-based decision making. Specifically, fidelity data can help to identify where the source of adjustments are needed in an intervention and affirm what parents and other team members are doing well. Repeated checks of fidelity also help to establish the consistency/inconsistency of intervention efforts. Although there are no recommended timelines for fidelity checks, periodic checks (e.g., weekly or bi-weekly) are important to establish the consistency of intervention efforts, especially in situations in which child progress is slow or negligible.

Putting It All Together

When considered separately, child progress monitoring and intervention fidelity checks are not sufficient to ensure that the interventions TODs implement will result in improved child outcomes. A more comprehensive approach to monitoring intervention programs merits both information about children’s progress in response to interventions (progress monitoring) and the implementation of the intervention strategies and program features (fidelity). This point is illustrated well in the following question: “If there is no child progress, can we say the intervention is not working, if in fact, it is not being implemented as intended?” When both progress monitoring and intervention fidelity data are collected, analyzed, and interpreted, TODs now have access to valuable information (i.e., evidence) about the effectiveness of their interventions that can then be used to make informed decisions about future programming. Through this process of systematic progress monitoring and fidelity checks, TODs can establish a reliable and effective framework for improving the social, linguistic, and communicative development of children who are DHH and their families.

Conclusion

Program monitoring is a necessary and important process for determining the effectiveness of early intervention services. The monitoring of intervention programs designed for infants and toddlers who are DHH requires collection of data on both child progress toward targeted outcomes and the fidelity with which the interventions are delivered. To carry out effective program monitoring, TODs need to consider possible questions they can ask about children’s linguistic, social, and communicative behaviors, as well as the various procedures available for collecting meaningful data about children’s progress toward these targeted outcomes. In addition, TODs may need to monitor the fidelity with which team members implement interventions they have designed. By engaging in a process of systematic child progress monitoring and fidelity checks, TODs can achieve effective program outcomes for children who are DHH.

The absence of publications describing TODs’ current approaches to program monitoring with young children or the barriers to relevant data collection with young children who are DHH prompts a need for more research in this area. Descriptive and qualitative studies could provide the documentation of current data collection practices or lack thereof, but also could enhance our understanding of what approaches to program monitoring are uniquely relevant, successful, or problematic for TODs and the population of infants and toddler who are DHH.

Teacher efficacy regarding program monitoring and the subsequent decision making for ongoing intervention planning also needs to be examined to assess the need for professional development efforts in these areas at a preservice and/or in-service level. Currently, CEC Specialty Standards for Deaf and Hard of Hearing do address program monitoring with three skill sets, one in Standard 4: Assessment and two in Standard 5: Instructional Planning. However, more work may be needed to aid professional development coordinators and supervisors of employed TODs as they design training or mentoring in program monitoring. By examining how various preservice programs provide future TODs with training in collecting, interpreting, and using data for planning instruction and adjusting interventions, we may learn valuable information to guide professional development for TODs on this important aspect of program monitoring.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

American Speech-Language-Hearing Association. (2008). Service provision to children who are deaf and hard of hearing, birth to 36 months [Technical report]. Joint Committee of the American Speech-Language-Hearing Association and Council on Education of the Deaf. Available from www.asha.org/policy

Banerjee

Luckner

J. L.

(2013). Assessment practices and training needs of early childhood professionals. Journal of Early Childhood Teacher Education, 34, 231–248.

Barton

Reichow

(2012). Guidelines for graphing data with Microsoft® Office 2007™, Office 2010™ and Office for Mac™ 2008 and 2001. Journal of Early Intervention, 34, 129–150.

Bhasin

T. K.

Brocksen

Avchen

R. N.

Van Naarden Braun

(2006). Prevalence of four developmental disabilities among children aged 8 years—Metropolitan Atlanta Developmental Disabilities Surveillance Program, 1996 and 2000. Morbidity and Mortality Weekly Report, 55, 1–9.

Bricker

(Ed.). (2002). Assessment, evaluation, and programming system for infants and children (2nd ed.). Baltimore, MD: Paul H. Brookes.

Buzhardt

Greenwood

Walker

Carta

Terry

Garrett

(2010). A web-based tool to support data-based early intervention decision making. Topics in Early Childhood Special Education, 29, 201–213.

Carotta

(in press). Supportive early childhood practices and learning environments. In Moeller

M. P.

Ertmer

D. J.

Stoel-Gammon

(Eds.), Promoting speech, language, and literacy in children who are deaf or hard of hearing. Baltimore, MD: Paul H. Brookes.

Carta

J. J.

Greenwood

C. R.

Walker

Buzhardt

(2010). Using IGDIs: Monitoring progress and improving intervention results for infants and young children. Baltimore, MD: Paul H. Brookes.

Division for Early Childhood. (2014). DEC recommended practices in early intervention/early childhood special education. Retrieved from http://www.dec-sped.org/recommendedpractices

10.

Dunst

C. J.

Bruder

M. B.

Trivette

C. M.

Raab

McLean

(2001). Natural learning opportunities for infants, toddlers, and preschoolers. Young Exceptional Children, 4(3), 18–25.

11.

Early Childhood Technical Assistance Center. (2014). Outcome measurement: Instrument crosswalk. Retrieved from http://ectacenter.org/eco/pages/crosswalks.asp

12.

Gomez

C. R.

Walis

Baird

(2007). On the same page: Seeking fidelity of intervention. Young Exceptional Children, 10(4), 20–29.

13.

Greenwood

C. R.

Carta

J. J.

McConnell

(2011). Advances in measurement for universal screening and individual progress monitoring of young children. Journal of Early Intervention, 33, 254–267.

14.

Gresham

F. M.

MacMillan

D. L.

Beebe-Frankenberger

M. E.

Bocian

K. M.

(2000). Treatment integrity in learning disabilities intervention research: Do we really know how treatments are implemented? Learning Disabilities Research & Practice, 15, 198–205.

15.

Hojnoski

R. L.

Gischlar

K. L.

Missall

K. N.

(2009). Improving child outcomes with data-based decision making: Collecting data. Young Exceptional Children, 12(3), 32–44.

16.

Individuals With Disabilities Education Improvement Act (IDEA) of 2004, 20 U.S.C. § 1400 (2004).

17.

Jennings

Hanline

M. F.

Woods

(2012). Using routines-based interventions in early childhood special education. Dimensions of Early Childhood, 40(2), 13–23.

18.

Libertus

Landa

R. J.

(2013). The Early Motor Questionnaire (EMQ): A parental report measure of early motor development. Infant Behavior & Development, 36, 833–842.

19.

Luze

G. J.

Peterson

C. A.

(2004). Improving outcomes for young children by assessing intervention integrity and monitoring progress: “Am I doing it right and is it working?” Young Exceptional Children, 7(2), 20–29.

20.

Marge

D. K.

Marge

(2005). Beyond newborn hearing screening: Meeting the educational and health care needs of infants and young children with hearing loss in America—Report of the National Consensus Conference on Effective Educational and Health Care Interventions for Infants and Young Children with Hearing Loss. Syracuse: State University of New York, Upstate Medical University.

21.

McConnell

S. R.

(2000). Assessment in early intervention and early childhood special education: Building on the past to project into our future. Topics in Early Childhood Special Education, 20, 43–48.

22.

McKenna

J. W.

Flower

Ciullo

(2014). Measuring fidelity to improve intervention effectiveness. Intervention in School and Clinic, 50, 15–21.

23.

McWilliam

R. A.

(1996). Family-centered intervention planning: A routines-based approach. San Antonio, TX: Communication/Therapy Skill Builders.

24.

McWilliam

R. A.

(2010). Routine-based early intervention: Supporting young children and their families. Baltimore, MD: Paul H. Brookes.

25.

Rous

Hyson

(2007). Promoting positive outcomes for children with disabilities: Recommendations for curriculum, assessment, and program evaluation. Missoula, MT: Division for Early Childhood, Council for Exceptional Children.

26.

Sandall

S. R.

Hemmeter

M. L.

Smith

B. J.

McLean

M. E.

(2005). DEC recommended practices: A comprehensive guide for practical application in early intervention/early childhood special education. Longmont, CO: Sopris West.

27.

Sandall

S. R.

Schwartz

I. S.

Lacroix

(2004). Interventionists’ perspectives about data collection in integrated early childhood classrooms. Journal of Early Intervention, 26, 161–174.

28.

Snyder

P. A.

Wixson

C. S.

Talapatra

Roach

A. T.

(2008). Assessment in early childhood: Instruction-focused strategies to support response-to-intervention frameworks. Assessment for Effective Intervention, 34, 25–34.

29.

Walker

Carta

J. J.

Greenwood

C. R.

Buzhardt

J. F.

(2008). The use of individual growth and developmental indicators for progress monitoring and intervention decision making in early education. Exceptionality, 16, 33–47.