Abstract
A multiple-probe, single-case design was used to determine the effects of bug-in-ear eCoaching on teachers’ use of two targeted naturalistic communication strategies and focus children’s responses to these strategies. Results indicated that bug-in-ear eCoaching enhanced teachers’ use of communication strategies and the appropriate responses of children with communication difficulties. Moreover, novice teachers reported that bug-in-ear eCoaching was a socially valid intervention.
Keywords
Introduction
Naturalistic instruction is the process by which teachers use everyday activities and routines to embed learning opportunities for children to practice skills such as communication (Snyder et al., 2015). This type of instruction has been recognized as an effective intervention for enhancing children’s communication skills (Stanton-Chapman, Kaiser, Vijay, & Chapman, 2008). Young children with disabilities need many opportunities to practice a skill to attain it, and these opportunities must be carefully planned and skillfully implemented (Johnson, Rahn, & Bricker, 2015; Snyder et al., 2015). A primary benefit of naturalistic instruction is that it promotes multiple, high-quality opportunities for children to practice communication skills across settings, which increases the likelihood that a child will generalize the skill (Division for Early Childhood, 2014; Johnson et al., 2015). Unfortunately, research examining teachers’ use of naturalistic instruction in early childhood classrooms suggests teachers may not provide the number of high-quality opportunities required for young children with disabilities to attain new skills (Dinnebeil, Pretti-Frontczak, & McInerney, 2009). The initial years of early childhood special education (ECSE) teachers’ careers may be an optimal time to support them in providing naturalistic instruction to meet the needs of diverse learners within inclusive environments (Barton & Smith, 2015; Bruder, Dunst, Wilson, & Stayton, 2013).
One method to support novice ECSE teachers to develop into expert teachers who use evidence-based practices, such as naturalistic instruction, is through professional development (PD) induction supports. High-quality PD is characterized by inclusion of adult learning strategies such as alignment to target outcomes and interactive and collaborative delivery (Hemmeter, Snyder, Kinder, & Artman, 2011; Marturana & Woods, 2012). Unfortunately, ineffective approaches to PD (e.g., traditional didactic trainings or workshops) remain the most frequent form of PD in ECSE settings (Snyder, Hemmeter, & McLaughlin, 2011). In addition, most ECSE educators indicate they receive no feedback, follow-up, or evaluation of their teaching (Barton, Pribble, & Chen, 2013; Snyder et al., 2011).
Coaching to Support ECSE Teachers’ Use of Evidence-Based Practices
One of the more effective methods for providing PD is coaching (Coogle, Ottley, Storie, & Rahn Burt, 2017; Coogle, Rahn, Ottley, & Storie, 2016; Kretlow & Bartholomew, 2010; Marturana & Woods, 2012). According to the National Center on Quality Teaching and Learning (2014), coaching is a cyclical process designed to support teachers in their implementation of practices with fidelity. During coaching, goals are created that are contextually relevant to the classroom. Then, an expert or peer conducts focused observations and provides individualized feedback for ongoing reflection (National Center on Quality Teaching and Learning, 2014). Coaching can be diverse in its duration and delivery. For example, it can be delivered face to face or from a distance. Coaching models in early childhood commonly use performance-based feedback, which focuses feedback on the quality and quantity of teachers’ instructional practices (Hemmeter et al., 2011).
Performance-based feedback has been used in early childhood settings to increase teachers’ use of evidence-based practices (Barton, Chen, Pribble, Pomes, & Kim, 2013; Fettig, Barton, Carter, & Eisenhower, 2016; Marturana & Woods, 2012; Ottley & Hanline, 2014). The research suggests positive impacts from coaching that include an element of performance-based feedback on teachers’ use of behavioral strategies (Fettig et al., 2016), interaction strategies (Barton, Chen et al., 2013; Barton, Pribble, & Chen, 2013), social–emotional strategies (Fox, Hemmeter, Snyder, Binder, & Clarke, 2011), communication strategies (Friedman & Woods, 2015), and literacy strategies (McCollum, Hemmeter, & Hsieh, 2011). Importantly, in each of these studies, after early childhood teachers received coaching, their instructional practices on children’s targeted outcomes improved.
Bug-in-ear (BIE) coaching
One way to offer performance-based feedback is through BIE coaching. BIE coaching in the midst of classroom sessions may be more effective than providing teachers with coaching after the conclusion of the coach’s observation because of the immediacy with which feedback is provided (Scheeler, Congdon, & Stansbery, 2010). Immediate feedback provides teachers information on the strategies they are using in real time (Scheeler, Ruhl, & McAfee, 2004). This literature has included research documenting a positive impact of immediate feedback on early childhood teachers’ use of naturalistic strategies and small benefits extending to the expressive communication outcomes of children with delays or disabilities served in the educators’ Early Head Start classrooms (Ottley & Hanline, 2014). BIE coaching is characterized by teachers wearing a small earpiece to receive coaching in a discrete manner within classroom sessions as they practice their skills in implementing new/targeted evidence-based strategies with children. Although the use of BIE methods may be perceived to be an intrusive and resource-intensive form of coaching, research suggests that the immediacy of feedback received through BIE coaching affords the opportunity for teachers to acquire skills quickly and effectively (Coogle et al., 2017; Coogle, Rahn, & Ottley, 2015; Coogle et al., 2016).
Technological advances have offered an avenue for providing coaching from a distance in a manner that is natural and unobtrusive (Fettig et al., 2016; Scheeler et al., 2010). The reason for using these technological methods is related to resources (i.e., time, money), immediacy of feedback, and logistics such as travel associated with coaching (Hemmeter et al., 2011; Marturana & Woods, 2012). Although using technology may be associated with some potential challenges (e.g., access to Internet, cost of technology), it offers a potential solution to providing PD in a cost- and time-efficient manner. BIE eCoaching employs web-based technology (e.g., Skype) to conduct observations and provide feedback in real time. Research examining the effects of BIE eCoaching for special education teachers of school-age students indicates that this approach to coaching leads to an increase in teachers’ use of evidence-based practices (Scheeler et al., 2010; Scheeler, McAfee, Ruhl, & Lee, 2006).
The Current Study
Research is warranted to identify methods to individualize PD to support novice ECSE teachers’ transfer of knowledge to practice. This support is critical so that ECSE teachers grow as professionals, remain in the field, and become well prepared, expert teachers who promote the development of young children. The purpose of this study was to determine the impact of a brief training plus BIE eCoaching on novice teachers’ use of naturalistic communication strategies when interacting with a focus child exhibiting an identified delay or disability related to communication development. We focused on in sight out of reach and choice making, two evidence-based strategies that have documented efficacy in enhancing the communication skills of young children experiencing communication challenges (Hancock & Kaiser, 2006; Wetherby & Woods, 2006). We targeted communication outcomes because children experiencing delays in communication who do not receive intervention often face social and academic challenges across the life span (Kaiser & Roberts, 2011). Our research questions were as follows:
Method
Participants and Setting
ECSE teachers and focus children
Participants included four female, Caucasian ECSE teachers between 20 and 30 years of age who were teaching for less than 2 years (see Table 1). Inclusionary criteria included novice teachers who were teaching within inclusive, public preschool classrooms. Kiara, Paige, Jada, and Mia (pseudonyms) completed an undergraduate program in child development and family studies with licensure in ECSE. Their licensure within ECSE included six courses and an internship that emphasized the importance of naturalistic instruction. Each teacher began teaching within 3 to 15 months from the start of intervention. Each teacher taught in an inclusive classroom in one of four elementary schools in a small mid-Atlantic city. All teachers taught within the same district. None received PD specific to naturalistic communication strategies prior to the start of the study. Each classroom included up to 16 typically developing children and four children with disabilities between 3 and 5 years of age. Children with disabilities presented developmental delays and autism spectrum disorders.
Participant Characteristics.
Child inclusionary criteria included being a full-time preschool student in a participating teacher’s classroom, and presenting an expressive language delay, or the teacher having concerns related to the child’s expressive language. All teachers were asked to select a focus child within their classroom who had an individualized education plan (IEP) and developmental delay in expressive language (see Table 1). One teacher (Paige) did not have a student with a current IEP. Consequently, Paige selected a student with whom she had concerns regarding the way in which he communicated with others.
The focus child in Kiara’s classroom was Kyle, a 5-year-old Caucasian boy with autism spectrum disorder. The focus child in Paige’s classroom was Tyson, a 4-year-old Caucasian boy who was at risk of a social–emotional delay. The focus child in Jada’s classroom was Ruben, a 5-year-old, Caucasian boy with a communication delay. The focus child in Mia’s classroom was Marcus, a 4-year-old Caucasian boy who was diagnosed with autism spectrum disorder. All focus children had an IEP in place except Tyson, who was in the process of being evaluated for special education services. All teachers indicated that a goal for their focus child was to increase their use of expressive, functional communication with others. The abilities of the remaining children participating in each activity varied. Activities included typically developing children and children with disabilities across all teachers.
Setting
Intervention sessions took place from a distance using BIE eCoaching during the teachers’ regular teaching hours. Implementation of the intervention occurred during a variety of small-group activities. The activities occurred where they typically would (i.e., child-sized table in the classroom) had the research not been occurring. Some of the small-group activities were structured activities from the county curricula (i.e., Jada’s and Kiara’s small-group activities were teacher-led activities focusing on literacy or mathematic skills); however, some teachers chose to receive coaching during a child-led choice activity (i.e., Paige’s and Mia’s small-group activities were child-led activities in which the small groups of children played with blocks or other toys). Teachers consistently received coaching in the type of routine that they selected. The number of children participating in each intervention activity varied. On average, two to three children participated in each activity (SD = 1.34, range = 1-5). The focus child consistently participated in all sessions and was the interactional target during intervention sessions, unless an infrequent absence took place (Kyle: 3, Tyson: 0, Ruben: 3, and Marcus: 1). In this case, we asked the teacher to include another child with a disability in the activity, and we continued coaching the teacher due to our primary research question being impacts of BIE eCoaching on teachers’ practice.
Research Design
We used a single-case, multiple-probe design across participants to examine the effects of BIE eCoaching on novice ECSE teachers’ use of communication strategies (Gast, Lloyd, & Ledford, 2014; Horner & Baer, 1978). We selected a probe design because our previous research indicated teachers were unlikely to begin using these strategies without intervention (Coogle et al., 2017, 2015, 2016) and probe designs require only intermittent data collection during baseline. To meet What Works Clearinghouse (2014) design standards, sessions across participants and phases were concurrent. In addition, the independent variable was systematically manipulated, each outcome variable was measured by more than one observer, and interobserver agreement (IOA) was established. The design included at least three attempts to demonstrate an effect at three different points in time (i.e., three A and three staggered B phases for six total phases). Baseline and intervention included at least five data points. In addition, because this was a multiple-probe design, each time intervention was introduced to a new participant, we collected a probe data point for all participants (see Figure 1). To strengthen the design of our study and its internal validity, we randomized a priori the order in which participants received intervention by randomly selecting their names, which were folded on a piece of paper, off of a desk (Kratochwill & Levin, 2010); by doing so, the order in which the intervention commenced across tiers was not influenced by the data.
Novice teachers’ use of evidence-based communication strategies within 6-min observations.
Our goal was to provide eight eCoaching sessions for all teachers, but intervention continued until a teacher met the a priori criterion of using the communication strategies more frequently than baseline in at least six consecutive sessions. Once a teacher completed intervention, she received an intervention probe at least once every eight sessions until all teachers completed intervention. Beginning the week after the last teacher concluded her BIE eCoaching sessions, we collected maintenance and generalization probes for all teachers. The purpose of maintenance probes was to assess teachers’ use of the communication strategies without BIE eCoaching. The purpose of generalization probes was to examine teachers’ ability to generalize use of the strategies to different types of classroom routines.
Measures
Observational event-recording measure
We used an observational event-recording measure to record intervention prompts, teachers’ use of the communication strategies, and children’s responses for the first 6 min of each activity. We coded the number of prompts delivered, the prompt type (i.e., choice making: teacher provided two appropriate choices to the child by labeling them; and in sight out of reach: teacher withheld desired items where the child could see them but could not obtain them to create a communication opportunity) and whether the teacher responded to BIE eCoaching by using the communication strategy correctly (i.e., use in a manner that provided the child a communication opportunity) or incorrectly (i.e., use of the strategy in a way that did not provide a communication opportunity or no use of a strategy when prompted to use one). We also coded the frequency of teachers’ correct spontaneous use of the strategies, which was defined as teachers’ use of a strategy in its intended manner to provide a communication opportunity, without being prompted prior to using the strategy. We coded all teacher uses of the strategy, regardless of whether the teacher used the strategy with the focus child. We also coded (a) whether a strategy was directed at a focus child and (b) how the focus child responded.
eCoaching Effectiveness Questionnaire
To determine the feasibility and perceived effectiveness of the intervention, we created a questionnaire. We used a BIE eCoaching Effectiveness Questionnaire to examine teachers’ perceived effectiveness of the intervention. The questionnaire included nine items with a 5-point Likert-type scale and five open-ended questions. Likert-type scale items ranged from 0 representing strongly disagree to 4 representing strongly agree. These items included statements related to the perceived helpfulness of the intervention, teachers’ perceived skill and confidence in using the coached strategies, and teachers’ perceptions of child responses to the strategies. Open-ended questions asked about the perceived impact of the intervention on the participants as new teachers and their students, and benefits and challenges of BIE eCoaching.
Procedures
Before beginning the study, we obtained approval from the university institutional review board and the teachers’ school district, as well as informed consent from teachers and children in the teachers’ classrooms. Children whose parents did not provide consent were asked by teachers to participate in other classroom activities during data collection sessions.
Baseline
No training was provided prior to baseline. During baseline, all teachers were given the iPad with a stand and were asked to record at least five 6-min videos of themselves engaging in any small-group activity that included the focus child. Because all teachers were familiar with an iPad and how to record, there was not a need for training on how to capture videos. One video was recorded each session and following the multiple-probe guidelines indicated in What Works Clearinghouse (2014) standards. When there was a need for three consecutive sessions (i.e., right before intervention), the videos were recorded on consecutive days. Typically, four sessions were recorded each week as children in preschool in the state where this study took place attend school Monday through Thursday. The number of videos recorded varied from five to nine sessions depending on when intervention was introduced to the teacher. Once teachers captured a video, it was automatically shared with the first author through the university’s secure cloud system.
Intervention
After establishing a stable or decelerating baseline, teachers were asked (in the order of a priori determined randomization) to begin intervention. The intervention consisted of two primary components. The first was a 15-min eTraining (i.e., presentation with voiceover describing naturalistic communication strategies; Coogle, Floyd, Hanline, & Kellner-Hiczewski, 2013). It was distributed via email upon completion of the baseline condition and immediately prior to the start of the BIE eCoaching condition. The first author provided the voiceover within the presentation, which included information and examples describing the two target strategies. Teachers confirmed that they watched the narrated presentation by emailing the first author.
The second intervention component was BIE eCoaching on teachers’ use of communication strategies during small-group activities. The coach (i.e., the first and third authors) delivered at least six 6-min eCoaching sessions to each teacher. The session length of 6 min was chosen based on the amount of time the teachers felt all focus children could engage in an activity, and our previous research suggesting that feedback delivered for a short duration could be effective in improving practice (Coogle et al., 2017, 2015, 2016). During eCoaching, the following pieces of technology were used: iPad, Skype™, Bluetooth® device, laptop computer, and ECamm™ (for Mac) or Evaer™ (for PC) software. The iPad with Skype™ was used as the communication system to provide BIE eCoaching. Both the video and audio features of Skype™ were used by the coach so the coach could see and hear the teacher interacting with the children, but only the audio feature was used by the teacher who wore a Bluetooth® device that allowed her to hear prompts delivered by the coach and minimize child distractions by withholding the video. All BIE eCoaching sessions were delivered from the coach’s office. Intervention sessions were recorded using ECamm™ or Evaer™ software.
During each BIE eCoaching session, the teacher placed the iPad in the stand in a location where the activity could be recorded and asked children to join the small-group activity. Wearing a Bluetooth® device, the teacher then called the coach using Skype™ on the iPad. The coach answered the call and disabled her video so that the children could not see her during the session. During the first minute of the activity, the coach only observed; however, this first minute was still included for data analysis. This was done to allow a period of time for the teacher to begin the activity and for the coach to understand the nature of the activity.
The coach then delivered prompts to the teacher to use strategies at a rate of approximately one prompt per minute. The coach delivered prompts on the two targeted naturalistic communication strategies: choice making and in sight out of reach. Choice making was defined as offering the child a choice of two or more specifically stated options (e.g., “Ask Shawn if he wants to play with circle or square blocks.”). In sight out of reach was defined as intentionally placing items within view but out of the child’s reach (e.g., “Put the markers out of reach and wait for him to ask for one.”).
When a spontaneous strategy was used during a minute, a prompt was not provided. The researcher did observe and wait during Minutes 2 to 6 to determine whether the teacher might spontaneously use a strategy. If it appeared as though the teacher was not using naturalistic opportunities half way through the minute, the researcher provided a prompt to use choice making or in sight out of reach. The rate of BIE eCoaching decreased as teachers increased their spontaneous use of the strategies. After each prompt, the coach provided affirmative feedback when the teacher used a strategy correctly (e.g., “Good job giving a choice.”) or corrective feedback when the teacher did not use the strategy or used it incorrectly. Affirmative feedback was also given for spontaneous use of the strategies.
Our goal was to deliver eight total sessions as this would equal two full weeks of daily intervention to each participant. However, we set a minimum criterion of providing eCoaching during at least six sessions because of the staggered introduction of the intervention and to allow for flexibility as needed due to circumstances outside of our control (e.g., absences, school cancellations due to snow). Including intervention probes, Kiara and Paige received 10 sessions of BIE eCoaching, Jada received eight sessions, and Mia received six sessions. Sessions were coded live by the coach who then viewed the video immediately following the session to confirm the accuracy of the coding. If an error was found in the original coding (e.g., the coach missed a teacher’s use of a strategy), the final coding was modified to reflect the change.
Maintenance and generalization
Researchers determined a priori that after all teachers completed intervention, each teacher would use the iPad to video record themselves teaching during five 6-min small-group (maintenance) and five 6-min activities different from those in which they received BIE eCoaching (generalization). At least 1 week after intervention ended, teachers began collecting maintenance and generalization videos. Although waiting longer would have been preferable, it was necessary to begin maintenance and generalization 1 week after intervention due to the time constraints of the school-year ending. Maintenance and generalization sessions were collected concurrently, which is why they overlap on session days in Figures 1 and 2. During the generalization videos, teachers engaged in activities such as teacher-led story time. Kiara, Paige, and Mia were able to collect all these videos, but Jada was only able to collect two maintenance videos and four generalization videos. All sessions were coded using the same coding method used during intervention. After all videos were collected, the BIE eCoaching Effectiveness Questionnaire was sent to each teacher via email (see Table 2).
Novice teachers’ use of child-directed evidence-based communication strategies within 6-min observations and the responses to those strategies by focus children.
BIE eCoaching Effectiveness Questionnaire.
Note. BIE = bug-in-ear.
Data Analysis
We graphed teachers’ prompted and spontaneous communication strategies in each session with focus children and their peers. We also graphed the responses of the focus children to the naturalistic communication strategies. We analyzed data using visual inspection of graphs, which was guided by recommended practices for single-case design studies (Kratochwill et al., 2013). In addition, to analyze data from the BIE eCoaching Effectiveness Questionnaire, we used descriptive statistics and pattern coding. We calculated the mean and range of responses on the Likert-type scale items and identified units of meaning and chunked these small meaning units into themes for the open-ended questions (Miles & Huberman, 1994).
IOA
An undergraduate student naïve to the study design coded at least 20% of randomly selected sessions for each participant and each phase. IOA was coded for nine sessions for Mia, seven sessions for Paige, seven sessions for Jada, and five sessions for Kiara (range = 20%-36%). Reliability was coded for nine baseline sessions, 11 intervention sessions, four maintenance sessions, and four generalization sessions (range = 21%-32%). IOA was calculated for number of prompts delivered by the coach, teachers’ correct prompted and spontaneous use of the strategies, teachers’ use of strategies with the focus child, and appropriate child responses. IOA was calculated for each 10-s interval using the total agreement method (Kennedy, 2005). IOA was 99% (range = 97%-100%) for number of prompts delivered by the coach, 99% (range = 98%-100%) for teachers’ correct prompted use of strategies, 97% (range = 83%-100%) for teachers’ spontaneous use of strategies, 99% for teachers’ use of strategies with the focus child, and 100% for appropriate child responses.
Fidelity of Implementation (FOI)
FOI was coded for a random 20% of intervention sessions using an intervention fidelity checklist. Items on the checklist included prompts delivered (i.e., number, type), feedback on teachers’ correct and incorrect use of communication strategies, and whether or not coaches decreased coaching based on teachers’ spontaneous use of the strategies. For FOI, coaches were required to (a) provide no prompts during the first minute, (b) provide one prompt per minute for Minutes 2 to 6, (c) refrain from providing a prompt in Minutes 2 to 6 if teachers spontaneously used the strategies without prompting, (d) provide affirmative feedback each time teachers used a strategy correctly, and (e) provide corrective feedback when teachers used the strategies incorrectly. FOI was calculated by dividing the number of yes responses by total responses (withstanding nonapplicable responses; for example, if teachers did not incorrectly use a strategy then corrective feedback would not have been necessary and would have been coded as nonapplicable) and multiplying by 100. FOI was 99%; range = 88%-100%. Although teachers were spontaneously using the strategies throughout the sessions, during some minutes teachers did not use a strategy, so the coach provided prompts in those instances to ensure adherence to the implementation protocol. This is why prompts were still provided during sessions in which strategies were used multiple times.
Results
Overall, four demonstrations of an effect were observed at four different points in time, which indicates a functional relation was present. Data patterns were similar across teachers, with little use of the strategies by any teacher prior to intervention and immediate change prior to the start of the intervention. Whereas all teachers demonstrated an increase in level, Paige and Mia were the only teachers who demonstrated accelerating trends. Variability was observed across participants for maintenance and generalization.
Kiara and Kyle
Kiara had a zero-celerating slope with no instances of communication strategies used across the five baseline sessions (see Figure 1); consequently, there were no opportunities for appropriate child responses (see Figure 2). At the start of intervention, there was an immediate effect on Kiara’s prompted and spontaneous use of communication strategies. Specifically, Kiara produced a correct response for 79.26% of prompts and increased her level of spontaneous use of strategies. However, she demonstrated variability in the number of strategies used across sessions. There were no overlapping data points between baseline and intervention. During intervention, the focus child in Kiara’s classroom responded appropriately to her use of the communication strategies an average of 59% of the time (range = 0%-100%).
Kiara had no overlapping data between baseline and maintenance conditions and one overlapping data point between baseline and generalization. Kiara’s focus child transitioned to a new school prior to the maintenance phase, and, therefore, we were unable to collect appropriate response data for maintenance and generalization phases. Visual analysis suggests that, although she used the strategies more than she did during baseline sessions, she did not maintain or generalize strategies at her intervention level.
Paige and Tyson
Paige had a zero-celerating slope with no instances of communication strategies used across seven baseline sessions, resulting in no opportunities to measure appropriate child responses during baseline. Once intervention began, there was a small but immediate effect on Paige’s prompted and spontaneous use of communication strategies. Paige responded correctly to 85.71% of prompts. The data also revealed an increase in Paige’s level of strategies used and an accelerating trend. There were no overlapping data points between baseline and all other phases (intervention, maintenance, and generalization). Visual analysis suggests Paige maintained and generalized strategies. Overall, Paige’s use of communication strategies resulted in the focus child in her classroom responding appropriately an average of 87% of the time.
Jada and Reuben
Jada had a zero-celerating slope with one instance of communication strategies used across eight baseline sessions. Once intervention began, there was an immediate effect on Jada’s prompted and spontaneous use of communication strategies. Jada responded correctly to 91.25% of prompts. Jada’s data reveal an increase in level but variable use of strategies. There were no overlapping data points between baseline and intervention and baseline and maintenance, but there were two overlapping data points between baseline and generalization. Visual analysis suggests Jada maintained use of strategies; however, she collected only two maintenance probes. During two generalization sessions she used an increased number of strategies above baseline; however, she did not generalize at her intervention level. Throughout all sessions, Jada’s focus child appropriately responded to her use of communication strategies an average of 88% of the time.
Mia and Marcus
During baseline, Mia had a zero-celerating slope with four instances of communication strategies used across eight baseline sessions. Once intervention began, there was an immediate effect on Mia’s prompted and spontaneous use of communication strategies. Mia responded correctly to 90% of prompts. Visual analysis of Mia’s data demonstrated an increase in level and accelerating trend. Although she had variable data, there were no overlapping data points between baseline and intervention or between baseline and maintenance conditions. Visual analysis suggests Mia maintained use of strategies; however, she generalized strategies above baseline during only one session. Overall, Mia’s focus child responded appropriately to her use of communication strategies an average of 89% of the time.
Social Validity
Three of the four teachers completed the eCoaching Effectiveness Questionnaire to share their perceptions of the intervention. For reasons unknown, Jada did not complete the questionnaire. Themes that emerged included benefits of BIE eCoaching and future considerations. Teachers indicated that this experience was helpful in changing their communication practices (M = 3.67) and provided them with feasible strategies they would continue to use (M = 3.33). All of them indicated that BIE eCoaching enhanced their PD experiences and that they would recommend BIE eCoaching to other teachers (M = 4.00). They perceived that the strategies were effective in enhancing the frequency (M = 3.33) and effectiveness (M = 3.67) of children’s communication. The second theme of future considerations contained data related to specific challenges each teacher experienced with technology, scheduling, obtaining consent, and receiving BIE eCoaching individualized to child goals. For example, when asked whether the use of technology to receive feedback was manageable, teachers were either neutral or agreed (M = 2.67).
Discussion
As there is a push for highly effective teachers, and a notable need to provide effective supports to enhance teacher quality, it is important to explore PD avenues that are logistically feasible for both the coach and novice teacher, such as PD that includes technology (Hemmeter et al., 2011; Marturana & Woods, 2012). This study aligns with other BIE eCoaching literature indicating that the intervention changes the behaviors being coached (e.g., Rock et al., 2012; Scheeler, McKinnon, & Stout, 2012) and also extends the literature by demonstrating a method that can enhance the induction supports of novice ECSE teachers.
Impact on Novice Teachers in Inclusive Environments and Child Responses
Teachers experience challenges in meeting the diverse needs of the children in these settings (Berry, 2010). Similar to previous research, results from our study indicate BIE eCoaching has the potential to increase teachers’ use of targeted skills with both focus children and their peers (Coogle et al., 2017, 2015, 2016). In addition, our study suggests that BIE eCoaching can be used to increase teachers’ use of learning opportunities at a rate that is consistent with child acquisition of targeted skills (see Snyder et al., 2015). Teachers in our study provided an average of 3.7 to 10.6 learning opportunities per session; teachers in Snyder and colleagues’ (2015) review provided an average of 7.2 learning opportunities per session (range = 3-20) with sessions averaging 23.3 min in length. These data indicate that results of this eCoaching intervention were similar in size with somewhat less variability. To the best of our knowledge, our intervention is the shortest in duration when compared with other coaching interventions, and it is the first to examine the effect of eCoaching on naturalistic learning opportunities across ECSE teachers and focus child responses to these opportunities. This is an important consideration as it is a manageable amount of time to engage children and to receive feedback.
All novice ECSE teachers maintained communication strategies with their focus children and peers, and one teacher consistently generalized strategies with both the focus child and peers. Although three of the four teachers decreased their level of communication strategies when eCoaching was removed, our findings align with previous research that suggests teachers maintain practices at levels above baseline upon removal of BIE eCoaching (Scheeler et al., 2012). Our data provide more generalization data than other BIE eCoaching studies and data across various activities. Our anecdotal data suggest that use of communication strategies was not related to the type of activity as one teacher was able to successfully generalize to a variety of activities, whereas others used a limited number of strategies despite the type of activity. Because these are only anecdotal descriptions, more research is needed to better understand teachers’ generalized use of practices. Collectively, our findings align with other research suggesting that teachers may need various levels of support in generalizing strategies to new routines and maintaining usage when the coaching is removed (Coogle et al., 2017, 2015, 2016). This is an important consideration when teacher effectiveness and individualized PD are factors of concern.
Among all our focus children, children’s responses to the teachers’ use of the communication strategies were high, but typically not 100%. This is an important finding as it demonstrates that children were provided multiple opportunities to practice communicating, and they responded to several instances. This finding aligns with research suggesting that it is important to provide multiple learning opportunities as young children may not respond to all opportunities provided; however, when children are provided more opportunities, they take more turns to practice a skill (Douglas, Light, & McNaughton, 2013).
Limitations
Limitations are related to intervention procedures, data collection, and data analysis. First, our study procedures make it difficult to determine whether the narrated presentation, BIE eCoaching, or the combination affected teachers’ use of communication strategies, because they occurred as a PD package. In addition, teachers captured their own videos during baseline, maintenance, and generalization, and researchers collected the videos during intervention. This difference could have influenced strategy implementation as there was not a researcher presence during all sessions, and teachers could potentially have rerecorded these sessions if they chose. Moreover, this difference could have played a role in teachers’ report of the social validity of the intervention because intervention sessions needed to be scheduled with the coach, whereas baseline, maintenance, and generalization videos could occur at any time.
An additional limitation was related to data collection. At times, we were unable to collect data for the focus child due to child absences, which could have an effect on the accuracy of children’s data. Although the maintenance and generalization aspects of this research add to the literature, one teacher did not provide an adequate number of sessions to see trends in maintenance and only provided four generalization sessions. In addition, our goal was to provide eight eCoaching sessions for all teachers, but one teacher received only six sessions and two teachers received seven. The prior determination of the number of sessions and the variability in our results limit the interpretation of findings and describe challenges with obtaining in natural environments. Another limitation is that we collected only intervention fidelity data, and not fidelity data for baseline, maintenance, and generalization as well. Finally, it is important to note that these data are specific to four ECSE teachers and focus children in their classrooms, which limits the external validity of the findings.
Finally, we report limitations regarding data analysis. We coded our IOA using a summary-level agreement, which is less stringent than a point-by-point recording method for coding interval data and may have inflated our IOA (Artman, Wolery, & Yoder, 2012).
Future Research
One important consideration when contemplating the adoption of a PD system is the acceptability of the approach. Multiple scholars have demonstrated the benefits of eCoaching with delayed feedback (e.g., Barton, Fuller, & Schnitz, 2016) or immediate feedback (e.g., Rock et al., 2012; Scheeler et al., 2012). However, research is necessary to compare effects based on these two feedback approaches. These data, combined with information on the social validity of both approaches, can give guidance to teacher educators, administrators, and PD providers about the best approach to delivering eCoaching to maximize outcomes.
All teachers demonstrated an increase in level in their use of strategies with the focus child and their peers during intervention; however, data suggest variable findings in teachers’ percentage of correct responses to prompts. It is important to note that prompts to use strategies were provided only when teachers did not spontaneously use strategies and as the coach observed opportunities to use strategies. Moreover, the frequency of strategies used (a) among teachers, (b) by a teacher from one session to another, (c) with focus children, and (d) during maintenance and generalization was also variable. It is difficult to determine why one teacher (Paige) maintained and generalized strategies and the other teachers did not, but this finding suggests that coaching in itself may need to be more individualized with respect to duration or context to support all teachers’ sustained and generalized use of strategies across typical activities and routines of early childhood classrooms.
This research focused on two naturalistic communication strategies to use with children experiencing communication delays. Although these strategies have evidence of effectiveness, identifying naturalistic strategies that would be most effective for children based on their IEP goals is an important consideration. An avenue researchers might consider is aligning strategies coached with children’s IEP goals to tailor supports to the specific children in a teacher’s classroom and then tracking child progress on their specific IEP goals. Researchers might also base methodological phase changes on child outcomes instead of educator outcomes.
It is also important for researchers to consider other aspects of coaching related to what is being coached and how is it being coached. In addition, researchers might examine the length of the intervention and different types of feedback other than BIE eCoaching. As more evidence related to coaching becomes available, it is important that researchers synthesize this information to determine those key aspects that are effective in producing, generalizing, and sustaining change to bridge the research-to-practice gap and ultimately enhance child outcomes.
Conclusion
This research was focused on the use of PD with BIE eCoaching to enhance novice ECSE teachers’ use of naturalistic instruction, an evidence-based practice. Our results were promising, in that the eCoaching PD intervention was effective in creating more learning opportunities for children with communication delays and supporting their direct responses to teachers’ use of the strategies. Next steps for this PD are to (a) continue finessing the individualization of the eCoaching, (b) examine methods for supporting the feasibility of receiving immediate feedback, and (c) examine the intervention’s effectiveness at supporting teachers’ acquisition and implementation of other types of evidence-based practices.
Footnotes
Author’s Note
Naomi L. Rahn is now at Wisconsin Department of Children and Families.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.