Part C Coordinators’ Use of Data Through Implementation of Results-Driven Accountability

Abstract

State administrators have reported data for compliance since the inception of the Individual with Disabilities Education Act, Part C in 1986. Recently, Results-Driven Accountability (RDA) introduced a shift to use data for continuous improvement at the state and local levels. While ample research addresses how teachers learn to use data, none has focused on the state administrators who have to implement RDA. This act assumes that administrators can apply their data to continually improving programs for children with disabilities: however, this has been unexplored by researchers. I used a multiple-case study to investigate the environmental factors that influence the behavioral shift from using data solely for compliance to using data for continuous improvement. This research finds that multiple environmental factors contribute to the use of data for continuous improvement.

Keywords

Results-Driven Accountability continuous improvement data practices compliance

To ensure that families have accessible and high-quality programming under the Individuals with Disabilities Act (IDEA, 2004), federal administrators from the United States Department of Education, Office of Special Education Programs (OSEP, 2014a) created a requirement, Results-Driven Accountability (RDA). The theory of action embedded within RDA states that all children receiving Part C Early Intervention (EI) and Part B 619 Early Childhood Special Education (ECSE) should have quality programs and services available to families (OSEP, 2014a). Federal administrators from OSEP signaled hope to improve these services by transitioning from using data for compliance (e.g., reporting the percent of eligible infants and toddlers with IFSPs for whom an initial evaluation and initial assessment and an initial IFSP meeting were conducted within 45 days) to using data for the support of continuous quality improvement (CQI; e.g., looking across the system to understand what is preventing the program from meeting the 45-day timeline). CQI involves using feedback about processes and outcomes to make evidence-based changes that align with the organizational goals and infrastructure (Torres & Preskill, 2001). In Part C programs, data can be used to continuously identify gaps in EI such as access, quality, cost for the efficient delivery of services, and the continued need to show improved outcomes for children and families (Hebbeler et al., 2012). This shift aligns with a national movement toward data-driven decision-making (DDDM) to support education reform (Datnow & Park, 2022; Gullo, 2013; Mandinach & Jackson, 2012; Mandinach & Schildkamp, 2021).

The implementation of RDA falls on the IDEA Part C coordinators who have the appropriate responsibility and authority to use the data to ensure children with disabilities and families receive high-quality supports and services (Hebbeler, 2015) and asks them to begin using data to inform EI practices in addition to regulatory federal compliance (Hickman, 2022; OSEP, 2012). Yet their historical role as monitors of accountability at the local level (IDEA Infant and Toddler Coordinators Association [ITCA], 2022) is significantly different from the nature of “implementing and sustaining evidence-based strategies in agencies and communities as an ongoing process” (Van Dyke & Naoom, 2016, p. 4). If RDA is to be successfully implemented, then the current barriers to a state administrator’s use of data for more than compliance must be identified, along with potential support to build their capacity to use data (Yazejian & Bryant, 2013). This study seeks to understand the environmental factors that support state administrators using data for CQI required by RDA.

Environmental Factors

Environmental factors are events or situations that surround an individual that would influence a behavior shift (Bandura, 1986). These events and situations happen within the organizational settings surrounding the Part C coordinator and may influence their use of data (Coburn & Turner, 2011; Datnow et al., 2012, 2013) beyond compliance as desired under RDA. Because of the lack of research on Part C coordinators, this study leverages research from educator DDDM. This is appropriate given the focus across general education and early intervention on using data for CQI leading to improved outcomes for children. Based on the research, the following four environmental factors emerged in the literature as areas that could influence the use of data beyond compliance: (a) access to data, (b) data governance, (c) leadership support, and (d) collaborative practices.

Access to the Data

The state administrators have been collecting and reporting data for IDEA Part C to ensure compliance since the program’s inception in 1975 (Bruder, 2010). Given the long history of data collection, it is not a surprise that Thayer et al. (2022) reported that in 2019 all state administering agencies had an IDEA Part C data system, and 98% of state administrators could access child-level data in their system, but only 76% of states reported access to be real-time information. Although there were data systems in states prior to RDA (Derrington et al., 2013), the information captured with the systems varied across states and could influence the types of decisions Part C coordinators are making using data. For example,

one-fourth of state Part C data systems contain data about program structure (e.g., service model, number of regular or contracted staff, administrative agency) . . .. none of the Part C program data systems contain data about program costs or quality (e.g., program quality measures such as the Infant/Toddler Environment Rating Scale, Quality Rating and Improvement System [QRIS] rating, accreditation). (Derrington et al., 2013, p. 7)

Similarly, in 2019, Part C coordinators reported collecting data on child and family demographics, child outcomes, referrals, workforce, participation or transition to other programs, and service data, but not all states have the access to the data needed to answer their questions about the quality of service delivery or workforce (Thayer et al., 2022). To successfully use data for CQI, Part C coordinators need access to web-based, real-time data systems (Buzhardt et al., 2010).

In addition, state administrators need access to data across programs. Because many children and families enrolled in IDEA Part C are also enrolled in other programs such as Early Head Start, Part C coordinators may need data external to their systems to help them make decisions (Derrington & Cochenour, 2015). Fortunately, many state administrators have been working to create integrated and longitudinal data systems. According to the Institute of Education Sciences (IES, 2011)

[with the] influx of more than half a billion federal dollars over the last eight years, nearly every state has invested substantial new resources in building data systems to track key information on their students and educators over time . . . the purpose of this effort is to enable state educational agencies to design, develop, and implement statewide, longitudinal data systems to efficiently and accurately manage, analyze, disaggregate, and use individual student data. (p. 3)

According to a national survey conducted in 2022, of the 10 operational early childhood integrated data systems (ECIDS), seven states included Part C in their integrated data system (Coffey et al., 2023). Including Part C data in the ECIDS would support information sharing to inform statewide decisions about CQI for Early Intervention.

Data Governance

Data governance “is both an organizational process and a structure; it establishes responsibility for data, organizing program area staff to collaboratively and continuously improve data quality through the systematic creation and enforcement of policies, roles, responsibilities, and procedures” (National Forum on Education Statistics, 2011, p. 9). The establishment of data governance provides the foundation to use data to inform decision-making by identifying the processes necessary to collect, analyze, and report data. Having such a data governance system in place allows Part C coordinators to use high-quality information for their decisions (Cochenour & Hebbeler, 2015) and thereby builds the Part C coordinators’ ability to use data for CQI. According to Thayer et al. (2022), a majority of the states’ IDEA Part C data systems do not have a data governance body with the responsibility of overseeing the data and the data system. If only a minority (30%) of Part C coordinators have established data governance, that could impact their use of the data to make decisions. If data governance does not exist then the Part C coordinators would not have a sustainable way of collecting, analyzing, and reporting data which impacts their ability to use data for state leadership decision-making (Allard et al., 2018). Thus, data governance may play a significant role in Part C coordinators’ use of data to inform their decisions.

Leadership Support

Research has shown the important role of leadership in setting up successful structural supports for individuals to use data for CQI (Datnow et al., 2012, 2013; Halverson et al., 2007; Park & Datnow, 2009). Given how Part C coordinators are situated within a larger early childhood system, representing one program (Part C), their ability to receive support to use data beyond compliance is relevant (Gupta et al., 2023). Leadership support for the Part C coordinator comes directly from the executive or cabinet level administrators of the state or agency leadership. State executive leadership can acquire data, develop programs, provide formative feedback, and align the task of using the data to inform practice at the state and local levels. These educational leaders play an important role in designing a structure that ensures effective data use (Halverson et al., 2007; Jimerson et al., 2021). Specific to RDA, Van Dyke and Naoom (2016) state that “without effective leadership then RDA may be viewed as another ‘externally oriented accountability,’ compliance-only measure, with no real change in educational opportunities or growth for students with disabilities” (p. 12). Leadership needs to set the priority for DDDM and provide the relevant “time, energy, and resources” (Van Dyke & Naoom, 2016, p. 12) to implement effectively. That includes targeted professional development on using data (deMonsabert et al., 2022). Although these leadership roles are essential, there are also limitations. Executive or cabinet-level positions are often political, and as research by Coburn and Turner (2011) notes, political processes can bias program data to support a specific agenda, diverting focus from objective data analysis.

Collaboration Process

Collaboratively using data in job-alike roles has been found to play an important part in developing the necessary skills to use data to inform decisions in education (Datnow et al., 2012, 2013; Honig & Venkateswaran, 2012; Means et al., 2012; Slavit et al., 2013). These state systems were initially designed to respond to federal accountability requirements under No Child Left Behind Act (NCLB, 2001) but are now embedded in state education policy discussions. Supporting collaborative practices could be one form of group intervention (Coburn & Turner, 2011) to support Part C coordinators’ use of data and to promote the use of data for more than compliance. Recent research focused on collaborative practices that enable the use of data for decision-making articulates the necessary commitment of the group to using data, allocating time, and creating a data culture that embraces the risks of using data collaboratively (deMonsabert et al., 2022; Jimerson et al., 2021).

In summary, the literature on environmental factors relating to DDDM indicated that influencing environmental factors such as (a) access to data, (b) data governance, (c) leadership support, and (d) collaborative practices would support the use of data to inform decisions for continuous improvement. RDA policy requires Part C coordinators to use collected data in decision-making, beyond its traditional use for compliance monitoring. However, the environmental factors that support the Part C Coordinators’ implementation of RDA are unknown.

Research Question

The field of early childhood care and education has not addressed the issue of DDDM adequately, especially not in Early Intervention where Part C coordinators are asked to use data for more than compliance. The purpose of this case study is to understand what capacity IDEA Part C coordinators that are currently demonstrating the ability to use data beyond compliance have in supporting the implementation of the RDA policy from OSEP. Specifically, how do environmental factors such as access to data, data governance, leadership support, and collaborative practices influence Part C state administrators’ use of data for decision-making beyond compliance?

Method

To respond to the research question, a multi-case study was conducted focusing on four Part C coordinators. They were selected for their exemplary use of data for more than compliance as desired under RDA and are referred to as the exemplars throughout this study. The Part C coordinators are a special group of data users with a history of focusing on data for federal compliance, but with RDA they are being asked to use data for CQI which goes beyond their traditional role. Given the small number of people in the role of Part C coordinator and the need to understand the state context that is relevant to implementation when looking at how Part C coordinators use data for CQI, a case study across multiple coordinators using data beyond compliance is appropriate.

Participant Selection

A maximum variation sampling strategy (Patton, 2015) was employed to purposefully select exemplar Part C coordinators out of the 56 potential coordinators in each state and territory that administers the Part C program. Multiple case studies need enough cases to understand similarities between the states with rich context with generalizability, at least four, but too many (i.e., more than 15) can add diminishing returns and make the analysis unnecessarily complex (Stake, 2006). Given Stake’s recommendation for 4 to 15 cases for a multiple case study and the small size of the total population, my goal was to have four to six Part C coordinators with a range of tenure, percentage of children served, comprehensiveness of their data system, and their implementation strategy area participate in the study. To ensure a broad range of experiences among state Part C coordinators were represented, I undertook two steps. First, a national panel of experts were surveyed to identify exemplar Part C coordinators. Second, comprehensive profiles of the identified exemplar Part C coordinators were developed and used to assure representation across different criteria. These two steps are described in the next sections.

National Expert Survey

Using selection criteria based on the role in which they work with Part C coordinators to use data, 24 experts were identified and were asked to help identify exemplar Part C coordinators using data beyond compliance. Experts were recruited because they had a position where they were explicitly working with Part C coordinators to use data. Staff from OSEP that focus on data, national researchers with expertise on data use in Part C, project directors of national IDEA centers, national associations, and owners of technology solutions working with states were asked to participate in a survey. The survey asked the national experts about their demographics, experience, and asked for examples of specific implementation strategies in technical, collaborative, and governance supports to help Part C coordinators use their data to inform program and policy decisions. These three categories were developed to align with the four environmental factors identified in research but to be more general to allow for ideas that may be outside of the literature. Eight out of 24 national experts recruited over email completed the survey. At least one person from each role type responded to the survey. Their average experience in their role was 10.42 years, and they interacted with an average of 17 states on a regular basis. The survey of national experts led to the identification of exemplar state administrators using data beyond compliance and those with examples of implementation strategies. Of the 56 states and territories that provide Part C services, the national experts identified 22 different exemplar states, 10 were highlighted for implementation strategies using technology solutions, eight for collaborative practices, and six for governance supports. Some states were identified with more than one strategy area. Three of the 10 technology states were consistent across national experts, three of the eight in collaborative practices, and two were common for governance.

Comprehensive Profiles

Twenty-two state profiles were developed and reviewed to maximize variation. Geographic region, percentage of children served in Part C, tenure in role, and comprehensiveness of data system were used to identify state participants. The geographic region and population served were used so that the exemplar state administrators could be helpful to all state administrators, the percentage of children served ranges across states from 1% to 8.89% (OSEP, 2017). Given the recent increase in Part C coordinator turnover nationally (ITCA, 2016), the length of time in the role was critical to make sure that those included in the study had enough experience to respond to the interview questions about decisions made in this role. For purposes of this study, participants needed to have more than 2 years of experience in their current positions as a Part C coordinator. In addition, the Part C coordinators needed comprehensive data systems to pull information to use in decision-making. Comprehensive data systems have child, workforce, and program level data into one integrated data system (Derrington et al., 2013). The DaSy State of the States survey (Derrington et al., 2013) gathers national data on data within IDEA Part C and Part B. Part of the information gathered analyzes what data was included in the Part C program data system, how it was accessed, and whether it was subject to data governance were some of the many things that varied across the states and territories. This information was used to help identify the comprehensiveness of the data collected by the Part C data system for variation.

Using the 22 comprehensive profiles, I narrowed down to six Part C coordinators where there was variation across geography, size of population served, tenure, and comprehensiveness of the data system. This allowed for greater understanding of patterns that cut across the areas of implementation strategies identified in the national expert survey. Three alternates were selected in case any of the original six selected did not participate, ensuring that at least four states were secured. Each of the alternates were selected based on similar criteria and had similar state profiles or a similar focus on the categories. A summary of the selected Part C coordinators and their state data context is given in Table 1.

Table 1.

Exemplar Part C Coordinator Selection Based on Part C Coordinator Profile and National Expert Survey.

State geographic type	Number of national expert nominations by category	% of children served^a	Part C coordinator time in role	Comprehensive data system^b
Southeast^c	2 technology solution & 1 collaborative practices	Approx 2%	—	—
Northwest	2 technology solution & 1 collaborative practices	Approx 2%	5 years^d	Child and workforce data
Midwest	1 technology solution	Approx 2%	5 years	Child
West	2 collaborative practices & 3 governance supports	Approx 2%	2 years	Child and workforce data
Midwest	2 technology solution & 2 collaborative practices & 1 governance supports	Approx 2%	6 years	Child and workforce data
Mid-Atlantic	1 technology solution & 1 collaborative practices	Approx 3%	—	Child and workforce
Northeast	2 technology solution	Approx 4%	24 years^d	Child and workforce data
Mid-Atlantic	3 collaborative practices & 2 governance supports	Approx 4%	2 years	Child and workforce data
Northeast	2 technology solution & 1 collaborative practices	Approx 4%	—	Child and workforce

Note. Alternates are identified in italics; states that participated are bold.

Percentage of children served by Part C in the state (OSEP, 2017). ^b DaSy State of the States survey (Derrington et al., 2013) identified which states had child and workforce data included in their state Part C data systems. ^c The southeast state’s coordinator’s time in the role and comprehensive data system rating is unknown because they did not respond to the DaSy survey (Derrington et al., 2013). ^d Time in role obtained from LinkedIn profile.

In summary, six exemplar states and three alternates were selected from the 56 Part C coordinators in the United States and its territories. I recruited these states over email and provided with information about the study. Three of the original six state administrators agreed to participate and were from the Midwest, Mid-Atlantic, and West. One alternate, the Midwest Part C Coordinator, agreed to participate as an alternate for the Southeast and Northwest states given the percentage of children served by the program and the focus on technology solutions. In addition, for each of the selected states, the participants included the Part C coordinator and the data manager responsible for helping them gather and use the data to provide additional context on factors that influence the use of data by their Part C coordinator.

Data Collection

National Experts Survey

The first step of the data collection was to survey national experts for identification of exemplar states. The eight-question online survey asked experts to identify states that demonstrated exemplary data practices in the following categories: technology solutions, collaborative practices, and governance supports. I designed this survey to gather basic demographic information, including their Part C data experience, and three open-ended questions asking for state examples of specific implementation strategies in technical, collaborative, and governance supports to help Part C coordinators use their data to inform program and policy decisions.

State Profiles

In addition, Part C coordinator state profiles were created for all 22 identified possible exemplar states to understand the context that surrounds the Part C coordinators. Each state profile combined data that are publicly available and components such as the state agency structure, which agency houses IDEA Part C, the number of staff in IDEA Part C, the documented role of the Part C coordinator based on the ITCA (2016) survey and national Part C data system information (Derrington et al., 2013). Together the data from the national expert survey and the state profiles were used to identify the exemplar states.

Demographic Survey

Once the state administrators agreed to participate, demographic data from the Part C coordinator and data managers were gathered using an online survey tool. The questions focused on education, background, and tenure in the current position. The Part C data managers responded to an additional question about their reporting structure to the Part C coordinator in their state for context to the environmental factors.

Interviews of Part C Coordinators

Then, individual, semi-structured telephone interviews (Silverman, 2005) were conducted with each of the exemplar Part C coordinator participants. The interviews were scheduled at 60-minute session and ranged from 45 to 76 minutes, averaging 63 minutes. The interviews were guided by an interview protocol with 17 open-ended questions designed around the factors identified in the literature that contribute to using data beyond compliance. Given this study focuses on the environmental factors that increase their ability to use data, of the 17 questions, 5 were specific to environmental factors identified in the literature, with 12 sub-questions, and four general questions about their data use practices. Examples of these questions are: “Would you please tell me about your process for data collection and how your program makes decisions about data collection and reporting?” and “Please tell me a bit about the leadership structure in your agency? How do they use data?”

Interviews of Data Managers

Based on the initial interviews conducted with the exemplar Part C coordinators, follow-up interviews with Part C data managers to get a comprehensive understanding of the case were conducted. The data managers participated with the support of the Part C coordinator in their state. These were designed to triangulate the Part C coordinators’ responses (Silverman, 2005). The data manager interviews were scheduled at 60-minute sessions and ranged from 49 minutes to 72 minutes with an average interview of 64 minutes. The follow-up semi-structured interviews were guided by a 15 open-ended question protocol, three of which focused on environmental factors and six on their role as a support to the Part C coordinator and their data use practices. The three environmental questions were the same as those asked of the Part C coordinators, but different questions about data use were asked given the role of data managers and Part C coordinators. Examples of the unique questions asked of data managers are: “Can you describe your role in helping others use Part C data?,” “Can you share a bit about how you work with the Part C coordinator to use data?,” and “What reports or products do you create to help Part C coordinators make decisions?”

With participants’ consent, all interviews were recorded and transcribed by Rev.com and reviewed for accuracy. The interview protocols were validated by a qualitative methodologist and the interpretive community used for this study. All eight of the original national experts were recruited for the interpretive community, the three that participated were all national experts funded by OSEP to support states in using data to inform their decisions beyond compliance. Also, after each interview, memos were used to understand how the data collection informed my thinking (Maxwell, 2013; Saldaña, 2021). Specifically, the memos were used to help test assumptions about the interviews as well as any preliminary thoughts about how the interviewee used data to inform decisions.

Data Analysis

As Miles et al. (2014) recommend, I used a process of three data analysis activities: (a) data condensation, (b) data display, and (c) conclusion drawing/verification (p. 12). In the first activity, data condensation, which is the “process of selecting, focusing, simplifying, abstracting, and/or transforming the data that appear in the full corpus of . . . materials” (Miles et al., 2014, p. 12), all interview data were transcribed verbatim and read through twice in their entirety before any initial coding (Ryan & Bernard, 2003). During this stage, each of the interviewees were asked to review their transcribed data to see if there were statements that were not captured correctly and needed to be clarified. The second activity, data display, “is an organized, compressed assembly of information that allows conclusion drawing and action” (Miles et al., 2014, pp. 12–13). A series of matrices were developed for the environmental factors. For example, a relationship matrix of the types of decisions and environmental factors was analyzed and coded. As is commonly done in qualitative research, multiple waves of coding (Saldaña, 2021) were used iteratively and flexibly to explore the factors that influence the shift from using data for compliance to using data for CQI. In the initial coding cycle, data were coded in the following three ways: (a) a priori coding based on the literature, (b) attribute coding, and (c) structural coding. In the second cycle of coding, data were synthesized across the case studies using (a) taxonomic coding, (b) process coding, (c) values coding, (d) magnitude, and (e) patterns coding.

During the final analysis activity, an interpretive community and member checks were used to review the accuracy of the coded data collected through the profiles and interviews. This group of three researchers were experts who worked with state administrators on using data and could help to ensure the trustworthiness (Lincoln & Guba, 1985) and authenticity of the research (Patton, 2015). The interpretive community was asked to review selections of coded data (Miles et al., 2014) over email to confirm themes and identify missing information in the data. For example, what were the participants not saying in the interviews that might have been expected? Based on the input of this interpretive community, I revisited my analysis to make richer connections among factors and participants.

Member checks were used to ensure the interpretations accurately reflected the participant’s intended meanings and experiences (Lincoln & Guba, 1985). These checks were conducted individually with each Part C coordinator and data manager over email. They were provided with time to review the summarized findings, reflect on how the findings align with their experience, and review how their statements were used to support a finding. In each case, they identified additional data that could be used to justify the finding, such as updated job descriptions, State Systemic Improvement Plans (SSIPs). No participant asked for a statement to be removed.

Findings

The state administrators (Part C coordinators state administrators and their data managers) in this study articulated three environmental factors that influenced their exemplary use of data for CQI: (a) access to relevant data, (b) collaborative data practices, and (c) alignment between the administrator and executive state leadership about using data beyond compliance.

Access to Relevant Data

Exemplar State Administrators Had Comprehensive Part C Data Systems That Provide Current Data

Not only did all four exemplar states have a Part C data system, but these particular state administrators also have access to “real-time” data, not only the annual data, which is intended for basic federal reporting, “We run data monthly, and then quarterly, so we’re not just relying on our year-end data to provide information to our local programs of their performance and to make changes” (Data Manager 4). The need to access recent data to inform decisions was something these exemplar state administrators focused on and shared the impact current data had on providing better services to children and families (States 1, 3, 4), explicitly saying “our new system is live data, so if they make a change we see it immediately” (Part C Coordinator 4). This immediacy of the data also allows the Part C coordinators to make decisions in a timely manner that will be most beneficial for the children and families they serve. In addition to accessing current data, they described their access to comprehensive data. For example, Data Manager 1, described how “we have invested heavily in what is called the early childhood real-time data warehouse. And so all of our data elements are being tied together in a way that they’ve never been tied together before.” If the exemplar state administrators were only focusing on compliance, then collecting the annual data would be sufficient, but when using data for program decisions that occur all year long, the ability to access comprehensive, real-time data becomes critical.

Exemplar State Administrators Had Access to Integrated Data With Other Relevant Programs

The comprehensiveness of the data depends on integrating other state data sources that are related to Part C, but not governed by Part C, programs like the Early Hearing and Detection Intervention (EDHI) and Part B 619. According to Part C coordinator 1, “[We have] some [data] with early hearing detection to look at how the kids are being served, how are they doing on their outcomes, those different types of things.” Three of the four Part C coordinators have access to integrated EDHI data (Part C coordinators 1, 2, 3), “And so our role, because we have a data system that allows us to use that data, is to help identify populations and collaboration with other departments (EDHI)” (Data Manager 2).

The other critical program to integrate with is Part B 619 because this is where Part C transitions the children who continue to need services when they turn three. These data are one of the compliance indicators, but state administrators have struggled to connect these two programs. The exemplar state administrators in three states are integrating Early Intervention with Part B 619 either through sharing agreements (States 3, 4) or because EI and Part B Early Childhood Special Education are both housed in the same data system (States 1, 2).

Child Abuse Prevention and Treatment Act (CAPTA, 1974) is a federal law and supplemental grant program that provides funding to states to improve their child protective services systems. Under the Keeping Children and Families Safe Act (2003) CAPTA was amended to require states to include “provisions to refer children under age three who are involved in a substantiated case to early intervention services under IDEA Part C” (Keeping Children and Families Safe Act, 2003, p. 811). As such CAPTA and EI work closely to track referrals across programs. State administrators in States 1, 3, and 4 all use the CAPTA to understand the referral process.

We’ve been looking at our CAPTA referrals to (early intervention) program, and again, it’s something that’s federally mandated to happen, but we never really had deeper conversations about how many children are being referred to the (early intervention) program, what happens to them, how many children are enrolling, what do we know about outcomes for children through the CAPTA referrals. So, we’ve looked at a lot of data to do some analysis of that, and found some maybe unexpected numbers, and also some reasons to pause on what we may or may not be doing as a system to support those children. That led back up then to bigger policy questions about children that are referred to the CAPTA system. (Part C Coordinator 3)

In State 4, the state administrators track the number of referrals coming through CAPTA: “Right now our Early Intervention program reports on the number of . . . children referred through CAPTA that complete the eligibility process, that’s one of our measurements” (Part C Coordinator 4). Many grant programs have limited resources, so the description of decisions based on referral and eligibility are likely applicable to other programs and if the data are not integrated as Part C Coordinator 3 mentioned, then there is no transparency into the process at the state level, and the state administrators cannot provide additional resources or support to help the local programs if there is no quality data to inform the conversation.

Collaborative Data Practices

Exemplar State Administrators Collaborated With Data Managers to Prepare the Data to be Used to Inform Decisions

Part C coordinators from all exemplar states described how the data manager is critical to the use of data to inform program and policy decisions. The role of the data manager is to collect and review the quality of the data. The Part C coordinators commonly referenced reaching out to the data manager when they had a question about the data, “I usually defer to [the data manager] to save me time, but also, I know that [they are] going to be able to have the answer a lot quicker than I would if I tried to look for it” (Part C Coordinator 2).

Every state has a data manager, but the difference seems to be in the role the data manager plays in using the data. Not only did the exemplar state administrators reference this role to help them dig into the data but also to help prepare the data so it could be used for decisions making. Part C Coordinator 3 shared how having the data manager opened the door for new ways to look at the data, “they’re able to actually use the data and display it in a different way, and to do deeper analysis of that data so that we have a better conversation of it” (Part C Coordinator 3). Adding this new role in State 3 helped the Part C coordinator better use the data they had and think of new ways to use it that they could not have before. Part C Coordinator 1 has the largest Part C program team out of all the exemplar states. Because of the additional staff support, Part C Coordinator 1 had the capacity to do more:

We’ve been able to add a couple of staff people to my team over the last few years, which is giving us the capacity to do a lot more . . . is it’s really exciting that we’re moving beyond just the ability to look at fiscal and compliance data to really be able to start tackling the quality of the works that we’re doing. (Part C Coordinator 4)

The critical role of the data manager is illustrated by how all the exemplar state administrators not only had a data manager check the quality of the data but also had the capacity to analyze and prepare it so the data could be used to inform the decisions.

Exemplar Part C Coordinators Used Data to Make Decisions in Collaborative Leadership Teams

The exemplar state administrators do not make decisions in a silo. Instead, they either lead, or are part of, decision-making structures that necessitate collaborative use of data. In all four states, there were leadership teams where collaborative interpretation of the data was essential to it being used for decisions related to Part C program implementation and policies. Three of the four Part C coordinators discussed the teams that existed in the Part C program (Part C Coordinators 1, 2, 4), and Part C Coordinator 3 described an agency-wide team that included but was not limited to the Part C coordinator.

In two of the four states, the Part C coordinators discussed explicit collaborative decision-making (Part C Coordinators 2, 4). State 4 meets monthly to review progress toward the annual goals outlined in the State Performance Plan/Annual Performance Report (SPP/APR). They use data to discuss the implications and any needed changes, “We report that data in a big meeting on a monthly basis and then it gives the [leadership] team the opportunity to ask us questions and sort of put everybody’s heads together to see how we can improve things.” (Part C Coordinator 4). Beyond the annual goal setting and progress tracking, the exemplar Part C coordinators brought the data relevant to the types of decisions they would be making in a meeting and the data was intentionally aligned with their decision-making timelines and processes, “I’ve really tied it [data] in a way that’s really meaningful to the work that’s being done” (Data Manager 4). If they were working to address a specific gap in service delivery the Part C coordinator would ensure they had the most recent data on the service area, needs, and any relevant information about why the gap might exist to review with the team. For example, they said they were able to pull data from the surrounding geographic areas to see if families were moving or if there was a shortage in providers based on enrollments in higher education or compensation in surrounding areas. Using a team to review the data was stated to be one of the most critical elements of their use of data because it leverages the knowledge across the team to inform the decisions, “as a group, we have a really nice complement of people and bring to the table a lot of good information and background knowledge that’s allowed us to be successful” (Data Manager 2).

Alignment Between the Administrator and Executive State Leadership About Using Data Beyond Compliance

Exemplar State Administrators Had Expectations From Their Managers to Use the Data for More Than Compliance

The exemplar state administrators who participated in this study made it clear that the state agency leadership was supportive and wanted to use the data to inform their leadership decisions across programs, not only Part C, to the extent that one of the Part C coordinators described the state agency leadership as “really data-driven” (Part C Coordinator 4). The Part C coordinators and data managers see the shift that is happening and are working to be able to meet the evolving needs of leadership, so they too can use data, “I do think that since coming on board there has been a big change in data-driven decisions . . . and I see them definitely moving away from making decisions based on theories and feelings and more doing it on data” (Data Manager 3). The data manager in State 2 described how the leadership expects data to be used across programs (beyond Part C) to understand the number of children served in all state and federally-funded programs as well as the quality of the programs (Part C Coordinator 2). The participating exemplar state administrators were expected by the state agency leadership to use the data to improve the program, not simply report compliance data, and they described how that helped them to better use the data in the Part C program.

Exemplary Part C Coordinators Believed Data Should Be Used Beyond Compliance

All exemplar Part C coordinators believed that data should be used beyond compliance. In addition, they all believed that their role was to help local program administrators move beyond compliance reporting and start using data for CQI. These exemplar state administrators did not describe it as compliance or CQI, but more as a cycle where compliance is the beginning and should lead to improvement strategies. Part C coordinators from the exemplar states describe the cyclical nature of the decisions they make using data, and how data use for CQI builds from compliance, aligns with their role as a support to the local programs, and their use of data in the EI program and state agency to improve services. The state administrators in State 2 had a detailed example of the action they were able to take because they had a statewide perspective on the compliance data (timely services) and how they were able to help the program improve practices by allocating other staff to the area based on an identified need.

We saw one of the programs was having late delivery of services, and we could look at the data, it was all focused on physical therapy shortages. When we actually talked to the county administrator that it was related to, she had a person that was fired, and a person that was on maternity leave, and another person that had left for another job. We knew that we could then assist her in trying to find some physical therapists to help out with the service delivery for that county. (Part C Coordinator 2)

The Part C coordinator in State 4 described how they changed policies to support programs because of the compliance data they had and leveraged to understand a broader need, saying “We just used the data to determine how much Medicaid dollars we were under-utilizing and made some policy decisions around our programs contracting with only providers who are enrolled with Medicaid” (Part C Coordinator 4). In each state, there were examples of the integrated efforts to build from compliance to support program CQI.

Discussion

There is ample research that addresses how teachers use data to inform their decisions (Datnow & Park, 2022; Gullo, 2013; Mandinach & Jackson, 2012; Mandinach & Schildkamp, 2021), but much less research has focused on the people who have to implement RDA. No research has examined what factors might help state administrators build their capacity to use data beyond compliance. This study found that there are environmental factors related to the administrator’s ability to use data beyond compliance as required under RDA. State administrators need (a) access to relevant data, (b) collaborative data practices, and (c) alignment about using data beyond compliance with state leadership. Exemplar state administrators all had these commonalities that supported them during the implementation of RDA.

Although there were four factors identified in the literature (collaborative practices, leadership support, data governance, and access to data), this study did not find data governance to be salient. Perhaps data governance is not a stand-alone factor, but it was highlighted as a part of access to relevant data, meaning it was relevant to ensure they had access to the data. However, since the Part C coordinators oversee the data and data system, they typically play a key role in data governance and may not see it as a separate support needed from their state agencies. Similarly, the political context was embedded under leadership support and was not a finding mentioned by the exemplary Part C Coordinators, but it could be seen as part of the alignment between the administrator and state leadership to use data for more than compliance reporting. Unfortunately, beyond the framework for data use of Coburn and Turner (2011), there has not been much research on the political context around data use. Political context, and specifically policies as the driver to use data beyond compliance, would benefit from future research. Without these environmental factors and the identification of other factors that promote or inhibit these practices, state administrators such as Part C coordinators will likely struggle to implement policies that use data for CQI such as RDA (Van Dyke and Naoom, 2016).

This study focused on exemplary state administrators, yet all states are expected to implement RDA and may not have what is needed to move from using data for compliance to CQI. Other state administrators may have a few of these environmental supports to use data beyond compliance and are often outside the authority of other Part C coordinators (e.g., leadership support). In fact, Part C programs often collect data on child demographics but lack information about service delivery, workforce, and program characteristics needed in a comprehensive data system to inform statewide decisions (Thayer et al., 2022). Building comprehensive data systems will take time, but given the history of data collection in IDEA, Part C Coordinators are enhancing their data systems so the data can be used to inform decisions. As this study reports, even the exemplar states noted how challenging the environmental factors can be to influence, given the capacity and authority of the role of the Part C coordinator. However, the picture is not all grim. In fact, as new data systems are modernized, there are opportunities to make the data more comprehensive and can report data using business intelligence tools that display the data in ways that help inform decisions (Thayer et al., 2022).

Despite a lack of research on how state administrators are using data to inform practice in early childhood education, federal program offices and state administering agencies are investing in expensive ECIDS to support the use of data to inform policy and practice (Coffey et al., 2014; Morrison et al., 2021). In conjunction with RDA, staff at OSEP allocated over 50 million dollars to develop national technical assistance centers dedicated to supporting outcomes for individuals with disabilities. These centers aim to help state administrators transition beyond compliance toward CQI (OSEP, 2014b). Future research on the technical assistance centers’ role in developing the capacity to make the shift would inform the field and understand the return on federal investment.

Limitations

As with any study, there are limitations. First, the state administrators who participated in the study were selected as exemplary state administrators. It is clear in my analysis that the state administrators were exemplars for more than technology solutions, collaborative practices, or governance supports. They had well-staffed teams that worked at the state level, including data managers who had statistical skills to analyze large data sets and report the information in a way that supported decision-making. Second, the state administrators that participated in this study only represent four states of the 56 possible states and territories that offer Part C. Drawing inferences from this sample to Part C state administrators in 56 states and territories that have differences in the state context is not feasible. The third limitation focuses on using exemplar Early Intervention state administrators to describe the role of an entire set of professionals across the country. Although the Part C coordinators and data managers are critical roles for the success of using data beyond compliance in the Early Intervention program, the exemplar state administrators mentioned the other state administrator roles they support, including professional development specialists. In addition, this study focused on the state level, not the direct service local programs, but they were often mentioned, and a future study should look at all the roles that influence the use of data in Early Intervention. Fourth, and finally, this study used the number of children served in EI as a proxy measure, but it did not account for the state economics (e.g., budget toward these programs, supplemental services). Without complete state economics, the inferences that can be made across state agencies providing EI services that may or may not have similar state economics are limited.

Future Research

This study explicitly focused on environmental factors related to using data beyond compliance, but there may be other factors to consider. The Division for Early Childhood (DEC, 2014) recommends that Part C, “leaders collaborate with stakeholders to collect and use data for program management and continuous improvement . . .” (p. 9), but as Bruns et al. (2017) points out, there have been no empirical studies on Part C coordinators practices. In future studies, investigating other factors that affect Part C coordinators’ ability to use data beyond compliance, such as personal factors (e.g., competencies to use data), may further inform the implementation of RDA and other data-informed decisions being made by state administrators. Gupta et al. (2023) went so far as to recommend that IDEA define expectations for Part C coordinators. Continued research on the barriers that prevent Part C coordinators from becoming exemplary users of data for CQI would not only benefit Part C programs as they implement RDA, but also other programs such as Early Childhood Special Education and Head Start programs that have similar program requirements to use data for CQI. The process for decision-making and barriers to implementing data governance would be helpful in understanding the barriers to using data beyond compliance. In addition, looking further into the role of Part C coordinators could contribute to the conversations about what authority state administrators have to use data beyond compliance, not just for RDA, but for other state legislative priorities related to using data for more than compliance reporting.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Missy Coffey

References

Allard

S. W.

Wiegand

E. R.

Schlecht

Datta

A. R.

Goerge

R. M.

Weigensberg

(2018). State Agencies’ use of administrative data for improved practice: Needs, challenges, and opportunities. Public Administration Review, 78(2), 240–250. https://doi.org/10.1111/puar.12883

Bandura

(1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall, Inc.

Bruder

M. B.

(2010). Early childhood intervention: A promise to children and their families for the future. Exceptional Children, 76(3), 339–355. https://doi.org/10.1177/001440291007600306

Bruns

D. A.

LaRocco

D. J.

Sharp

O. L.

Sopko

K. M.

(2017). Leadership competencies in U.S. Early Intervention/Early Childhood Special Education service systems: A national survey. Infants & Young Children, 30(4), 304. https://doi.org/10.1097/IYC.0000000000000104

Buzhardt

Greenwood

Walker

Carta

Terry

Garrett

(2010). A web-based tool to support data-based Early Intervention decision making. Topics in Early Childhood Special Education, 29(4), 201–213. https://doi.org/10.1177/0271121409353350

Child Abuse Prevention and Treatment Act of 1974, Pub. L. No. 93-247, 42 U.S.C. § 5101 et seq. (2004).

Coburn

C. E.

Turner

E. O.

(2011). Research on data use: A framework and analysis. Measurement: Interdisciplinary Research and Perspectives, 9(4), 173–206. https://doi.org/10.1080/15366367.2011.626729

Cochenour

Hebbeler

(2015). Early childhood data governance: A prerequisite for answering important policy questions. In Kagan

S. L.

Gomez

R. E.

(Eds.), Early childhood governance (pp. 112–120). Teachers College Press.

Coffey

Chatis

Irvine

Sellers

Duarte

(2014). What is an early childhood integrated data system (ECIDS). National Center for Education Statistics. https://slds.grads360.org/#communities/pdc/documents/4441

10.

Coffey

Sirinides

Dabbs

E. R.

(2023, August 9-11). 2022 Early childhood integrated data national study [Conference session]. National Center for Education Statistics STATS-DC Data Conference, Washington, DC, United States.

11.

Datnow

Park

(2022). Evidence-informed decision making in schools and districts: The role of equity (pp. 169–176). https://doi.org/10.1016/b978-0-12-818630-5.05033-8

12.

Datnow

Park

Kennedy-Lewis

(2012). High school teachers’ use of data to inform instruction. Journal of Education for Students Placed at Risk, 17, 247–265. https://doi.org/10.1080/10824669.2012.718944

13.

Datnow

Park

Kennedy-Lewis

(2013). Affordances and constraints in the context of teacher collaboration for the purpose of data use. Journal of Educational Administration, 51, 341–362. https://doi.org/10.1108/09578231311311500

14.

deMonsabert

Brookes

Coffey

M. M.

Thornburg

(2022). Data use for continuous instructional improvement in early childhood education settings. Early Childhood Education Journal, 50(3), 493–502. https://doi.org/10.1007/s10643-021-01168-3

15.

Derrington

Cochenour

(2015, December). Big data for little kids: The importance and utility of early childhood integrated data systems [Poster presentation]. Annual Zero to Three National Training Institute, Seattle, WA.

16.

Derrington

Spiker

Hebbeler

Diefendorf

(2013). IDEA Part C and Part B 619 state data systems: Current status and future priorities. https://dasycenter.sri.com/downloads/DaSy_papers/DaSyNeedsAssessmentReport.pdf

17.

Division for Early Childhood. (2014). Division for Early Childhood recommended practices in early intervention/early childhood special education 2014. http://www.dec-sped.org/dec-recommended-practices

18.

Gullo

D. F.

(2013). Improving instructional practices, policies, and student outcomes for early childhood language and literacy through data-driven decision making. Early Childhood Education Journal, 41, 413–421. https://doi.org/10.1007/s10643-013-0581-x

19.

Gupta

S. S.

Sherif

Zhu

(2023). Re-examining state Part C Early Intervention program coordinators’ practices through a positive lens on leadership: A qualitative secondary analysis. The Qualitative Report, 28(2), 517–543. https://doi.org/10.46743/2160-3715/2023.4786

20.

Halverson

Grigg

Prichett

Thomas

(2007). The new instructional leadership: Creating data-driven instructional systems in school. Journal of School Leadership, 17, 159–194. https://doi.org/10.1177/105268460701700202

21.

Hebbeler

(2015). Setting the course and steering the ship. In Division for Early Childhood (Ed.), DEC recommended practices: Enhancing services for young children with disabilities and their families (pp. 23–33). https://books.google.com/books?id=WG3CsgEACAAJ

22.

Hebbeler

Spiker

Kahn

(2012). Individuals with Disabilities Education Act’s early childhood programs: Powerful vision and pesky details. Topics in Early Childhood Special Education, 31(4), 199–207. https://doi.org/10.1177/0271121411429077

23.

Hickman

B. J.

(2022). The implementation of Results-Driven Accountability: A systemic approach to large-scale initiatives. Journal of Education, 202(1), 3–14. https://doi.org/10.1177/0022057420943175

24.

Honig

M. I.

Venkateswaran

(2012). School–central office relationships in evidence use: Understanding evidence use as a systems problem. American Journal of Education, 118(2), 199–222. http://dx.doi.org/10.1086/663282

25.

IDEA Infant and Toddler Coordinators Association. (2016). 2016 ITCA tipping points survey part C implementation: State challenges and responses. http://ideainfanttoddler.org/pdf/2016-ITCA-State-Challenges-Report.pdf

26.

IDEA Infant and Toddler Coordinators Association. (2022). Why intervene early? https://www.ideainfanttoddler.org/about-early-intervention.php

27.

Individuals with Disabilities Education Act of 2004 (IDEA), 20 U.S.C. § 1400 (2004).

28.

Institute of Education Sciences. (2011, September 15). Grants for statewide, longitudinal data systems. http://ies.ed.gov/funding/pdf/2012_84372.pdf

29.

Jimerson

J. B.

Garry

Poortman

C. L.

Schildkamp

(2021). Implementation of a collaborative data use model in a United States context. Studies in Educational Evaluation, 69, 100866. https://doi.org/10.1016/j.stueduc.2020.100866

30.

Keeping Children and Families Safe Act of 2003, Pub. L. No. 108-36, 42 U.S.C. § 5101 et seq. (2004). https://www.govinfo.gov/content/pkg/PLAW-108publ36/pdf/PLAW-108publ36.pdf

31.

Lincoln

Y. S.

Guba

E. G.

(1985). Naturalistic inquiry. Sage.

32.

Mandinach

E. B.

Jackson

(2012). Transforming teaching and learning through data-driven decision making. Corwin.

33.

Mandinach

E. B.

Schildkamp

(2021). Misconceptions about data-based decision making in education: An exploration of the literature. Studies in Educational Evaluation, 69, 100842. https://doi.org/10.1016/j.stueduc.2020.100842

34.

Maxwell

J. A.

(2013). Qualitative research design: An interactive approach (3rd ed.). Sage.

35.

Means

Padilla

Gallagher

(2012). Use of education data at the local level: From accountability to instructional improvement. Department of Education, Office of Planning, Evaluation, and Policy Development.

36.

Miles

M. B.

Huberman

A. M.

Saldaña

(2014). Qualitative data analysis (3rd ed.). Sage.

37.

Morrison

Coffey

Peyton

(2021). Federal investment in early childhood data systems. ECDataWorks. https://repository.upenn.edu/entities/publication/5ef8c0b1-e1a8-42c9-ac6b-5c74e5dffc4f

38.

National Forum on Education Statistics. (2011). Traveling through time: The forum guide to longitudinal data system. Book three of four: Effectively managing LDS data (NFES 2011–805). U.S. Department of Education. http://nces.ed.gov/forum/pub_2011805.asp

39.

No Child Left Behind Act of 2001, Pub. L. No. 107-110, §115, Stat. 1425 (2002). https://www.govinfo.gov/content/pkg/PLAW-107publ110/pdf/PLAW-107publ110.pdf

40.

Office of Special Education Programs, U.S. Department of Education. (2012). Results-Driven accountability in special education summary. https://www2.ed.gov/about/offices/list/osers/osep/rda-summary.pdf

41.

Office of Special Education Programs, U.S. Department of Education. (2014a). Slides to explain Results Driven Accountability. http://www2.ed.gov/about/offices/list/osers/osep/rda/index.html

42.

Office of Special Education Programs, U.S. Department of Education. (2014b, June 24). New accountability framework raises the bar for state special education programs. http://www.ed.gov/news/press-releases/new-accountability-framework-raises-bar-state-special-education-programs

43.

Office of Special Education Programs, U.S. Department of Education. (2017). OSEP child count data-number of infants and toddlers ages birth through 2 and 3 and older, and percentage of population, receiving early intervention services under IDEA, Part C, by age and state [Data file]. https://www2.ed.gov/programs/osepidea/618-data/static-tables/index.html

44.

Park

Datnow

(2009). Co-constructing distributed leadership: District and school connections in data-driven decision-making. School Leadership & Management, 29, 477–494. https://doi.org/10.1080/13632430903162541

45.

Patton

M. Q.

(2015). Qualitative research and evaluation methods: Integrating theory and Practice (4th ed.). Sage. https://us.sagepub.com/en-us/nam/qualitative-research-evaluation-methods/book232962

46.

Ryan

G. W.

Bernard

H. R.

(2003). Techniques to identify themes. Field Methods, 15(1), 85–109. https://doi.org/10.1177/1525822X02239569

47.

Saldaña

(2021). The coding manual for qualitative researchers (4th ed.). Sage.

48.

Silverman

(2005). Doing qualitative research (2nd ed.). Sage.

49.

Slavit

Nelson

T. H.

Deuel

(2013). Teacher groups’ conceptions and uses of student-learning data. Journal of Teacher Education, 64(1), 8–21. https://doi.org/10.1177/0022487112445517

50.

Stake

R. E.

(2006). Multiple case study analysis. Guilford Press.

51.

Thayer

Coffey

Hebbeler

Spiker

(2022). IDEA Part C and Part B 619 state data systems: Current status and changes from 2013 to 2019. SRI International.

52.

Torres

R. T.

Preskill

(2001). Evaluation and organizational learning: Past, present, and future. American Journal of Evaluation, 22(3), 387–395. https://doi.org/10.1016/S1098-2140(01)00170-9

53.

Van Dyke

M. K.

Naoom

S. F

. (2016). The critical role of state agencies in the age of evidence-based approaches: The challenge of new expectations. Journal of Evidence-Informed Social Work, 13(1), 45–58. https://doi.org/10.1080/15433714.2014.942021

54.

Yazejian

Bryant

(2013). Embedded, collaborative, comprehensive: One model of data utilization. Early Education & Development, 24(1), 68–70. https://doi.org/10.1080/10409289.2013.736128