Abstract
Policymakers, funders, and researchers today view research–practice partnerships (RPPs) as a promising approach for expanding the role of research in improving educational practice. Although studies in other fields provide evidence of the potential for RPPs, studies in education are few. This article provides a review of available evidence of the outcomes and dynamics of RPPs in education and related fields. It then outlines a research agenda for the study of RPPs that can guide funders’ investments and help developing partnerships succeed.
There is currently more talk than ever about the role research should play in improving educational practice. Three major task force reports have investigated the relationship between research and practice, calling for changes in business as usual (Donovan, Wigdor, & Snow, 2003; National Academy of Education, 1999; National Research Council, 2012). Policymakers, funders, and researchers are creating new mechanisms to ensure that research plays a stronger role in educational improvement, developing new resources for practitioners to increase access to research findings, introducing new funding streams that encourage researchers to focus on problems of practice, and pioneering new ways for researchers and practitioners to work together.
As part of this trend, there is a growing interest in research–practice partnerships (RPPs). RPPs are long-term collaborations between practitioners and researchers that are organized to investigate problems of practice and solutions for improving schools and school districts (Coburn, Penuel, & Geil, 2013). Advocates argue that RPPs can enable greater use of research in decision making (Tseng, 2012), address persistent problems of practice, and improve educational outcomes (Donovan, 2013; Fishman, Penuel, Allen, & Cheng, 2013). Perhaps because of the perceived success of high-profile RPPs, such as the University of Chicago Consortium for Chicago School Research, the Strategic Education Research Partnership (SERP), the Carnegie Foundation’s Networked Improvement Communities, and others, funders are beginning to invest resources to support the development of new RPPs. The most prominent entrant into this arena is the Institute for Education Sciences (IES). In the past 5 years, IES has developed new initiatives to fund the development of RPPs, support projects that foster cycles of continuous improvement in districts, and reconfigure the Regional Education Laboratories with some qualities of RPPs. 1 IES is not alone. Some universities and a myriad of local and national funders are also providing funding to support this new strategy. For example, in 2015, the Spencer Foundation ran its first competition to support established RPPs. 2
But what do we know about the degree to which RPPs foster research use and support educational improvement? How do researchers and practitioners actually work with one another in these projects? How does the design of partnership or particular strategies they use matter for the process and outcomes of RPPs? What are potential negative or unintended consequences of doing research in the context of RPPs—for the district, the researchers involved, or the field as a whole?
In this article, we review empirical research on RPPs in education and other fields. We argue that although there is evidence of success of the interventions developed within RPPs in other fields, research on the impact of RPPs in education is sparse and focused on a narrow range of outcomes. We also argue that the research on the dynamics of partnership—how RPPs actually work and the mechanisms by which they foster educational improvement—tends to focus on the challenges, providing little insight into how partnership designs and strategies used by participants can address these challenges. We argue that building a body of research on these questions is crucial given the current investment of scarce resources in this approach and the potential for assisting newly funded partnerships. We close by framing a research agenda for investigating RPPs as a strategy for improving U.S. public schools.
What Are RPPs?
The term partnership is used widely in U.S. education to refer to a broad range of arrangements between researchers and practitioners, such as consulting agreements, use of schools or districts as places to test university-developed innovations, and sites for teacher training and internships. RPPs are a very specific form of partnership. They are long term—rather than being focused on a single study, researchers and system leaders share an open-ended commitment to build and sustain a working collaboration over multiple projects. Rather than developing studies that address gaps in existing theory or research, they focus on problems of practice—key dilemmas and challenges that practitioners face. They are mutualistic—the focus of the work is jointly negotiated and there is shared authority. RPPs employ intentional strategies to foster partnerships, with carefully designed rules, roles, routines, and protocols that structure interaction. Finally, they involve original analysis of data—participants collect their own and sometimes use sophisticated analytic techniques to answer districts’ questions using administrative data (Coburn et al., 2013).
RPPs can involve different types of partners and focus on a wide variety of problems of practice. Some RPPs form between researchers and leaders in a single district; others form among researchers, schools, and other youth-serving agencies in a region. Still others comprise networks of geographically distributed institutions, such as community colleges or school districts. Some are initiated by researchers, others by practitioners (e.g., Cooper, 2007; Penuel, Coburn, & Gallagher, 2013), and still others by funders. RPPs can focus on doing independent analyses of district policies and their implementation, providing findings to educational decision makers, and working with them to develop solutions (e.g., Research Alliance for New York City Schools). They can involve researchers and practitioners codesigning and testing solutions for improving teaching and learning or orchestrating systemwide change (e.g., Middle-School Mathematics and the Institutional Setting of Teaching; Cobb, Jackson, Smith, Sorum, & Henrick, 2013). They can also involve systematic methods for fostering continuous improvement research among networks of schools, districts, or other institutions (e.g., Bryk, Gomez, Grunow, & LeMahieu, 2015). Some focus on in-school and out-of-school spaces and the connections between them (e.g., the John W. Gardner Center for Youth and Their Communities), whereas others focus on issues of policy and practice within public schools. Some involve university researchers (e.g., Stanford University/San Francisco Unified School District Partnership), whereas others involve researchers in intermediary organizations (SERP; Donovan, 2013). RPPs of all types are currently proliferating, fueled by increased interest from local and national funders but also by researchers’ and practitioners’ desire to find new models of research that make a difference in public schools in more tangible ways. But what do we know about when and under what conditions RPPs make a difference in schools and school districts?
What Do We Know About the Outcomes of RPPs?
Most research on the outcomes of RPPs in education and other fields has focused on the impact of interventions developed in the context of a partnership. Thus, they do not investigate the impact of the partnership itself or other outcomes of RPPs.
Our review of the literature detailed below suggests that many interventions developed in the context of partnerships have shown positive outcomes. While there are quite a few studies of the outcomes of RPP in education, many more come from fields as diverse as public health, mental health, and criminology. However, the research is relevant for education because the partnerships have features similar to those in education. They are long term, mutualistic, and focused on collaboratively defined problems of practice (King et al., 2010; Metzler et al., 2003). In addition, these fields involve professionals whose work, like the work of teachers, focuses on changing people’s thinking and behavior (Cohen, 2011).
In public health, RPPs can involve multiple organizations that serve community members working alongside researchers to develop and test interventions collaboratively (e.g., Bullock, Morris, & Atwell, 2011; Israel, Schulz, Parker, & Becker, 1998; Wallerstein & Duran, 2010). For example, in the Communities That Care (CTC) model, researchers work with multiagency collaboratives to assess needs and then select and implement evidence-based programs in primary prevention for adolescents to meet those needs. The model aims to build a culture of evidence-based decision making and commitment to primary prevention across a community.
Researchers tested whether the CTC model can help build such a culture and produce positive impacts on youth development outcomes in a large-scale, cross-community randomized controlled trial. They found that leaders in communities that formed partnerships with researchers to implement CTC were more likely to devote resources to primary prevention than in comparison communities (Brown, Hawkins, Arthur, Briney, & Fagan, 2011). They also found positive impacts on youth in CTC, documenting lower levels of alcohol and cigarette use and fewer delinquent behaviors compared to youth in control communities (Hawkins et al., 2008, 2009).
In the field of mental health, one RPP composed of social service providers, parents, and researchers developed and tested interventions that targeted behavior of elementary school–age youth (Garland, Plemmons, & Koontz, 2006; Horsfall, Cleary, & Hunt, 2011; Kazak et al., 2010). Researchers studied the impact of an intervention called Multiple Family Groups that sought to reduce parenting stress and children’s oppositional behavior. The protocols were iteratively refined over multiple sessions based on input from stakeholders and evidence from pilot studies (see McKay et al., 2010, for descriptions). A random-assignment study found the protocol for the intervention was successful in reducing parent reports of children’s oppositional behavior and reducing parents’ stress (McKay et al., 2011). Another partnership formed to support care for infants and toddlers at risk for autism spectrum disorders found that the collaboratively designed efforts to enhance evidence-based practice resulted in improvements in child communication and engagement and enhanced parent skills to support children’s development (Brookman-Frazee, Stahmer, Lewis, Feder, & Reed, 2012).
In criminology, there have been multiple RPPs focused on community policing, in part due to strong federal support through such programs as the National Institute of Justice’s Building and Enhancing Criminal Justice Researcher–Practitioner Partnerships initiative. 3 One partnership that has been studied extensively focuses on reducing gang violence in Boston. Researchers facilitated the use of data by the police department to define the scope of the problem, helped develop response efforts, and evaluated those efforts (Braga, Kennedy, & Tita, 2002). They found that this approach reduced overall levels of youth violence, including when compared to trends in similar cities (Braga, Kennedy, Waring, & Piehl, 2001).
Research on the efficacy of interventions developed by RPPs in education also has, by and large, found promising impacts on student learning (e.g., Barab, Gresalfi, & Ingram-Goble, 2010; Booth et al., 2015; Fishman, Marx, Best, & Tal, 2003; Geier et al., 2008; Penuel et al., in press; Snow, Lawrence, & White, 2009; Yarnall, Shechtman, & Penuel, 2006). For example, Word Generation, an intervention developed as part of SERP that targets middle school students’ academic language, has strong evidence of efficacy from a randomized controlled trial (Snow et al., 2009). A recent collaboration between SERP and district members of the Minority Student Achievement Network developed and tested an intervention in algebra. A random-assignment study found the intervention to be successful in improving students’ conceptual understanding in mathematics (Booth et al., 2015). Comparison group and randomized controlled studies of innovative science curriculum units developed by RPPs have also shown promise for improving student learning on standardized tests (e.g., Geier et al., 2008) and researcher-designed measures (e.g., Barab et al., 2010; Penuel et al., in press). Other research has documented positive effects of interventions developed by partnerships on teaching and assessment practice (e.g., Yarnall et al., 2006) and on curriculum implementation (e.g., Fishman et al., 2003).
However, there are myriad interventions developed in the context of RPPs in education and other fields that have not been subject to systematic inquiry. Furthermore, we do not know whether these results would be different if the intervention was developed outside the context of a RPP. And, importantly, these studies do not address the value of the partnerships themselves, above and beyond the particular innovations they produce.
There are a handful of studies that investigate the influence of the partnerships themselves. Several studies provide evidence that participation in partnerships is associated with greater access to research (Bickel & Cooley, 1985; Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006). However, there is mixed evidence about whether participation in partnerships is associated with increased use of this research for making decisions, with some studies showing extensive use (Allensworth, 2015), others showing limited use (Coburn, Toure, & Yamashita, 2009), and still others showing that research use varied within and between districts (Honig, Venkateswaran, McNeil, & Twitchell, 2014; Hubbard, 2010). In addition, with the exception of Honig and her colleagues (2014), these studies are single case studies of partnerships, making it difficult to investigate how different strategies used by researchers in the context of partnerships or different contexts influence use.
Many other potential outcomes of RPPs remain unexplored. Advocates argue that collaborative design in RPPs can lead to more usable interventions, which in turn can foster scale-up and sustainability (Anderson & Shattuck, 2012). Although some research reports on teachers’ perceptions of the usability of particular tools (e.g., Booth et al., 2015), we know of little systematic research on whether and how participation in codesign influences the degree to which districts adopt, scale, and sustain the innovations they help produce. Building the capacity of educational systems to engage in research-informed improvement efforts is another goal of RPPs (Bryk et al., 2015). But existing research has not investigated whether participation builds a deeper understanding of the research process or research findings, an appreciation for the value of research to inform decision making, or capacity to engage in research-informed practices and policies or use research as part of continuous improvement efforts. Finally, we know little about unintended outcomes of RPPs, for example, if efforts may sour practitioners’ views of the value of research if researchers produce findings that are contrary to their beliefs, fail to produce research in a timely manner, or have difficulty securing funding for the long term (see Cousins & Simon, 1996, on this last point).
What Do We Know About the Dynamics of RPPs?
There is a bit more research on the dynamics of partnerships: how they actually work and the mechanisms by which they foster educational improvement. The vast majority of this work is first-person reflections written by researchers (rarely practitioners) who are participants in partnerships (e.g., Roderick, Easton, & Sebring, 2007, but see Cooper, 2007, for an exception). These insider accounts often describe strategies that partnerships employ for organizing their work and learning from one another (e.g., Edwards, 2011). As such, they provide insight into the workings of RPPs that others may use to inform their work. However, these accounts do not derive from systematic research design, data collection, and analysis. They also typically involve retrospective analyses, making them subject to hindsight bias. Outside researchers may have a greater ability to probe, surface, and report on perspectives and experiences of the full range of participants. They are more likely to be able to study partnerships in real time, thus avoiding bias introduced in retrospective analysis. And they have a greater ability to engage in comparative research across multiple partnerships.
There is a small body of research conducted by outside researchers on RPPs. This work largely focuses on the challenges (e.g., Coburn, 2010; Firestone & Fisler, 2002; Heckman, 1988; Penuel, Roschelle, & Shechtman, 2007; Vozzo & McFadden, 2001). For example, existing research highlights the difficulties that researchers and practitioners have communicating because they lack a common language with which to talk about issues facing practitioners (e.g., Sirotnik & Goodlad, 1998). Other studies demonstrate how RPPs depend upon atypical norms of interaction and new roles and responsibilities that are often unfamiliar to both researchers and practitioners (Coburn, Bae, & Turner, 2008). Differences in expectations regarding norms, roles, and responsibilities can lead to confusion, uncertainty, and even conflict (Coburn et al., 2008; Rosen, 2010).
Studies of the dynamics of partnership have also documented a range of challenges stemming from the organizational realities of educational systems. For example, school districts have complex organizational structures, with multiple divisions and multiple levels. This structure tends to foster the development of different points of view as those in different parts of the district have different disciplinary backgrounds, work roles, and ways that they focus their attention (Coburn et al., 2009; Spillane, 1998). This raises questions about whom in the district one should partner with and how to coordinate across multiple goals and agendas that are present in the district (Coburn & Stein, 2010). Turnover is also endemic, especially at upper levels of the system. When key staff turn over, new relationships must be formed, trust rebuilt, and work routines and understandings established. Change in leadership may also come with new goals and priorities that require the direction of joint work to be renegotiated (Rosenquist, Henrick, & Smith, 2015).
Finally, research has highlighted the degree to which RPPs exist in highly politicized environments (Scott, Lubienski, DeBray, & Jabbar, 2014). Multiple interest groups inside and outside the district pressure district administrators to make particular decisions (Englert, Kean, & Scribner, 1977). This research suggests that RPPs do not exist apart from these pressures, and they must learn to navigate the complex and shifting political contexts of school districts.
Although research on RPPs from outside researchers identifies no shortage of challenges, we know little about the extent to which the strategies that participants document in their first-person accounts are effective at addressing challenges (D’Amico, 2010, is an exception). At present, there is little basis for recommending some partnership designs or particular strategies to address challenges over others. Furthermore, studies of RPPs in education and other fields tend to investigate either the dynamics of partnerships or their outcomes but not both. For this reason, we have little insight into the mechanisms by which RPPs foster the full range of intended and unintended outcomes.
Research Agenda for RPPs
In light of the rather large holes in the existing knowledge base on RPPs, we believe that it is time for a more focused research effort on the process and outcomes of RPPs. This research is necessary to assess the impact of not only funders’ investments but also the time that researchers and practitioners are investing in this work. It is also necessary to inform the growing number of researchers and practitioners who are involved in these partnerships, providing information about when and under what conditions different partnership strategies bear fruit. Furthermore, research in this realm could help researchers and practitioners new to this work learn the skills, strategies, roles, and identities that may be necessary to do it well. Here, we outline areas of focus that may help address these needs of policymakers and partnership participants.
Outcomes
We need to learn more about the consequences of RPPs. Many funders are making significant investments in partnerships with the assumption that partnerships can help improve outcomes for students. Research is needed to investigate whether this is true for a wider range of partnerships, for a wider range of innovations they create, and for partnership activities beyond codesign (e.g., research on district policies or practices, continuous improvement structures). Given long time frames, this task requires the development of appropriate metrics to judge the early progress of partnerships, well before student outcomes are likely to be affected.
We also need studies that investigate other outcomes of RPPs, including individual and organizational change, research use, and spread and scale of innovation. For example, studies of RPPs that focus on codesign should also focus on the impact of the design and development process on beliefs and practices of teachers, researchers, and district leaders who participate. Such studies are important because they can shed light on whether and how the process itself contributes to improved student outcomes through changes in beliefs and practices of adults who implement them. In addition, advocates argue that participation in RPPs fosters greater understanding of the research process and greater use of research by school and district leaders in their decision making. The small handful of existing studies find mixed results here. We need to better understand when and under what conditions RPPs foster research use and when they do not.
Crucially, we need studies that attend to unintended or negative outcomes. For example, critics of RPPs argue that the results of studies done in specific contexts are not generalizable (Kelly, 2004) or are focused on only a narrow range of important issues. They argue that although the research may inform a specific district, it may not contribute to educational improvement more broadly. Some (not all) RPPs do seek to design tools and conduct research that informs the wider field (e.g., Donovan et al., 2003). We need studies that investigate whether they accomplish these aims. Other critics express concern that researchers who work closely with the districts they study lose their objectivity (Anderson & Shattuck, 2012; Kelly, 2004) and that findings may not go through peer review, with the potential to lead districts and the field astray. We also need studies that investigate these claims as well as studies that are attentive to other negative or unintended outcomes. This information is crucial in understanding the degree to which investments in RPPs are bearing fruit.
The field would also benefit from studies that investigate failed partnerships. Existing research and writing largely focus on partnerships that are perceived to be successful. But many partnerships fail to get off the ground in the first place, and others fail to be sustained over the long term (Penuel et al., 2013). It is only by investigating multiple partnerships with different levels of performance that we will begin to understand when and under what conditions RPPs develop into productive partnerships, are sustained, and meet their goals. Findings from such studies can also inform criteria funders use to judge whether proposed partnerships have in place needed supports to launch joint work productively.
Comparative Studies
We need comparative studies that investigate how RPPs of different designs interact with their contexts to impact various outcomes of interest. RPPs vary quite a bit in their designs, providing a great opportunity to uncover how partnership design matters for the full range of relevant outcomes. For example, some partnerships emphasize maintaining researcher independence. 4 In these RPPs, researchers engage with practitioners at the front end to develop ideas for study and at the tail end when they have results (Roderick et al., 2007). By contrast, other RPPs involve codesign of interventions or ongoing researcher engagement in district problem solving. How do these varying strategies impact research use? It is likely that any partnership design involves trade-offs, and comparative studies of partnerships with similar aims but different designs could help to illuminate those trade-offs.
Targeted Studies of Strategies
We need targeted studies of specific strategies that partnerships use. Existing research tends to focus on the challenges, providing little insight into how tools, strategies, and routines used by participants address these challenges. Studies could investigate the relative strengths and weaknesses of strategies for addressing such common challenges as persistent turnover, the need to create shared language or work practices, fostering trust, and ways to work with and across multiple levels of educational systems. They could investigate if and how it matters if partnerships or specific lines of work are initiated by the district, by researchers, or by funders. This knowledge is important given increased funding initiatives that support new RPPs, bringing researchers and practitioners who have little experience in building or sustaining partnerships into the work. Furthermore, it is critical, if we are to move beyond the exhortation that building partnerships between researchers and practitioners is hard, to understand the relative efficacy of different approaches for addressing challenges.
Political Dimensions of Partnerships
Finally, we need studies that examine the political dimensions of partnerships. Many researchers believe that politics gets in the way of effective research use and hope that producing higher-quality research will assist districts in moving decisions out of the political realm (see, e.g., Haskins & Baron, 2011). Few studies of RPPs address these concerns. Given evidence that research is often born out of and feeds back into political struggles and debates in central ways (Henig, 2012; Lubienski, Scott, & DeBray, 2014), it is important that we understand the role that RPPs play in the political context of the district as well as strategies for navigating the politics such that partnerships can continue to help districts address their critical needs.
Conclusion
Given new investments in RPPs, we need parallel investments in research on the dynamics and outcomes of partnerships. Studies of partnership dynamics are crucial to enable new partnerships to learn how to do this work and for existing partnerships to learn from each other’s experiences. Research on an expanded set of outcomes for RPPs—negative as well as positive—can help funders and partnerships alike gain a better sense early on as to whether their partnerships are likely to lead to improvements in student achievement and other outcomes. With a broader evidence base in both the dynamics and outcomes of RPPs, we can develop a better sense of whether, when, and how RPPs are a viable and effective way for research to support broad and sustainable improvements to educational systems.