Abstract
Successful quality improvement (QI) requires a supportive context. The goal was to determine whether a structured curriculum could help QI teams improve the context supporting their QI work. An exploratory field study was conducted of 43 teams participating in a neonatal intensive care unit QI collaborative. Using a curriculum based on the Model for Understanding Success in Quality, teams identified gaps in their context and tested interventions to modify context. Surveys and self-reflective journals were analyzed to understand how teams developed changes to modify context. More than half (55%) targeted contextual improvements within the microsystem, focusing on motivation and culture. “Information sharing” interventions to communicate information about the project as a strategy to engage more staff were the most common interventions tested. Further study is needed to determine if efforts to modify context consistently lead to greater outcome improvements.
Keywords
Successful quality improvement (QI) requires the right evidence-based change content and rigorous use of QI methods to test, adapt, and implement changes; however, it is also dependent on a context that supports improvement.1-4 In spite of the important role of context in supporting QI, context has rarely been examined systematically in the setting of QI initiatives and even fewer QI efforts have explicitly addressed ways to make context more supportive.5-9
To determine whether a curriculum focused on context could help QI teams examine the context supporting their QI work, the research team partnered with the Vermont Oxford Network (VON) and teams from those hospitals participating in VON’s Eighth Newborn Intensive Collaborative for Quality (NICQ 8). 10 A curriculum focused on context was introduced to help NICQ 8 teams assess their context and design interventions with the potential to improve context. The team hypothesized that QI teams participating in the VON NICQ 8 collaborative could successfully utilize a guided curriculum to identify gaps and design improvements in QI context.
Methods
Overview
This article first describes the study design and the collaborative structure in which the study was conducted. The methods used by collaborative QI teams to apply the Model for Understanding Success in Quality (MUSIQ) model are then presented followed by an explanation of the methods to evaluate how MUSIQ was used by QI teams. Next, the results describing how QI teams assessed their context and designed interventions with the potential to improve context are presented. Based on these results, the research team provides recommendations for how teams might address context to better support their future QI efforts.
Study Design
This was a prospective exploratory field study 11 of QI teams participating in the VON NICQ 8 improvement collaborative (January to October 2013). The study protocol was reviewed by the Cincinnati Children’s Hospital Medical Center Institutional Review Board and was determined to be non-human subjects research.
VON NICQ Collaborative Structure
VON is a not-for-profit voluntary collaborative of neonatal intensive care units (NICUs), interdisciplinary health care providers, and families partnering to improve the quality, safety, and value of newborn care. 12 The network includes more than 950 NICUs from around the world that submit data on the care and outcomes of high-risk newborns to a robust registry. Network participants also have the option of participating in face-to-face (NICQ) or Internet-based intensive collaboratives. Since 1995, teams from more than 550 NICUs have participated in one or more of these collaboratives.
VON NICQ 8 was an intensive, face-to-face QI collaborative that continued the focus on clinical and value improvement that began during the previous 2-year collaborative (NICQ 7), but also introduced a specific focus on standardization of processes and understanding context. Forty-three QI teams representing different hospitals in the United States participated (see Supplement 1, available online). Hospitals worked in multidisciplinary QI teams charged with identifying, testing, and implementing evidence-based, potentially better practices to achieve improvements in quality, safety, and value. Each QI team included physicians, nurses, other health professionals, and parent advisors. 13 Teams were organized into “homerooms” to facilitate collaboration on similar clinical topics. 13 Participants met face to face at a homeroom-specific meeting (called the ONSITE meeting) between February and May 2013 and at a collaborative-wide meeting in October 2013.
NICQ 8 MUSIQ Curriculum
In addition to focusing on making improvements in clinical care and outcomes, NICQ 8 teams were guided through a curriculum designed to help them modify the context of their organization, microsystem (NICU), and QI team so that it would better support their QI projects. The curriculum was developed using MUSIQ—a model designed to assist organizations and QI researchers in systematically examining those contextual factors that are believed to influence the success of a QI project.14,15 MUSIQ identifies 25 contextual factors that are likely to affect the success of QI initiatives and organizes these factors based on the level of the health care system in which they function. 14 The goals of the MUSIQ curriculum were to (1) assist teams in evaluating their context for supportiveness; (2) help them identify the aspects of context that may serve as barriers in the success of their project; and (3) support them in generating and testing ideas to improve their QI context. 2
Participants were initially introduced to MUSIQ via an online presentation in January 2013. Teams were then given a preliminary version of a Web-based survey designed to quantitatively assess contextual factors in MUSIQ (February 2013). The research team provided QI teams with their survey results (March 2013), which included simple summary statistics based on results from all team members who responded. The main purpose of the survey was to generate discussion and help teams develop a shared understanding of context and opportunities for improvement. A workbook was provided to help guide examination of their survey results and stimulate conversation. Each team was asked to select between one and 3 contextual factors that it intended to address during the course of NICQ 8. Teams were then given journals (see Supplement 2, available online) to turn their knowledge about gaps in their local QI context into changes designed to modify these contextual factors. Teams were asked to complete a section of the journal at each of 3 points: (1) after reviewing their MUSIQ survey results and meeting with their QI team (prework section), (2) after meeting with the other homeroom teams at the in-person ONSITE (ONSITE section), and (3) prior to the final collaborative-wide meeting (action period section).
The design of the journal was based on an adaptation of the Appreciative Inquiry model.16,17 Teams were encouraged not only to identify gaps but also to envision what the best possible future would look like if this aspect of context was truly supportive. Journals also encouraged teams to compare their actual state to the ideal in order to develop a tactical plan to modify their context so that it more closely resembles the ideal state.18,19 Teams were asked to use the Model for Improvement and Plan-Do-Study-Act (PDSA) cycles to develop and test interventions designed to make their context more supportive. 2 The final section of the journal asked teams to reflect on whether and how context had changed as a result of testing and implementing their interventions.
At the request of the VON leadership team, participants were asked to submit their journals in July 2013 so that information could be synthesized and presented at the collaborative-wide meeting in October 2013.
Evaluation of the NICQ 8 MUSIQ Curriculum: Data Sources and Analysis
The research team used data from the MUSIQ survey and journals to understand how teams identified areas of context to target for improvement, how they planned and designed PDSAs to modify context, and what they learned from completed PDSAs.
MUSIQ Survey
Data from the MUSIQ survey were used to understand how the results of the preliminary MUSIQ survey informed teams’ efforts to modify context during NICQ 8. Median scores for each contextual factor on the MUSIQ survey (across all teams in the VON collaborative) were reviewed in relation to contextual factors selected as targets of interventions.
Journals
Journals submitted by participating QI teams were used to assess how teams developed ideas for change and worked toward modifying context. Each team submitted one journal for each contextual factor selected. Free text information captured included (1) a description of the context gap (eg, why this aspect of context is weaker than desired) and whether that gap was determined via observations and/or MUSIQ survey results; (2) a description of the team’s vision of a best imagined future; (3) a list of changes designed to modify context that were either planned or implemented; and (4) a description of any results achieved or observations about what was learned from PDSAs. Qualitative data analysis included abstraction of key data elements from the journals (including supporting quotes) plus classification and categorization of the types of changes reported by QI teams.
Codes to describe interventions were developed by one of the researchers (HG) after reviewing 26 journals for commonalities among the types of interventions utilized. These codes were triangulated with the QI and general management literature, where appropriate, or were generated using coding analysis based on journal text. 20 The resulting coding framework provided a typology organizing the interventions using 11 categories. The final coding framework was reviewed and agreed upon by the researchers.
Journals were evaluated by 2 of the researchers (HG and HK) using a standard data collection format. Researchers abstracted data on the key elements and categorized each intervention based on the coding framework. After independently completing data abstraction, the 2 researchers reviewed the data abstraction forms together, discussed the interventions and categories assigned to each intervention, and identified meaningful quotes. Differences were discussed and consensus achieved.
Results
Teams actively participated in the NICQ 8 MUSIQ curriculum by completing the MUSIQ survey and journals (Table 1). At least one member from every QI team completed the MUSIQ survey. Of the 364 participants identified for the survey, 210 (58%) responded. The median number of responses per team was 4 (interquartile range 3-5). Forty teams returned all or a portion of at least one journal. Teams submitted a total of 66 journals, each equating to a contextual factor the team decided to address. A summary of the organizational levels targeted and contextual factors examined is provided in Table 1.
Team Participation and Journal Responses.
Abbreviations: MUSIQ, Model for Understanding Success in Quality; NICQ, Newborn Intensive Collaborative for Quality; QI, quality improvement.
Each journal represents a contextual factor teams selected for focus. Teams were able to address and submit more than one contextual factor.
Name of the specific contextual factor targeted as an area of focus for NICQ 8 MUSIQ Curriculum was missing in n = 3 journals.
Teams used a combination of quantitative survey results and self-reflection to identify specific contextual factors to address. In nearly half of the journals, teams explicitly reported using the MUSIQ survey results to select a contextual factor to address (Table 1). The relationship between the specific factors most frequently selected by teams and the VON aggregate scores on the MUSIQ survey is summarized in Figure 1. Of the factors selected by teams as a focus of improvement, 60% were among the lower-scoring factors on the MUSIQ survey.

Scatterplot of contextual factors selected compared to Vermont Oxford Network (VON) factor scores.
In addition to the MUSIQ survey, teams used the self-reflective journal to reflect on weak areas of their context. Table 2 summarizes representative evidence of the gaps that drove teams to select specific contextual factors. After identifying gaps, teams were encouraged to envision their ideal context as a first step in developing an action plan to modify that aspect of context. For example, one team focused on changing microsystem QI culture envisioned “Having the staff understand and be able to speak about our [NICU] QI efforts and relate to the family about our improvements in clinical outcomes.” Another team working to make their QI team more effective described an ideal state where “. . . a few individuals . . . would be assigned as leaders [and] these leaders would work together . . . to communicate and formulate plans before initiating new practices.”
Top 5 Contextual Factors Identified by Teams as Targets for Improvement.
Abbreviations: MUSIQ, Model for Understanding Success in Quality; NICU, neonatal intensive care unit; PDSA, plan-do-study-act; QI, quality improvement.
After identifying target contextual factors, teams planned and tested interventions to modify these areas of context. There were 158 interventions planned and/or executed with an average of 2.4 interventions per contextual factor. Interventions were classified into 11 categories. The most common types of interventions tested and implemented by teams are summarized in Figure 2. Examples of specific interventions in the most common intervention categories are listed in Table 3. “Information Sharing” was the most common intervention category as teams planned and/or tested a variety of interventions designed to share information about the QI project as a strategy to engage people at varying levels in their improvement efforts.

Intervention category frequencies.
Most Common Intervention Categories and Example Interventions.
Abbreviations: IT, information technology; NICQ, Newborn Intensive Collaborative for Quality; NICU, neonatal intensive care unit; PDSA, plan-do-study-act; QI, quality improvement; VON = Vermont Oxford Network.
Boundary Spanning at the team level involves connecting to people outside the team for various purposes including directions such as vertical boundary spanning to leadership for support and resources, as well as horizontal boundary spanning for collaboration, information, and project coordination. 26
Group Structure Clarity was modeled on a factor of team effectiveness where group structure represents aspects such as role and goal clarity, work norms, and task control. 27
Results of the PDSA cycles associated with these interventions were reported in 34 (51.5%) journals. The vast majority of the results reported were qualitative. Approximately two thirds (64.7%) of the results reported by teams indicated that they were able to see some evidence of a positive impact from implementing their improvement strategy. For example, a team pursuing improvements in Data Infrastructure reported that the “. . . availability of pertinent patient data [has helped] us identify deviations in our standardization work [which has] been extremely helpful and has provided us with the ability to review data in a meaningful way focusing on our goals.”
Nearly one quarter (23.5%) of the results reported by teams suggested that the impact of the intervention tested was either inconclusive or it was too soon to tell whether the change had the intended impact. In conducting interventions to improve microsystem QI culture, a team noted that “early feedback suggests that interventions are beginning to create open conversations about teamwork and collegiality,” but they had yet to complete a planned follow-up survey. As might be expected, some results (11.8%) indicated that changes tested by teams showed no impact on context (eg, “fabulous failures”). A team focused on engaging senior leaders tested a range of interventions that were not successful:
We anticipated that getting senior leadership to the meetings was going to be difficult; they have busy schedules. We thought sending meeting minutes would be helpful in getting them up to speed . . . that has not [been helpful]; one would have to read the minutes to get up to speed.
Discussion
Contexts that do not support change contribute to the slow pace of health care improvements and innovations. 9 Therefore, efforts to make context more supportive can be seen as a means of speeding up and spreading improvement. 7 The results of this study suggest that a curriculum focused on context can help collaborative QI teams improve the context supporting their QI work. Using the MUSIQ curriculum, QI teams generally were able to identify the organizational levels and factors where they could focus their contextual improvement opportunities. Because NICQ 8 teams are microsystem-based teams, it was not surprising that the NICU microsystem was identified most frequently (55%) as the level of focus for improvement, as factors within the microsystem are more readily under the QI team’s control. Teams were able to develop and test more than 158 specific interventions they predicted would help them create a more supportive context. Some teams were able to see early signs of changes in their context, but the short project cycle in the NICQ 8 collaborative prohibited many teams from seeing the full impact of their efforts.
As part of the NICQ 8 MUSIQ curriculum, teams relied on both the MUSIQ survey results and reflective observations to develop contextual interventions. Similar to what was described by Shaw et al, the present study found that team-based reflection is a useful strategy to facilitate change in the setting of QI implementation. 21 In this case, teams’ self-reflection was enhanced by a theoretically informed model and quantitative information from the MUSIQ survey, both of which were used to help participants reflect on observed experiences and provide clarity about perceived opportunities. Similarly, teams’ efforts to modify context were enhanced by using the same methods—Model for Improvement and PDSA cycles—to test and implement changes designed to improve clinical care and also make context more supportive. MUSIQ was uniquely suited to support this curriculum because it delineates direct and indirect relationships among contextual factors that can help QI teams make explicit predictions as part of their PDSAs.
Despite the promise of more effective and faster improvements if QI efforts focus on ensuring that local context supports QI, few improvement initiatives systematically address the role of context. One exception is a recent QI effort to transform adult primary care practices described by Tomoaia-Cotisel et al. 22 As part of this QI effort, research teams were charged with retrospectively identifying the most important contextual factors in their studies of primary care QI and developing a template to assist in reporting relevant contextual factors in future work. Although there are isolated examples of systematic efforts to examine context in QI efforts, to the research team’s knowledge, this study is the first to help teams not only understand the role of context in their QI work but also to use QI methods to make their local context more supportive.
There were limitations that should be considered for a complete view of this study. Because this study occurred within the setting of an active QI collaborative, the research team was limited in the amount of data that could be requested from teams so as to limit burden. It is possible that work related to modifying context was being conducted at sites, but that work is not fully captured in the documentation made available to the research team. If anything, these methods are likely to underestimate the extent of the context-focused activities occurring at NICQ 8 sites. In addition, the data obtained in this study were subjective in nature. The research team relied on QI team self-reports because the goal was to understand how QI teams derived meaning from the systematic exploration of context and how this was translated into efforts to modify context. 23 The research team’s ability to understand the changes made and the impact of those changes was enhanced by analyzing these actions through the teams’ lens; however, this approach may have introduced bias if teams who are invested in seeing improvement overreport the positive impact of their changes. The research team attempted to limit the impact of this potential bias by applying systematic guidelines to analyze the teams’ self-reports of their experiences. 24 Furthermore, the fact that more than one third of QI teams reported that the interventions they tested led to either equivocal or no impact on context suggests that teams were honest and impartial in their reporting. Previous studies have examined the validity of MUSIQ for other purposes.14,15 Future work to examine the application of MUSIQ in practice may wish to supplement the methods used here with objective data that can be obtained from QI team activities or by the use of independent examiners. Although the potential for bias exists in any study that uses subjective or self-report data, this potential for bias does not diminish the recommendation for QI teams to employ a context assessment and management framework, such as MUSIQ, as part of their QI efforts.
The NICQ 8 collaborative focused not only on improving QI context but also on achieving clinical outcome improvement and standardizing care processes. Challenges with balancing the MUSIQ curriculum with these other goals meant that, by the time teams received their survey results, they had 5 months to assess their contextual gaps, plan and test changes to modify context, and complete all 3 sections of the journal. In addition, they had to balance work directed at modifying context with the work focused on improving processes and systems of care. By the time teams began to fully understand context and how to improve it, they were at the end of the collaborative. Although the research team did not expect most elements of context to change in the brief time available to the teams (changing organizational/unit culture can take years 25 ), it is encouraging that teams could successfully identify opportunities for improving context, design and implement changes to modify context, and see early evidence of positive impact. Future efforts to modify context within a QI collaborative should plan a longer time horizon to be able to measure changes in context and examine whether improvements in context truly lead to more effective improvement.
In light of the challenges associated with QI implementation and the variable results of QI efforts, addressing context to better support improvement is critical. This experience with 43 NICU QI teams suggests that using a structured MUSIQ curriculum in the setting of a QI collaborative can help teams systematically address the role of context in the success of their QI projects and begin to make changes designed to modify the aspects of context most directly under their control. By testing the use of a structured MUSIQ curriculum in partnership with the VON NICQ 8 collaborative, the research team was able to assess the generalizability of this approach across organizations; however, the true reproducibility of this approach to examining and managing context within improvement efforts will ultimately be seen when it is used in other settings as part of a comprehensive QI approach.
How could other improvement programs benefit from this approach? QI teams that want to speed improvement should slow down long enough to consider context, systematically address opportunities, and then move forward in a more receptive environment that is better able to support changing the outcome for patients and their families. Although collaborative QI efforts typically include educational curricula to teach QI skills (eg, Model for Improvement, PDSA methods), groups and individuals leading QI collaboratives should consider a more diverse meta-improvement strategy that also includes training on context and a means to evaluate current state through informed reflection and/or formal surveys. Even if interventions to modify context are not attempted or if results of efforts to modify context are not realized in the short term, a systematic examination of context will support the QI team’s learning about potential barriers to improvement. Based on the results of this study, the research team believes the MUSIQ curriculum and its associated tools could be used by QI teams and collaborative QI leaders in order to assist them in evaluating context through a systems lens and creating interventions to make context more supportive of QI.
Footnotes
Acknowledgements
We would like to thank all of the quality improvement team members, faculty, and QI facilitators that participated in the VON NICQ 8 QI Collaborative (see Supplement 1, available online). We would also like to thank Kathy Leahy, RN, NNP, Jeffrey Horbar, MD, and Madge Buus-Frank, DNP, APRN-BC, for their willingness to allow us to partner with VON and for their help in coordinating MUSIQ-related activities within NICQ 8. We would also like to thank Justin Bates, MA, for analyzing MUSIQ survey data to create reports for teams; April Mack-Williams for assisting with creating materials and coordinating communication with teams; and Daniel McLinden, EdD, for support in designing the MUSIQ journal and workbook.
Declaration of Conflicting Interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr Kaplan and Mr Handyside served as paid consultants of the Vermont Oxford Network. Mr Handyside served as the improvement leader for the NICQ 8 collaborative. Dr Kaplan was compensated for her time spent developing and implementing the MUSIQ curriculum in NICQ 8. The other authors declare that they have no financial relationships relevant to this article to disclose.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
