Abstract
The complexity of successfully implementing interdisciplinary care team approaches in primary care has challenged many delivery system stakeholders. One-year changes in clinicians’ and staff experiences of practice climate among 5 practices implementing interdisciplinary primary care teams and 28 other practices were compared. In adjusted analyses, practices implementing care teams reported improved team structure (78.0 in 2011 vs 79.3 in 2012), team functioning (75.7 vs 77.7), readiness for change (77.6 vs 77.7), and perceptions of skills and knowledge (48.0 vs 53.6) over time. However, the improvements were not significantly different from changes experienced by other practices. Achieving improvements in practice climate through care team redesign is challenging, even with structured learning opportunities for team members. Practice climate did not deteriorate over time, indicating that implementing a complex team redesign does not harm working relationships of frontline clinicians and staff.
Keywords
More than a decade ago, the Institute of Medicine identified interdisciplinary care teams as a core component of the 21st century’s New Health System. 1 Integrating interdisciplinary care teams into routine primary care practice holds promise for delivering integrated and well-coordinated care to patients with chronic illnesses and is an essential strategy for transforming the US health care system. Team-based primary care has the potential to support patients in ways that individual physicians would not be capable of doing alone.2-7 In addition to improving patient health outcomes, implementing team-based approaches may have a positive spillover effect on practice climate, including team functioning and readiness for change. In spite of this, the complexity of successfully implementing team-based approaches in primary care has challenged many delivery system stakeholders.8,9 Integrating team-based approaches in routine primary care is a complex social change influenced by organizational, team, and patient factors, such as organizational support and features of the redesign, resources available to clinicians and staff, relationships among team members, and patient engagement.10,11 Compared with other types of delivery teams, interdisciplinary primary care team approaches can be especially challenging to integrate into routine care because clinicians and staff have difficulty developing the skills and knowledge to continuously manage a wide range of patient medical, behavioral, and social issues. 9
In 2011, 5 primary care practices belonging to a large integrated physician organization implemented a care team redesign intended to improve patient health outcomes and the quality of chronic illness care by augmenting their traditional team structure with the incorporation of 2 new team members at each of 5 practice sites: registered nurse care managers and patient health coaches. At the outset of implementing the care team redesign, clinicians and staff from all pilot sites received external facilitation, which included team trainings and workshops focused on implementing new care team member roles. This study examines changes in clinician and staff member experiences of practice climate, including team structure, team functioning, perceptions of team members’ skills and knowledge, readiness for organizational change, and leadership facilitation, among the 5 pilot practices compared with 28 other primary care practice sites in the geographic region. The primary objective in this study was to examine the extent to which implementing a team-based approach to primary care had a positive impact on practice climate over a year.
Although previous evidence suggests that implementing team-based approaches can result in higher-quality chronic disease management, the impact of team redesign on clinician and staff experiences of team functioning and organizational readiness for change remains unexplored. There are few longitudinal studies characterizing changes in clinicians’ and staff experiences of practice climate as they implement team-based approaches to care and fewer that assess the effect of a team redesign on changes in multiple dimensions of practice climate (eg, team functioning, leadership facilitation).2,12 Evidence suggests that team interventions hold potential to improve team relationships and workplace climate over time.13,14 Research also indicates that implementing a team redesign can be a disruptive change and that stakeholders may experience challenges in adapting to change and encounter deficiencies in team processes.14,15 For example, challenging workplace experiences can negatively affect professional satisfaction and increase physician burnout.16,17 Recognizing the organizational and social complexity of implementing a care team redesign, the research team posited that clinicians and staff from pilot practices would report more positive experiences of practice climate one year after implementation, with the strongest improvements in team structure and team functioning because the practice redesign emphasized teamwork improvement.
Methods
Practice Climate Survey
To assess primary care practice climate, the research team developed a survey instrument consisting of select questions from previously validated clinician and staff surveys, including the TeamSTEPPS Questionnaire, 18 the Team Diagnostic Survey (TDS), 19 the TransforMed Clinician and Staff Questionnaire, 20 and the Organizational Readiness for Change Assessment 21 (see online Appendix A, available at http://ajmq.sagepub.com/supplemental). The survey instrument was intended to assess important aspects of practice climate, including clinician and staff experiences of team structure, team functioning, perceptions of skills and knowledge of team members, readiness for organizational change, and leadership facilitation. Questions referenced current team experiences using a 5-point response scale ranging from strongly disagree to strongly agree. Based on the unweighted average of all item responses, scale scores were transformed to a scale ranging from 0 to 100 points, with higher scores indicating more favorable team experiences; a score of 0 represents strongly disagree, 25 represents disagree, 50 represents neither agree nor disagree, 75 represents agree, and 100 represents strongly agree. Scale scores were developed using the half-scale rule, 22 which requires respondents to respond to at least half of the items comprising the scale in order to qualify for a score calculation.
Although all the survey questions fielded were validated in previous research, the research team wanted to assess the extent to which the questions across instruments measured distinct versus unique constructs. The team conducted exploratory factor analyses, treating responses as categorical, using both promax (oblique) and varimax (orthogonal) rotations to explore simple versus complex structures. 23 The team determined the optimal number of factors based on magnitude of eigen values, shape of the scree plot, and the fit for each grouping. Items with redundant content, weak loadings on all factors (<0.40), or high loadings (>0.30) on 2 or more factors were dropped. The research team then performed confirmatory factor analysis in the confirmation sample. The team created scales based on the final factor solution, considered the variation explained by each factor, and concluded with 5 scales of practice climate: team structure, team functioning, staff readiness for change, perceptions of team members’ skills and knowledge, and leadership facilitation.
Team structure (α = .89) and team functioning (α = .81) scales consist of measures of situation monitoring, mutual support, and team structure questions from the Agency for Healthcare Research and Quality TeamSTEPPS Questionnaire. 18 Staff readiness for change (α = .84) consists of questions from the team task design subscale of the TDS 19 and the adaptive reserve scale from the TransforMed Clinician and Staff Questionnaire. 20 Respondents’ perceptions of their team members’ skills and knowledge (α = .76) was measured by questions from the TDS survey. Finally, the leadership facilitation (α = .94) scale consists of the leadership behavior and change culture subscale of the Organizational Readiness for Change Assessment instrument. 21 Appendix A includes a mapping of the survey questions to each of the 5 scales.
Survey Administration
The survey was administered twice, first in October 2011 and again in November 2012, to all primary care clinicians and staff, including physicians, nurses (registered nurses, licensed practical nurses, and vocational nurses), medical assistants, clerks/receptionists, and other staff (social workers, referral coordinators, and patient liaisons). Surveys were administered across 33 practices, 5 of which were pilot practice sites implementing the primary care team redesign. The leadership of the physician organization purposively selected the 5 pilot practices because they were thought to represent a range of implementation readiness based on feedback from the quality improvement oversight group. All clinicians and staff received a personal electronic invitation to voluntarily participate in the practice climate survey via the Internet. Organizational leadership allowed for a 20-minute block of time during a workday during the survey implementation period to provide clinicians and staff with uninterrupted time to complete the survey. In both waves of the survey, a second, third, and fourth survey invitation were sent to nonrespondents 2, 3, or 4 weeks after the initial invitation, respectively. Each data collection effort lasted 6 weeks.
Analysis
To assess whether differences in team characteristics between pilot and nonpilot practices existed at baseline, the research team stratified the survey responses by pilot status (pilot respondents [n = 56] vs nonpilot respondents [n = 274]) and used χ2 statistics to compare differences. To compare changes in experiences of practice climate over time among pilot sites and nonpilot practices, the team specified 3-level mixed-effects multilevel regression models (XTMIXED, STATA 11; StataCorp LP, College Station, Texas) predicting change over time on each of the practice climate scales. These models accounted for the clustering of observations within individual respondents over time and the clustering of respondents within practice sites using respondent and practice random effects. The research team chose a multilevel approach because it was important to use a method that accounted for regression to the mean. To examine the extent to which differences between pilots and nonpilots had different changes over time for each of the 5 practice climate scales, the regression models included terms to capture the magnitude and statistical significance (P < .05) of (1) overall differences between respondents from pilot and nonpilot sites, (2) changes over time, and (3) differences in changes over time between pilot and nonpilot sites.
To evaluate differences in changes over time by occupation across the practice sites, the research team used the predicted estimates from the multilevel regression analyses and stratified the results by job title, pilot status, and survey wave to compare differences between job titles for pilot versus nonpilot practice sites over time.
Finally, to clarify whether the changes in practice climate between respondents from pilot sites and nonpilot sites over time were related to higher response rates to the second wave of the survey, the research team conducted a sensitivity analysis restricting the respondent sample to those individuals present in both survey waves only (ie, dual respondents). Respondent-level covariates, including respondent age, occupation, and race/ethnicity, were included as control variables in regression models for both the main and sensitivity analyses.
Results
Respondent Characteristics
A total of 330 and 401 respondents completed surveys in 2011 and 2012, respectively, resulting in overall response rates of 51% and 70%. Of these, 10 responses were omitted because of missing data on key study variables. The overall analytic sample included 721 responses from 542 unique individuals, including responses from primary care physicians (n = 146), registered nurses (n = 68), medical assistants (n = 439), and clerks/receptionists (n = 68). Of the 542 unique responses, 188 were from dual respondents who completed surveys in both the 2011 and 2012 survey waves, including physicians (n = 31), registered nurses (n = 24), medical assistants (n = 117), and clerks/receptionists (n = 16).
Characteristics of respondents from the 5 pilot sites were similar to those of the respondents from the 28 nonpilot sites during the baseline year (2011; Table 1). However, compared with pilot sites, a higher proportion of nonpilot respondents reported shorter tenure with the practice at baseline, whereas pilot practice respondents were more likely than nonpilot respondents to report a high likelihood of leaving their practice within 2 years (83.6% vs 67.4%).
Baseline (2011) Respondent Characteristics, Pilot Versus Nonpilot Site Respondents.
Abbreviations: LPN, licensed practical nurse; LVN, licensed vocational nurse; PCP, primary care physician; RN, registered nurse; SD, standard deviation.
Differences in Practice Climate Changes Over Time by Pilot Status
In adjusted analyses, pilot site respondents reported better overall practice climate experiences compared with non-pilots on 4 of the 5 scales, which included team structure, team functioning, readiness for change, and leadership facilitation (Table 2). Pilot practice respondents reported similar or improved team structure, team functioning, readiness for change, and perceptions of skills and knowledge over time. Nonpilot site respondents reported worse team structure and readiness for change over time. Respondents from both pilot and nonpilot sites reported worse experiences of leadership facilitation over time. Despite the improved practice climate reported by pilot site respondents, none of the differences were statistically significant (P < .05) compared with changes among nonpilot respondents over time. Leadership facilitation was the only measure to result in differential changes between pilot and nonpilot sites; nonpilot practice respondents reported worse leadership facilitation over time than pilot practice respondents (a decline in leadership by 2.9 and 0.5 points, respectively). Online Appendix B (available at http://ajmq.sagepub.com/supplemental) depicts the changes in each of the core practice climate measures over time for pilot and nonpilot practices.
Differences Over Time between Pilot Versus Nonpilot Practices on Practice Climate Measures. a
All scales are scored on a 0 to 100 scale, and the higher scores are more favorable responses. Predicted means are adjusted for occupation, age, race, and ethnicity and account for the clustering of observations within respondents over time and respondents within sites.
Statistically significant (P < .05) difference over time between pilot sites and nonpilot practices.
Occupational Differences in Practice Change Over Time by Pilot Status
Changes over time in the practice climate measures were similar for primary care physicians, registered nurses, medical assistants, and clerks, and similar patterns were observed for pilot and nonpilot respondents. Among the occupations, registered nurses reported better experiences on the practice climate scales, except for leadership facilitation. Clerks reported better leadership facilitation experiences at both pilot and nonpilot practices (Table 3).
Changes in Practice Climate Over Time: Differences by Occupation and Pilot Practice Status. a
Abbreviations: MA, medical assistant; PCP, primary care physician; RN, registered nurse.
Predicted mean scores are from the adjusted regression analyses. Nurse respondents from nonpilot sites were largely nurse supervisors, whereas at pilot sites, nurse respondents were nurse care managers.
Sensitivity Analysis
The changes in practice climate scales restricted to those who responded to both surveys (2011 and 2012) are shown in Table 4. In general, restricting the sample to dual respondents attenuated the small improvements observed among pilot sites over time. A notable exception was for the “skills and knowledge” scale; pilot sites had a 6.0-point improvement over time relative to nonpilot sites, which had a 1.3-point decline.
Pilot Versus Nonpilot Changes Over Time on 5 Primary Care Practice Climate Measures, Respondents to Both Surveys (2011 and 2012) Only. a
Analyses are limited to dual 2011 and 2012 respondents only; n = 188.
P < .05 compares survey years between pilot sites and nonpilot sites.
Discussion
This study of an integrated health care organization’s primary care team redesign among 5 pilot practices underscores that the integration of new team members and providing implementation support does not always translate into improvements in practice climate. The research team found that pilot and nonpilot clinicians and staff had fairly stable experiences of team structure, team functioning, skills and knowledge, and readiness for change over time. It is important to note that the sensitivity analysis restricting the comparisons to respondents to both survey waves highlights that team members’ skills and knowledge improved more among pilot practices than nonpilot practices. Medical home implementation research suggests that in attempting to implement a practice change as complex as fully integrating new team members, clinicians and staff run the risk of experiencing burnout and an erosion of practice climate.24,25 Recognizing that effective care team redesign is a complex organizational change, stable experiences of practice climate reported by frontline clinicians and staff is noteworthy.
The lack of change over time observed in this study also suggests that a practice’s ability to achieve improvements in experiences of care team integration may require an adjustment in concrete aspects of practice climate, including addressing perceptions of team members’ skills and knowledge and enhancing leadership facilitation of changes. Moreover, experiences of leadership facilitation worsened over time among respondents at both pilot and nonpilot sites. Previous research indicates that leadership facilitation is key to influencing improvements in care team functioning26,27 because organizational leadership provides a compelling direction, ensures role clarity, and facilitates transition from one care team model to another. Key informant interviews with care team members highlight differences in the implementation of core team redesign components across the pilot practices. 28 The practice-level samples are too small to reliably examine practice climate differences between pilot sites, 29 but implementation differences between practices might explain the lack of an overall team redesign effect on practice climate. Changing long-standing practice patterns likely takes longer than one year, so measurable and sustained improvements in practice climate could occur in the long run. To continue improving primary care team functioning, practice leaders may need to establish clear improvement goals and monitor teamwork experiences as part of the redesign and ensure active participation by all members, support innovation among the teams, and improve readiness for organizational change. 26
There are limitations to this study. First, response rates for waves 1 and 2 were 51% and 70%, respectively. Nonresponse may bias the findings because different respondents are represented in each survey wave. However, the analysis over time linked individual responses over time and controlled for certain factors such as job title, age, and race and ethnicity, which have been shown to be associated with job satisfaction, team performance, and team effectiveness.30,31 Moreover, results from the sensitivity analyses, which restricted results to dual respondents, were consistent with the unrestricted findings. Second, the pilot sites had relatively high baseline practice climate, and the changes over time experienced by the practices may not generalize to practices with low baseline practice climate. However, the fact that practice climate did not improve over time for practices with relatively higher functioning underscores the difficulty of improving team functioning and practice climate in a measurable way over a year. In spite of resources for redesign, even practices with high readiness for change face many challenges in improving practice climate when implementing a team redesign. Finally, the survey assessed the experiences and perceptions of clinicians and staff at the onset of the care team redesign and again one year later. Although one year may be insufficient to detect practice climate changes, a year reflects a time frame after which many leaders expect to realize improvement in the functioning of work units, given the amount of time and resources needed to integrate new care team members and the investment in teamwork training.
Conclusion
Primary care practices are increasingly adopting interdisciplinary care teams as part of the patient-centered medical home model to help improve the quality of chronic illness care, patient self-management, and health outcomes. Transforming primary care teams to support patient-centered chronic illness care can be a difficult change for practice stakeholders, and implementing these changes does not necessarily translate into more functional practice environments. Galvanizing improvement from a care team redesign requires more intensive leadership support and facilitation for all primary care team members. Overall, leadership facilitation tended to get worse over time for both pilot and nonpilot sites, highlighting the increasing challenges primary care practice leaders face in effectively facilitating change. Redesigning care teams without effective leadership facilitation may present challenges to frontline teams as they attempt to integrate team-based approaches into routine care. Efforts by practice and organizational leadership to continually support teams by establishing clinical performance goals and benchmarks that engender more teamwork and assessment of team performance 32 may be needed to accelerate meaningful improvements in primary care practice climate. Interdisciplinary care teams are widely considered an important strategy to efficiently meet the increased demand for primary care through more patient-centered chronic illness care and preventive care. The results of this study highlight that practice climate did not deteriorate as a result of implementing a complex care team redesign. This is noteworthy and suggests that although practice change is challenging, clinicians and staff are resilient and may embrace opportunities to improve the impact of team redesign efforts.
Footnotes
Acknowledgements
The research was reviewed and approved by University of California, Los Angeles South General Campus Institutional Review Board (IRB#11-002347).
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
