Abstract
Development of quality improvement (QI) skills and leadership for busy clinician-educators in academic medical centers is increasingly necessary, although it is challenging given limited resources. In response, the authors developed the Quality Scholars program for primary care teaching faculty. They conducted a needs assessment, evaluated existing internal and national resources, and developed a 9-month, 20-session project-based curriculum that combines didactic and hands-on techniques with facilitated project discussion. They also conducted pre–post tests of knowledge and attitudes, and evaluations of each session, scholars’ projects, and program sustainability and costs. In all, 10 scholars from all 3 generalist disciplines comprised the first class. A wide spectrum of previous experiences enhanced collaboration. QI knowledge increased slightly, and reported self-readiness to lead QI projects increased markedly. Protected time for project work and group discussion of QI topics was seen as essential. All 10 scholars completed projects and presented results. Institutional leadership agreed to sustain the program using institutional funds.
Since To Err is Human and Crossing the Quality Chasm were published by the Institute of Medicine a decade ago,1,2 quality improvement (QI) efforts within health care systems have increased dramatically. Physicians, administrators, and staff have been asked to collaborate in ways that many have never experienced before. Because physicians and other senior clinicians typically have been the leaders of most health care teams, their engagement is critical to the success of projects.
Physicians are typically leaders of their practice or clinical “microsystem”—the small, functional frontline unit that provides health care for most people. 3 As such, they have a deep practical knowledge of diagnostic and treatment processes as applied to their patients. They are frequently seen as opinion leaders, making them ideal mentors for others on the team, and are resources for health system administrators or others who are used to viewing health care delivery from a broader perspective. However, several factors complicate physicians’ involvement in this process: most physicians have no formal training in QI; many are accustomed to making decisions with relatively little input from other team members, especially nonclinical staff; and the systems in which they work may not have a history of cross-disciplinary collaboration. 4
Educating the next generation of physicians in quality and patient safety will require substantial faculty expertise to develop the necessary knowledge, skills, and attitudes. 5 An increasing number of curricula targeting trainees are being reported; however, the faculty who teach them also need training, both in leading QI efforts and in teaching it. 6 Most reported curricula involving clinician-trainers focus on specific clinical topic areas rather than the broader principles of QI as applied to clinical service and teaching. 4 Among reported barriers to educating clinician-trainers are inadequate time and resource allocation, 7 making it difficult to “stop the train” of clinicians’ activities in order to undertake intensive QI training. Most existing QI training programs are expensive, often requiring clinicians to take several days or more from their clinical work, and require travel, making the context in which they learn QI skills different from their daily work environment. These factors usually result in course attendees coming from a large number of different institutions, with only 1 or a few individuals from a given organization acquiring training. Furthermore, course faculty are unfamiliar with the specific microsystems present at attendees’ organizations.
This article describes the development and results of the initial implementation of a QI leadership training program for teaching clinicians that is feasible within their busy schedules and provides insights useful to others seeking to implement similar programs.
Methods
Setting
In 2007, the University of Massachusetts Medical School (UMMS) was awarded a 3-year Academic Administrative Units (AAU) in Primary Care training grant from the US Health Resources and Services Administration to develop a vertically integrated QI curriculum for all levels of learners, from predoctoral students through practicing physicians and academic generalist faculty. The grant was shared between UMMS’s Department of Family Medicine and Community Health, the Division of General Pediatrics, and the Division of General Internal Medicine. We developed the Quality Scholars program in collaboration with UMMS’s clinical partner, the UMass Memorial Health Care (UMMHC) Department of Quality and Patient Safety (DQPS).
At the time the project began, the DQPS had developed a cadre of 8 physician quality officers 8 to lead improvement efforts in the institution. Leaders of the AAU grant and the DQPS recognized an opportunity to jointly develop additional clinician leaders in QI who could speed the pace of QI efforts in the clinical setting.
Participants
Core faculty who were funded included 3 course directors (2 at the doctoral level and 1 at the master’s level), 1 from each generalist discipline who had extensive experience in QI and patient safety, and a master’s-level evaluator with expertise in educational research and program evaluation. Faculty and staff of the DQPS and others with expertise in QI assisted them. Department chairs nominated 10 “Quality Scholars” from the 339 primary care teaching faculty of UMMS to participate in the program. These were generalist clinicians from UMMHC clinical settings who actively taught residents and/or students at UMMS, and who were interested in taking on significant QI roles as part of their career development. Department chairs agreed to offset 10% of the scholars’ time for the 9-month course to enable focused study and dedicated time for project activities. Because the majority of scholars were junior faculty with little administrative time, this time was subtracted from the clinical workload for most scholars and was seen by department chairs as a worthwhile investment.
Curriculum Development
The curriculum was designed to fit into a 9-month academic year, to focus on projects of value to the scholars and their work environments, and not to disrupt participants’ other responsibilities. We conducted a comprehensive assessment of internal needs and resources to assess QI expertise among existing faculty. We then assessed external resources, searching existing literature on similar programs 4 and querying known quality leaders in primary care, generalist physician organization listservs, and available online open access learning. We selected the VA Quality Scholars program 9 as a starting point because it incorporated a 1-year curriculum for emerging physician scholars in quality.
Curriculum Structure
To fulfill the dual goals of teaching QI principles and conducting project-based learning, scholars’ time was split fairly evenly between didactic and project-based activities, with required prework assigned to establish baseline knowledge. We surveyed scholars before the start of the program to elicit preferred times that would not unduly disrupt clinical or family responsibilities and settled on biweekly 2-hour learning sessions from 7:30 to 9:30
Each scholar was required to identify, lead, and complete a QI project within their clinical setting. The project needed to be able to be completed within the program time frame, be of interest and value to the clinic setting, include a representative team of stakeholders (eg, provider, nursing, front desk, patient), and be in alignment with the system’s strategic goals. Identification of the planned project was part of the initial application process.
We assigned suggested project milestones for each session (eg, creating an aim statement, making a flowchart of the current and desired process, and data collection, analysis, and display). We assigned each scholar a project mentor, a volunteer who was a faculty member with experience in QI. All mentors had at least 2 years’ experience in QI as well as some degree of formal training; several were physician quality officers, and one was the chief medical officer of the medical group. We asked mentors and scholars to meet face-to-face at least twice monthly. Additional support was provided as needed from other program faculty because the primary barrier to mentor/mentee relationships (if any) was finding time to meet. We expected scholars to present their project results at the end of the program at a systemwide quality symposium sponsored jointly by the DQPS and the medical school.
Evaluation/Analysis
The program evaluation consisted of several components. First, scholars completed a pretest and posttest containing 26 knowledge-based questions (18 multiple choice and 8 fill-in questions; possible score, 3-51) and 10 attitudinal questions. After reviewing available tests in the literature and finding none with enough detail to test the concepts we wanted to teach, we asked lead faculty for each learning session (see the appendix) to develop 1 to 2 questions targeted to their session, then developed attitudinal questions based on work from the DQPS regarding performance and credentialing. We conducted traditional pre–post bivariate analyses using both parametric tests (t tests) and nonparametric tests (Wilcoxon sign rank tests because of small samples) to assess changes in knowledge scores. Second, the scholars evaluated each biweekly learning session. We used feedback formatively to guide subsequent sessions. We held 2 focus groups during 2 of the 20 learning sessions to obtain more detailed feedback from the scholars on several factors: the content and structure of the learning sessions, working with their mentors, facilitators and barriers to project progress, support and resources from division/department leadership, and future mentoring of residents and students in their practices. Finally, we conducted a postprogram summative evaluation to assess the program’s components and utility, resources, teaching modes and materials, mentor relationships, and future plans to champion QI in their practices and to plan future quality scholar programs.
The UMMS Human Subjects Committee determined the study to be exempt from oversight.
Results
Scholar demographics and project descriptions are presented in Table 1. There was a high degree of variability in age, clinical setting, academic rank, and experience with QI projects.
Scholar Demographics and Projects
Abbreviations: CHC, community health center; HPV, human papillomavirus; PCP, primary care physician.
F, family medicine; P, general pediatrics; I, general internal medicine.
Curriculum Format and Content
A detailed description of curricular components is listed in the appendix. The curriculum began with 8 to 10 hours of required prework, which included review of online training modules12,13 and specific readings from a QI text focused on clinical microsystems. 14 Most scholars completed all the prework successfully prior to the start of the course.
We chose the biweekly learning sessions in response to our needs assessment. Attendance at these sessions was high, with at least 7 of 10 scholars (median = 9) attending each session. In most sessions, the first hour was dedicated to didactic topics and the second to discussion of the scholars’ project work. There were 15 sessions that included both curriculum content and project discussion; only project work was discussed in 3 sessions, and the final 2 sessions included poster and oral presentations of projects. As the sessions progressed, we moved project discussion time to the start of each session, and teaching faculty presented material in a more concise format to allow for more project discussion time.
We designed the order of the sessions to facilitate “just in time” learning tied to the natural progression of projects from conceptualization through dissemination. For example, we presented data analysis techniques after most scholars began data collection, and we presented a session on the “business case” for QI as scholars were working on how to sustain their improvement work at the end of the program. Within these sessions, we led 2 experiential learning activities. The first was an exercise that focused on team dynamics and communication, team member roles, team leadership, and working across “silos.” The second was a simulation that demonstrated a lean approach to improving a workflow process that had multiple bottlenecks and constraints. Between-session assignments focused on supplemental readings from a variety of QI books, articles, and current publications as well as completion of tasks specific to their projects. The degree of reading done by scholars was variable, but because readings were designed mainly to complement course discussions, this did not have a negative impact on the program sessions.
Project Selection and Completion
All scholars were able to form and work with a team on a project relevant to their practice, map existing processes, identify areas for improvement, collect baseline data, conduct small tests of change, and make substantial progress during the program. Many projects involved developing a patient registry; although this often delayed collection of baseline data and improvement activities for 3 to 6 months, we saw this as an investment that could be used in future QI activities beyond the scholar’s project. Because several practice sites were connected through the clinical system, projects were often useful across multiple sites; for example, a project to improve accuracy of radiology results reporting in outpatient general internal medicine yielded a redesign of test ordering forms that benefited clinics in family medicine and general pediatrics as well. Additionally, several scholars helped one another complete their projects using prior expertise; for example, a participant with substantial expertise in the system’s electronic medical record and data presentation helped several others collect and organize their data from the system.
Assessment of Scholars’ Knowledge
Because of the small sample, statistical analyses were limited in this study. However, we calculated a composite “knowledge” score based on correct responses to the 26 knowledge-based questions in the pretest and posttest. Out of a possible score of 51, the pretest mean of 31.5 ± 4.6 (standard deviation) increased to a mean of 36.7 ± 5.3 on the posttest (t = 2.66; P = .028).
Scholar Attitudes
We asked scholars 6 questions about readiness of staff and providers in their practice to conduct QI activities and 4 questions about their own readiness to conduct and mentor QI projects. For each of the 6 questions on staff and provider readiness, there was less than a 20% difference (scattered in both directions) between pretest and posttest in the proportion of scholars answering “most of the time” or “always” over the 9-month period. However, scholars reported a 30% to 80% increase in feeling “moderately” or “very” confident on all 4 self-readiness questions.
Scholars’ Curriculum Evaluation
A majority of scholars completed evaluations of each biweekly session, and all 10 completed postprogram evaluations. In general, scholars consistently preferred group discussion and troubleshooting of one another’s project progress over topic-specific learning. They preferred teaching topics that enabled “hands-on” learning, such as process mapping and data management software as well as topics directly applicable to management of primary care processes. They rated the 2 experiential learning sessions the most highly.
Scholars rated sessions on using QI tools and techniques the most highly in their postprogram summative evaluations (Table 2). They valued in-person project discussions, didactic sessions, and online modules more highly than books and handouts that require additional reading time (Table 3). Overall strengths of the program included the opportunity to complete a project and to share ideas about future work with other scholars, mentoring, the variety of topics presented, institutional support, focus on the outpatient setting, and faculty support. Noted weaknesses of the program included a need for more project work time, a need for more time to discuss projects with other scholars, insufficient training in data gathering and analytics (especially early in the program), and a need for more integration of didactic sessions longitudinally.
Extent to Which Scholars Reported That Different Program Topics and/or Components Helped Understanding and Skill Building
Abbreviations: QI, quality improvement.
Scholars’ Ratings of Overall Value of Different Program Components and Materials
Abbreviations: QI, quality improvement.
Project Challenges
There was a 2- to 3-month start-up period for several scholars as they began to understand and prepare their practice teams for their projects. Despite the urging of program faculty, several scholars essentially began their projects on their own, though they realized the value of a team approach after a short time. There were few resources for data collection and management within the scholars’ microsystems, making data collection more time-consuming. However, by the end of the 9-month program, all scholars had collected data for at least 2 months.
Sustainability
In all, 8 of 9 scholars reported plans to continue either their specific project or related QI activities, and 5 reported plans to work directly with residents, though only 1 reported having obtained funding and/or protected time for QI activities from the division or clinic leadership in the year following the program. Moreover, at the end-of-year quality symposium, system leaders encouraged most scholars to present project findings outside the institution, and they identified several projects as being potentially worthy of publication. Finally, the DQPS is enrolling and training a second group of 11 quality scholars in 2010-2011, with plans for further expansion based on organizational needs.
Resources Required
Dedicated funding for the project from the grant was approximately $100 000, which included time for core faculty and support staff to develop and evaluate the curriculum, conduct sessions, provide mentoring for scholars, and provide scholars with books and course materials. It did not include the 10% time and effort that each scholar spent on the course and their project; each scholar’s academic department or clinical work site provided this as an investment. For Year 2, $82 000 was budgeted by the DQPS to continue the program.
Discussion
The quality scholars program was a unique opportunity to capitalize on the synergy between interested academic generalist faculty who were funded to develop a vertically integrated QI curriculum, a clinical system partner interested in developing additional QI resources, and a motivated group of scholars who were interested in obtaining this training. Our program is significantly different from many others reported; it was smaller but more intensive than most and focused on developing physician QI leadership rather than multidisciplinary expertise at the practice level. 4
Several “essential ingredients” made the program successful. First, commitment of a small but substantial amount of scholars’ time and effort was necessary. This resulted in good attendance at almost all sessions, which has been a serious problem with other initiatives, 15 and allowed time to undertake a project, collaborate with one another, and receive significant mentoring. Similarly, development of a coherent curriculum required sustained effort and dedicated time from core faculty and their support staff. Second, the scholars valued their project work higher than any other aspect of the program. Flexibility to prioritize project discussion over didactic instruction was essential to satisfy scholars’ needs. Third, clinical leadership enabled participation in projects by clinical support staff. Finally, support from leadership in the medical school and clinical system and the expectation that scholars would present their findings to leadership at the end of the program created a sense of importance, urgency, and validation of participants’ work.
As the curriculum focused on knowledge of QI principles and leadership skills, it makes sense that significant improvements were concentrated in the areas of scholars’ knowledge and self-reported readiness to conduct and mentor QI activities. Given that the program did not include any direct practice-level activities, the finding of no change in scholars’ assessment of the readiness of their practice environments to conduct QI activities over the 9-month period was not surprising. It is likely that the training received by the scholars was necessary but insufficient to achieve practice change without concomitant interdisciplinary training of practice providers and staff. As scholars’ discovery of the capabilities of their practices evolved over the course of the year, it is also possible that their increased knowledge about implementing QI made their assessments of the readiness of their practices more pessimistic. Additionally, scholars rated resident mentorship training quite low relative to other topics. This is likely a result of 2 factors. First, because scholars were relatively new to the subject matter, educating residents was a secondary goal of the program, with material presented later in the year. Second, several residency programs with which the scholars worked were not yet set up to facilitate resident participation during the quality scholars program, which resulted in frustration on the part of several scholars.
Lessons Learned
The experience from the first year of our quality scholars program contained much to inform future, similar programs. The mentoring program was critical to scholars’ success: in-person mentoring has been reported as critical to the success of other studies of QI in primary care. 16 Second, the synergy between projects, the ability of learners to teach one another based on common systems and experiences, and special knowledge of other components of the local systems on the part of some scholars assisted many in conducting their projects. Third, the diversity of participants’ prior experience pointed toward the need to provide knowledge-based teaching at multiple levels in future program iterations to ensure that all participants both understand and are challenged by the subject matter.
Participants indicated a need for practical training on software tools to assist with their projects, people and resources to assist with data collection and management, and special help in project selection and structure to help choose a feasible project. Although these factors slowed the progress of scholars’ projects, recognizing and overcoming these barriers were part of their learning process.
Limitations
Limitations of this study mainly concern generalizability to other settings, given the unique timing of the program with respect to grant funding and prioritization of QI by the clinical system in which the scholars worked. However, many health centers are currently focusing on improving quality on an institutional level, and the amount of funding required to develop and run the program is likely feasible to obtain internally in larger centers that decide to invest in QI training. The program was relatively small and was conducted within a single system, although this problem was mitigated by scholar and faculty representation from all 3 generalist disciplines as well as scholars from a variety of practice settings.
Conclusions
A 9-month program to train primary care clinicians in QI was successful in increasing QI knowledge and developing project-related expertise. The essential components of the program were modest funding to develop and teach the curriculum, investment in scholars by their clinical departments to provide time to attend the program and undertake a QI project useful in their clinical environment, collaboration between program participants in undertaking their projects and addressing barriers, and provision of mentoring resources and support by system leadership. The program was sustainable beyond 1 year; it assisted participants with career development related to QI; and it promoted development of QI leaders within the clinical system.
Footnotes
Appendix
Acknowledgements
The authors wish to thank Robert Klugman, MD, for his support and helpful advice on the project, and Francis Wanjau, MS, Alan Chuman, MPH, and Melissa McLaughlin, BA, for their expert technical and administrative support.
This material was presented in part in abstract form at the National Initiative for Children’s Healthcare Quality Annual Forum, Atlanta, GA, March 2010, and the AAMC Integrating Quality: Linking Clinical and Educational Excellence Conference, Chicago, IL, June, 2010.
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded in part by grant # D54HP01074 from the Health Resources and Services Administration.