Prevalence of Effective Audit-and-Feedback Practices in Primary Care Settings: A Qualitative Examination Within Veterans Health Administration

Abstract

Objective

The purpose of this study is to uncover and catalog the various practices for delivering and disseminating clinical performance in various Veterans Affairs (VA) locations and to evaluate their quality against evidence-based models of effective feedback as reported in the literature.

Background

Feedback can enhance clinical performance in subsequent performance episodes. However, evidence is clear that the way in which feedback is delivered determines whether performance is harmed or improved.

Method

We purposively sampled 16 geographically dispersed VA hospitals based on high, low, consistently moderate, and moderately average highly variable performance on a set of 17 outpatient clinical performance measures. We excluded four sites due to insufficient interview data. We interviewed four key personnel from each location (n = 48) to uncover effective and ineffective audit and feedback strategies. Interviews were transcribed and analyzed qualitatively using a framework-based content analysis approach to identify emergent themes.

Results

We identified 102 unique strategies used to deliver feedback. Of these strategies, 64 (62.74%) have been found to be ineffective according to the audit-and-feedback research literature. Comparing features common to effective (e.g., individually tailored, computerized feedback reports) versus ineffective (e.g., large staff meetings) strategies, most ineffective strategies delivered feedback in meetings, whereas strategies receiving the highest effectiveness scores delivered feedback via visually understood reports that did not occur in a group setting.

Conclusions

Findings show that current practices are leveraging largely ineffective feedback strategies. Future research should seek to identify the longitudinal impact of current feedback and audit practices on clinical performance.

Application

Feedback in primary care has little standardization and does not follow available evidence for effective feedback design. Future research in this area is warranted.

Keywords

clinical performance primary care audit and feedback content analysis

Audit and feedback (A&F, providing recipients with periodic summaries of their performance) has become a widely spread quality improvement (QI) tool for changing clinician behavior, and is an important way to guide individuals’ actions and decisions (Institute of Medicine (US) Committee on Quality of Health Care in America, 2000). According to feedback intervention theory (FIT; Kluger & DeNisi, 1996) and its more clinically focused sibling, CP-FIT (Brown et al., 2019), A&F works by providing clinicians with knowledge of their performance results, thus redirecting attention to the details of the task of interest (Dougherty & Conway, 2008). A&F has been used and shown to improve quality of care in a wide variety of healthcare settings; for example, a meta-analysis by Tannenbaum and Cerasoli (2013) found that feedback in the form of debriefs enhances both individual and team performance in subsequent performance sessions by as much as 25%. A&F has also been shown to improve both proximal and distal behaviors that impact quality of care, such as prescribing behavior and diabetes management.

The effectiveness of this QI intervention, however, is highly dependent on the presence of specific design features incorporated into the intervention (Ivers et al., 2012). For example, meta-analytic research indicates that features such as the format (e.g., written), content elements such as correct-solution information, appropriate comparators, data at the correct level of aggregation, and frequent feedback close in time to the behavior being observed yield significantly better results than A&F interventions without such features (Hysong, 2009). Other research also points to the absence of and need for using theory in the design of A&F to create a tool that works best for the given context in question. Together, these point to a portfolio of evidence-based best practices for design and delivery of A&F in clinical settings.

A&F has begun to more explicitly incorporate theory and evidence-based characteristics and features in their design; what is far less known is the extent to which these evidence-based best practices have been translated into everyday practice. To that end, this study aims to describe the types and prevalence of A&F strategies employed in a national sample of Veterans Health Administration (VHA) primary care practices, and assess the extent to which the strategies observed align with evidence-based best practices for A&F as reported in the literature.

Method

Design

Part of a larger study of A&F, this research consists of qualitative, primary analysis of telephone interviews with facility leadership and primary care personnel at 16 U.S. Department of Veterans Affairs Medical Centers (VAMCs), employing a cross-sectional design with purposive sampling guided by preliminary analyses of clinical performance data. Methods for this work have been described extensively elsewhere (Hysong et al., 2012) and are summarized here.

Site Selection

Sites were selected using a purposive stratified approach based on their scores on a profile of 15 outpatient clinical performance measures extracted from VHA’s External Peer Review Program (EPRP) representing five domains: diabetes care (five measures), hypertension care (two measures), immunizations (two measures), cancer screening (three measures), and tobacco use (three measures). EPRP is a nationally abstracted database containing performance data for all VHA medical facilities on over 90 indicators covering access, quality of care, cost-effectiveness, and patient satisfaction domains. Data are abstracted monthly, but reported quarterly, to facility leadership, who then disseminate as they deem necessary. We refer interested readers to our published protocol paper (Hysong et al., 2012) for further details about these measures.

We selected four high-performing, four low-performing, four consistently moderately performing (i.e., those with moderate average performance and the lowest variability across measures), and four moderate, yet highly variably-performing facilities (i.e., those with moderate average performance and the highest variability across measures; Hysong et al., 2012). Sites were selected based on their treatment of chronic and preventive care for two primary reasons: one is so that feedback regarding a specific set of patients can be implemented (e.g., likelihood of opportunity to implement feedback is greater than acute care settings) and the second is that patient acuity may be lower, inducing lower workload and increasing likelihood for learning from feedback to transfer to the work environment (Holton & Baldwin, 2003; Sweller, 1988).

Participants

We interviewed up to four participants at each of 16 geographically dispersed VAMCs, drawn from the following groups: the facility director, the associate chief of staff (ACOS) for primary care, one full-time primary care physician and/or physician extender, and one full-time primary care nurse. Table 1 summarizes which roles were interviewed at each facility.

Table 1

Site Characteristics and Roles Interviewed at Each Site

Performance Category	Site	Size (# Unique Patients)	Residents per 10k Patients^†	Number of Primary Care Personnel	Interviewee Role
Performance Category	Site	Size (# Unique Patients)	Residents per 10k Patients^†	Number of Primary Care Personnel	FD	ACOS	MD	RN
High performers	B	27,222	0.00	35	✓	✓	✓	✓
	H	27,851	8.62	62	--	✓	--	✓
	M	43,845	18.25	56	✓	✓	--	--
	R^*	49,813	31.42	83	--	--	✓	--
Consistently moderate	D	44,022	26.18	115	✓	✓	✓	✓
	E	63,313	10.63	94	✓	✓	✓	✓
	K	46,373	56.93	125	✓	✓	✓	--
	P^*	80,022	21.45	54	--	✓	--	✓
Highly variable	A	60,528	23.15	143	✓	✓	✓	✓
	G	49,309	26.24	27	✓	--	✓	✓
	L	21,327	7.03	30	--	--	✓	✓
	Q^*	39,820	2.89	10	✓	✓	--	--
Low performers	C	44,391	27.51	88	✓	✓	✓	✓
	F	19,609	0.00	46	✓	✓	✓	✓
	J	58,630	24.94	116	✓	✓	✓	✓
	N^*	24,795	0.00	23	--	✓	✓	✓

Note. *Sites listed in italic type were sites excluded from the study due to insufficient data (either insufficient number of interviews or insufficient information about feedback strategies was raised in the interviews, thus making any findings from that site unstable). †Number of residents per 10k patients is intended as a measure of the strength of the academic mission of the facility, which has been shown to be a more nuanced indicator than the dichotomous medical school affiliation measure used traditionally (Byrne et al., 2009).

Procedure

Participants were interviewed individually for 1 h by a research team member at a mutually agreed upon time; interviews were audio-recorded with the participant’s consent. Participants answered questions about (a) the types of EPRP information they receive, (b) the types of quality/clinical performance information they actively seek out, (c) opinions and attitudes about the utility of EPRP data (with specific emphasis on the role of targets), (d) how they use the information they receive and/or seek out, and (e) any additional sources of information or strategies they might use to improve the facility performance.

Data Analysis

Interview recordings were transcribed and analyzed using techniques adapted from grounded theory (Strauss & Corbin, 1988) and content analysis (Weber, 1990) using Atlas.ti, a qualitative analysis (Hwang, 2008) software program.

Strategy domains, dissemination methods, and prevalence

Each unique strategy for providing feedback found in interviews was assigned a code to observe its prevalence in the data. As several individual feedback strategies were uncovered, we further grouped strategies by the dissemination mechanisms through which feedback was delivered: computerized delivery, written reports, meetings, and conversations. Dissemination method(s) offer detail on how performance measures are delivered to providers when they do receive said feedback. The most frequently used strategies may not align with best practices and most effective strategies; thereby, we also identified the most frequently used strategies to provide contextualized guidance for enhancing A&F strategies.

Alignment with evidence-based best practice

Finally, to assess how well a given strategy followed evidence-based best practices, each feedback strategy was evaluated against each of the feedback cues (characteristics) demonstrated by previous research to either improve or hinder feedback effectiveness (Hysong et al., 2006; Kluger & DeNisi, 1996). We calculated a “feedback richness score” for each strategy derived from guidelines produced by Kluger and DeNisi (1996)’s (1996) FIT in combination with Hysong et al.’s (2006) principles for effective feedback by subtracting the number of characteristics that hinder feedback effectiveness (ineffective cues) from the number of characteristics that improve feedback effectiveness (effective cues; please see Supplemental Material 1).

Results

Strategy Domains and Prevalence

We found 102 feedback strategies used by our sample of sites; we further identified four primary delivery types through which feedback (i.e., computerized delivery [n = 1], reports [n = 12], meetings [n = 23], and conversations [n = 13]; please see Supplemental Material 2).

Second, we examined the prevalence of each specific strategy (high or low quality). The most common strategies used to deliver feedback on clinical performance were the default performance measure report furnished by EPRP (n = 56, 8% of 661 total coded instances of feedback strategies included in our qualitative examination), primary care meetings (regular meetings of members of the primary care service line including clerks, clinicians, and leadership; n = 41, 6%), the Primary Care Almanac (a web-based report of primary care specific measures; n = 38, 6%), clinical reminder report (a regular report of a clinician’s clinical reminder completion; n = 36, 5%), clinical reminders (pop-ups in the electronic medical record intended to remind clinicians to provide certain screenings or tests to patients; n = 32, 5%), and the standard employee performance evaluation (n = 36, 5%; Table 2 for examples as described by interviewees).

Table 2

Most Frequently Occurring Feedback Strategies

Strategy	Excerpt	Frequency
EPRP report	“…we received a combined EPRP report that had a lot of other data like flu vaccines, colon screening evaluations in addition to the usual things. And then we were compared to all other providers, not just in [Site G] but in the community-based outpatient clinics like CBOCS…. I talked to my nurses and just made it clear… to start askin’ patients about, uh fecal occult colon screening tests and to, you know, give ‘em the cards if they haven’t been screened in a year.”	56 (8%)
Primary care meetings	“Well you know it’s- there’s several huddles that goes on. First you have the primary care through the uh—here we have primary care. Our chief of primary care has a daily huddle or briefing of the activities from the day before because our doctor sometimes covers in-patient so they notify them.”	41 (6%)
Primary care almanac	“And the Patient Care Almanac gives you an overview of your panel size, the different disease conditions, the male/female ratio, employees, non-employee, and all that kind of stuff and um also… it gives you a report of you, comparing you with your peer n the system. It gives you your, primary care, your blood pressure, your blood pressure control, like the, performance measures are put in there and they tell you which of your patients are outliers and how many percent are outliers and how many patients … how many percent are in the group that you want, you are supposed to be, and we also get intermittently a list of our performance measures and how we are, we are doing compared to the rest of the group. Inter-clinic comparison.”	38 (6%)
Clinical reminder report	“…everybody had performance measures but initially primary care certainly seemed to own a majority of them, and at that time we used the clinical reminder dues reports and so we were very active. You know developing and improving clinical reminders within CPRS and then using clinical reminder dues reports and that was the bulk of what we did.”	36 (5%)
Clinical reminders	“I would say what we do like I said like if the reminders are there—the clinical reminders are on. We just do whatever it takes and the patient’s going to be seen that day we—we just address that.”	32 (5%)
Performance evaluations	“But from a, from a physician standpoint the, uh the aggregate data is reviewed twice a year with our providers. Um, it’s reviewed as part of their ongoing professional practice evaluation. So we have a, uh compilation of data, uh that they get to see and some of it we’re able to actually give them, uh enough information so they can compare their performance to, uh the rest of the facility, their peers, to VA and, and to the VISN, um and some data’s just, uh you know, data that we can give to them about themselves.”	36 (5%)

Note. EPRP is the standardized clinical performance measurement system employed across VHA at the time of data collection; CPRS is VHA’s nationwide electronic medical record at the time of data collection. CBOC = community-based outpatient clinic; CPRS = computerized physician reporting system; EPRP = External Peer Review Program.

Alignment With Evidence-Based Best Practice

Feedback strategies receiving the highest richness score(s), all of which received a richness score of 2 (out of a possible 10), disseminate information to clinic staff through use of visual means. Among these, clinical reminder report poster(s) (defined as a poster displayed where clinical staff members can see it that displays data on clinically relevant metrics derived from data available through the clinical reminders) and provision of relevant reports of performance on EPRP measures. EPRP reports providing feedback at the individual and clinic level are considered to be higher in richness than other means of providing EPRP reports. Conversely, several excerpts demonstrating low quality, ineffective strategies emerged from the data. The strategies receiving the lowest richness score include providing feedback to individuals during primary care nurse meetings (richness score of −6) and delivering feedback during meetings with providers who fall below par on the current clinical EPRP measures (richness score of −4). Other low-quality and seemingly less effective strategies emerging from the data include disseminating feedback during morning meetings (richness score of −2) and delivery of feedback during performance measure meetings (richness score of –3). Most ineffective strategies delivered feedback in meetings, whereas strategies receiving the highest richness score(s) did not occur in a group setting (i.e., individually); further, most effective strategies delivered feedback via visually understood reports that did not occur in a group setting (Table 3).

Table 3

Strategy Categorizations and Richness Scores

Strategy	Interview Excerpt	Richness Score	Dissemination Method
Clinical reminder poster	“We had some posters in the clinic... Everyone had a number and they could/would know who they were and they could [calculate] the average and they could see other people—how they were doing…pretty quickly over that year was that everybody got up to, you know, over 90% on doing these clinical reminders.”	2	Report
EPRP reviewer document	“The other tool is when the EPRP reviewer comes; there’s an extensive word document with all of the measures, not just the ones that relate to the network director’s performance plan …that not only looks at out-patient but in-patient and all of the various areas that the external peer reviewer has reviewed and that’s discussed once a month with the—uh with the external peer reviewer in the room with representatives from each of the service lines.”	2	Report
Individual EPRP reports	“We used to, when I first started working here, get regular EPRP reports, and so every month or every 3 months we would get our own individual numbers passed out to us saying this is how many patients you have who have met the measure for this, that, and the other, and I found that very useful early in my practice.”	2	Report
Morning meetings	“So the supervisors that are all there that I mentioned that are at our morning report every morning at 8:15 we get a, if there are any veterans or any care that’s not provided that’s, you know, related to any of these performance measures that list of, uh is given, uh to those leaders at that meeting at that time.”	–2	Meeting
Performance measure meetings	“Well some—some of that data is EPRP, some of it is other performance measures, some of it could just be simply from if I was to do a chart audit and see that notes aren’t being signed. So we don’t only use EPRP data; we use other data that comes from um our regular audits. And it’s disseminated—my performance measures person—she actually had weekly clinical reminders and performance measures meeting with doctors, nurses, she also attends their meetings.”	–3	Meeting
Meetings with providers not meeting measures	“Well what I—what I do is I—I uh farm it out to either the assistant chief and…they’re spoken with. You know we go over the data sheet and over what they should you know what—we would’ve uh liked them to have done or to have uh pursued. Uh so it’s uh—it’s feed back to them person to person.”	–4	Conversation
Primary care nurse meetings	I: “So … [in primary care meetings], what was important?” p: “Well let me give you the flaw. This person looks at these records. That person communicates with the nurse manager of ambulatory care… at the same time, each manager is in different areas… If there was some sort of lackadaisical attitude, we used a little bit of stick to just be sure that that was resolved.”	–6	Meeting

Note. EPRP = External Peer Review Program.

Strategy Dissemination

For added guidance and perspective, we analyzed overall effective versus ineffective strategies as categorized by their dissemination methods. Among these dissemination methods, ineffective strategies had a broad range of dissemination methods, including information regarding A&F delivered via meetings (n = 17), conversations (n = 9), reports (n = 5), and computer interface (n = 1). In the more effective dissemination strategies, feedback was typically delivered to recipients via reports (n = 7), meetings (n = 6), and conversations (n = 5). No significant differences in dissemination method prevalence were noted between ineffective and effective strategy types (χ² = 3.66, p = .30).

Discussion

In this paper, we sought to describe the types and prevalence of A&F strategies employed in a national sample of VHA primary care practices and assess the extent to which the strategies observed align with evidence-based best practices for A&F. We found great variability in the quality and frequency of feedback strategies across sites and found that the feedback strategies employed by most sites exhibited characteristics shown in the literature to make feedback less rather than more effective (Kluger & DeNisi, 1998). Although feedback strategies appear to vary widely, the low utilization of feedback strategies recommended by the scientific literature suggests a need for improvement in the manner in which feedback is delivered in healthcare settings.

This paper contributes to outpatient QI in a number of ways. First, it sheds light on the strategies leveraged in providing feedback in a particular clinical care context (i.e., outpatient care) and provides an indication of the quality and potential effectiveness of strategies employed. For instance, whereas feedback delivered (particularly to low performing care providers) in a group setting received the lowest richness scores, it is also one of the most frequently leveraged methods for providing individual-level feedback on clinical performance (Pedersen et al., 2018). Group meetings may pose a convenient opportunity to provide feedback to providers considering logistics and scheduling. However, most meetings were largely ineffective as a platform for feedback dissemination as receiving negative feedback in the presence of colleagues and peers may spike perceived judgment from fear, enhancing self-focus while receiving feedback on performance (Kluger & DeNisi, 1996, 1998). Interestingly, not all meetings to share feedback were the same. Feedback disseminated via individualized meetings arranged between supervisors and subordinates to discuss objective measures provided tangible evidence as to the current state and a forum free from peer influence or shame as a means to discuss necessary improvements. Reports and dashboards customized to individual provider needs are considered effective for similar reasons, as well as providing opportunity for future review and reference by the clinician. Effectiveness of such reports and feedback meetings can be increased by providing correct-solution information (e.g., providing reminders of preferred evidence-based medications for hypertension displayed alongside hypertension control metrics, or as part of the discussion),

Implications

Findings from this study attest to the quality of the feedback commonly delivered to primary care personnel. This study contributes a deeper knowledge of the current practices used in outpatient settings to disseminate and deliver clinical performance feedback, and how such strategies may be more or less aligned with evidence-based best practices as found in models of effective feedback in the literature. Interestingly, trends across facilities appear to show the uptake of anxiety-provoking feedback techniques, such as providing feedback in front of superiors or colleagues or focusing heavily on negative performance outcomes. Given self-focus and added stress, the literature on A&F suggests these strategies as being largely ineffective (Hughes et al., 2016; Kluger & DeNisi, 1998). Further, the most frequently used strategies do not appear to align with the best practices (Ivers, Grimshaw, et al., 2014; Ivers, Sales, et al., 2014); namely, meetings and use of computer interfaces appear to trump the methods used to disseminate clinical performance to primary care clinicians. Finally, although use of a computer interface to disseminate information may be convenient, more should be done to understand its impact on learner gains from feedback delivered in this manner.

Findings from this paper also contribute to evidence-based practice in team-based settings. While clinical reminders and feedback delivered electronically (e.g., via computer interface) require few added resources and can deliver performance information directly to a learner, the future of healthcare is delivered in teams. For instance, during the completion of this study, VHA outpatient care transitioned to a formalized team-based approach to delivering care in their adaptation of the Joint Commission’s Patient-Centered Medical Home Model (PCMH), Patient-Aligned Care Teams (PACTs). A&F strategies (e.g., debriefs) are critical to healthcare team performance (Gregory et al., 2021; Tannenbaum & Cerasoli, 2013) so that team members, whose work often overlaps considerably (Best et al., 2006), can learn from previous performance episodes. Identifying pertinent A&F practices to outpatient care teams may require unique considerations and as such is another avenue for future research.

Limitations

Although this study is helpful for uncovering practices for delivering clinical performance feedback within VA centers, it is not without limitations. First, we are unable to speak to the impact that a site’s given feedback practices may directly or indirectly have on their clinical performance data; namely, as the studies originally classified as high or low performers were not consistent across time in performance, we were unable to quantify the impact of feedback strategies on performance of a clinic. This may be due to the fact that as strategies varied across and within sites, performance may have fluctuated accordingly over time, making it impossible to link a specific strategy to performance changes (Byrne et al., 2009). Second, although our study aimed, by design, to examine facility-level differences in feedback strategies, new research suggests that variability in clinical performance is more prevalent, and more amenable to change, at the within-facility level. Future research should be directed at a lower level of analysis than that reported here.

Conclusions/Future Directions

Nearly two-thirds of clinical performance strategies currently in use by facilities in our sample are inconsistent with best practices for delivery of A&F. Findings speak to the variety of A&F practices used within the VHA outpatient setting. Future work should examine the effect of these strategies on specific and meaningful clinical performance indices longitudinally and examine A&F practices for outpatient care teams.

Key Points

Audit and feedback interventions are pivotal to fostering learning in health systems.

The quality, variety, and frequency of strategies used to deliver clinical performance feedback in primary care settings vary greatly, at both high- and low-performing VHA facilities.

Our findings suggest that several strategies leveraged for delivering feedback to primary care clinicians were linked to low quality.

Future research should examine impact of specific feedback delivery strategies on clinical performance improvement in individual and team-based healthcare settings.

Supplemental Material

Online supplementary file 1 - Supplemental material for Prevalence of Effective Audit-and-Feedback Practices in Primary Care Settings: A Qualitative Examination Within Veterans Health Administration

Supplemental material, Online supplementary file 1, for Prevalence of Effective Audit-and-Feedback Practices in Primary Care Settings: A Qualitative Examination Within Veterans Health Administration by Sylvia J. Hysong, Richard SoRelle and Ashley M. Hughes in Human Factors: The Journal of Human Factors and Ergonomics Society

Supplemental Material

Online supplementary file 2 - Supplemental material for Prevalence of Effective Audit-and-Feedback Practices in Primary Care Settings: A Qualitative Examination Within Veterans Health Administration

Supplemental material, online supplementary file 2, for Prevalence of Effective Audit-and-Feedback Practices in Primary Care Settings: A Qualitative Examination Within Veterans Health Administration by Sylvia J. Hysong, Richard SoRelle and Ashley M. Hughes in Human Factors: The Journal of Human Factors and Ergonomics Society

Footnotes

Acknowledgments

We would like to thank Kristen Broussard-Smitham, Melisssa Knox, and Thach Tran for their assistance with data collection and analysis for this study. The research reported here was supported by the US Department of Veterans Affairs Health Services Research and Development Service (grant nos. IIR 09–095, CD2-07-0181) and partly supported by the facilities and resources of the VA Center for Innovations in Quality, Effectiveness and Safety (CIN 13-413). Elements of this work were previously presented at the Annual Research Meeting of Academy Health, on Sunday, June 26, 2016. This study was reviewed and approved by the Baylor College of Medicine Institutional Review Board (IRB), protocol #H-20386.

Author(s) Note

The author(s) of this article are U.S. government employees and created the article within the scope of their employment. As a work of the U.S. federal government, the content of the article is in the public domain.

ORCID iD

Ashley M. Hughes

Supplemental Material

The online supplemental material is available with the manuscript on the HF website.

Author Biographies

Sylvia J. Hysong is a lead research health scientist at the Center of Innovations in Quality, Effectiveness and Safety (IQuESt) at the Michael E. DeBakey VA Medical Center and professor at Baylor College of Medicine. Her research interests include primary health care as a work environment, performance measurement and feedback systems, and care coordination in primary care settings.

Richard SoRelle is a research coordinator and health science specialist at the Center for Innovations in Quality, Effectiveness, and Safety (IQuESt) at the Michael E. DeBakey VA Medical Center. His research interests include qualitative social research and health services research and development within primary care settings.

Ashley M. Hughes is an assistant professor in the Department of Biomedical and Health Information Sciences at the University of Illinois at Chicago and a research health scientist at the Center of Innovations for Chronic, Complex Healthcare (CINCCH) at the Edward Hines JR VA Medical Center. She received her PhD in Applied Experimental Human Factors Psychology in 2016 from the University of Central Florida.

References

Best

R. G.

Hysong

S. J.

Pugh

J. A.

Ghosh

Moore

F. I

. (2006). Task overlap among primary care team members: An opportunity for system redesign? Journal of Healthcare Management, 51, 295–306.doi:10.1097/00115514-200609000-00005

17039689

Brown

Gude

W. T.

Blakeman

van der Veer

S. N.

Ivers

Francis

J. J.

Lorencatto

Presseau

Peek

Daker-White

. (2019). Clinical performance feedback intervention theory (CP-FIT): A new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implementation Science, 14, 40.doi:10.1186/s13012-019-0883-5

31027495

Byrne

M. M.

Daw

C. N.

Nelson

H. A.

Urech

T. H.

Pietz

Petersen

L. A

. (2009). Method to develop health care peer groups for quality and financial comparisons across hospitals. Health Services Research, 44, 577–592.doi:10.1111/j.1475-6773.2008.00916.x

19178585

Dougherty

Conway

. (2008). The “3T’s” road map to transform US health care: The “how” of high-quality care. Journal of the American Medical Association, 299, 2312–2319.

Gregory

M. E.

Hughes

A. M.

Benishek

L. E.

Sonesh

S. C.

Lazzara

E. H.

Woodard

L. D.

Salas

. (2021). Toward the development of the perfect medical team: Critical components for adaptation. Journal of Patient Safety, 17, e47–e70.doi:10.1097/PTS.0000000000000598

Holton

E. I.

Baldwin

. (2003). Improving learning transfer in organizations. John Wiley & Sons.

Hughes

A. M.

Gregory

M. E.

Joseph

D. L.

Sonesh

S. C.

Marlow

S. L.

Lacerenza

C. N.

Benishek

L. E.

King

H. B.

Salas

. (2016). Saving lives: A meta-analysis of team training in healthcare. Journal of Applied Psychology, 101, 1266–1304.doi:10.1037/apl0000120

Hwang

. (2008). Utilizing qualitative data analysis software: A review of Atlas.ti. Social Science Computational Review, 26, 519–527.

Hysong

S. J

. (2009). Meta-analysis: Audit and feedback features impact effectiveness on care quality. Medical Care, 47, 356–363.doi:10.1097/MLR.0b013e3181893f6b

19194332

10.

Hysong

S. J.

Best

R. G.

Pugh

J. A

. (2006). Audit and feedback and clinical practice guideline adherence: Making feedback actionable. Implementation Science, 1, 9.doi:10.1186/1748-5908-1-9

16722539

11.

Hysong

S. J.

Teal

C. R.

Khan

M. J.

Haidet

. (2012). Improving quality of care through improved audit and feedback. Implementation Science, 7, 45.doi:10.1186/1748-5908-7-45

22607640

12.

Institute of Medicine (US) Committee on Quality of Health Care in America . (2000). Kohn

L. T.

Corrigan

J. M.

Donaldson

M. S.

(Eds.), To err is human: Building a safer health system. National Academies Press. 10.17226/9728.doi:10.17226/9728

13.

Ivers

Jamtvedt

Flottorp

Young

J. M.

Odgaard-Jensen

French

S. D.

O’Brien

M. A.

Johansen

Grimshaw

Oxman

A. D

. (2012). Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, 6, CD000259.doi:10.1002/14651858.CD000259.pub3

22696318

14.

Ivers

N. M.

Grimshaw

J. M.

Jamtvedt

Flottorp

O’Brien

M. A.

French

S. D.

Young

Odgaard-Jensen

. (2014). Growing literature, stagnant science? Systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. Journal of General Internal Medicine, 29, 1534–1541.doi:10.1007/s11606-014-2913-y

24965281

15.

Ivers

N. M.

Sales

Colquhoun

Michie

Foy

Francis

J. J.

Grimshaw

J. M

. (2014). No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implementation Science, 9, 1–8.doi:10.1186/1748-5908-9-14

16.

Kluger

DeNisi

. (1998). Feedback interventions: Toward the understanding of a double-edged sword. Current Directions in Psychological Science, 7, 67–72.

17.

Kluger

A. N.

DeNisi

. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 119, 254–284.doi:10.1037/0033-2909.119.2.254

18.

Pedersen

M. S.

Landheim

Møller

Lien

. (2018). Audit and feedback in mental healthcare: Staff experiences. International Journal of Health Care Quality Assurance, 31, 822–833.doi:10.1108/IJHCQA-08-2017-0142

30354880

19.

Strauss

Corbin

. (1988). Basics of qualitative research techniques and procedures for developing grounded theory. Sage Publications.

20.

Sweller

. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.doi:10.1207/s15516709cog1202_4

21.

Tannenbaum

S. I.

Cerasoli

C. P

. (2013). Do team and individual debriefs enhance performance? A meta-analysis. Human Factors: The Journal of the Human Factors and Ergonomics Society, 55, 231–245.doi:10.1177/0018720812448394

23516804

22.

Weber

. (1990). Basic content analysis. SAGE Publishing.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.05 MB

0.18 MB