Redesigning the Peer Audit Process to Enhance Clinical Dialogue

Health Care Audit and Documentation

A recent Cochrane review (Ivers et al., 2012) examining the effectiveness of audits on health care professionals (mostly physicians) concluded that audit and feedback processes can play an important role in improving professional practice, but effectiveness is highly variable. The review identified feedback from a respected colleague and feedback that is delivered in both written and verbal formats as two important components contributing to the effectiveness of an audit process. The best way to deliver feedback remains uncertain (Ivers et al., 2012).

Consistent therapist involvement in documentation audits was important to therapy leadership because the completeness and quality of documentation is a frequent source of management concern and focus of improvement efforts. Many of the highest priorities in the design of an effective health care delivery system rely on effective communication. One of the ongoing challenges of developing or maintaining an effective documentation system is the inherent complexity of trying to respond to the needs of multiple audiences. The medical record is an important communication tool and is used by many different parties for differing purposes (Weed & Weed, 2011). Shared knowledge and the free flow of information, evidence-based decision making, the need for transparency, and cooperation among clinicians are all traits that are promoted through improved clinical documentation (Institute of Medicine, Committee on Quality Health Care in America, 2001). These traits match the opportunities for improvement in the peer audit process as identified by therapy leadership.

Essential components need to be included in therapy documentation, but clinicians may need to be reminded of the importance of documentation beyond justifying reimbursement. Clinicians are keenly aware of the amount of time and effort that goes into documenting therapy care. The peer audit should encourage clinicians to also view the medical record as an instrument for achieving important objectives, such as dealing with heavy workloads, setting priorities, learning efficiently, and using resources wisely (Weed, 1968). Highlighting aspects of the medical record that are essential to reliable continuity of care could help clarify the purpose and reinforce the value of a peer audit (Weed, 1968).

Redesign of the Peer Audit System

Therapy leadership at the facility set out to improve compliance with peer audits by using feedback from staff and therapy leaders. Satisfaction with the system at this facility was low despite the potential for mutual benefit in a peer audit. This dissatisfaction was clear in clinician comments and supervisor reports of poor compliance with timely completion of the annual audit requirement. The clinicians and supervisors believed that an increased emphasis on clinical relevance could improve compliance and satisfaction with the process.

Therapy leaders wanted to have a peer audit system that made a clear connection between clinical documentation and the ability to affect clients’ or patients’ lives, thereby tapping into a clinician’s greater sense of purpose. The redesign team, which included volunteers from the three therapy disciplines of physical therapy, occupational therapy, and speech–language pathology, incorporated principles of change management into its approach because the redesign would likely result in a new process that clinicians would perceive as a significant change. This effort began with a consideration of the concepts of intrinsic motivation and social motivation.

Clinicians need to have intrinsic motivation to accept and implement change. Intrinsic motivation refers to a person’s internal motivation to engage in an activity or behavior independent of external consequences or rewards. The three components of intrinsic motivation are purpose, mastery, and autonomy (Pink, 2009).

Clarifying the purpose of a structured peer audit system is an important component to promote intrinsic motivation (Pink, 2009). Mastery is the desire to get better and better at something that matters (Pink, 2009). Motivation to achieve mastery implies a belief that skills are improvable through deliberate effort and practice. This deliberate effort requires challenging oneself, testing oneself against a standard, and receiving clear and timely feedback. The redesign team attempted to draw analogies between the coaching opportunity presented by a peer audit and real-time coaching to learn or improve a skill, such as playing a sport or a musical instrument.

Autonomy in the audit process implies a high degree of self-directed learning. For example, the auditor has wide latitude to choose what clinical aspects to question or discuss. The redesign team wanted to give structure to direct the process and freedom to allow clinicians to have the learning experience that was most relevant to them.

Social motivation refers to the incentive or drive to interact with, and be accepted by, others that influences a person’s behavior. The key to social motivation is to harness peer pressure (Patterson, Grenny, Maxfield, McMillan, & Switzler, 2008). Social networks can be extremely powerful and are readily accessible. Enlisting the support of respected and connected colleagues to encourage open discussion about group expectations can tap into a person’s need for acceptance and connectedness (Patterson et al., 2008). Creating a process that generates experiences and stories of growth and learning for both the auditee and the auditor could help promote a reliable and sustainable change in behavior.

The purpose of this administrative case report is to describe the complex task of redesigning the Rehabilitation Therapies department’s peer audit system and the group facilitation process the clinicians engaged in to accomplish this task. Expectations were that the new system would be more meaningful to clinicians, lead to more clinical conversations, and improve audit participation rates.

Target Setting

The target setting for this project was the Rehabilitation Therapies department at the University of Vermont Medical Center in Burlington, a regional medical center in a rural setting. The department staff treats patients in a 413-bed university hospital, in a 34-bed inpatient rehabilitation unit, and at seven outpatient sites. The outpatient sites serve a broad range of patient populations and have different geographic locations. Patients with orthopedic, neurologic, medical, surgical, traumatic, cardiac, pulmonary, and oncologic conditions receive treatment throughout all of the sites. Fifty-five physical therapist full-time equivalents (FTEs), 3 physical therapist assistant FTEs, 31 occupational therapist FTEs, 1 occupational therapy assistant FTE, and 10 speech–language pathologist FTEs are employed at this facility. This project focused on these professionals across all sites.

Method

The director of Rehabilitation Therapies (DRT) sent an invitation to staff seeking volunteers to participate in a therapies project group to promote both clinical mastery and compliance with billing and regulatory requirements. The invitation included key elements of an authentic request to engage in and commit to a process of meaningful change, including a clear statement of participant expectations and project objectives (Block, 2008; Eyler & Kapusta, 2011). The stated objective of the group was to review and analyze the department’s current clinical peer audit process with the aim of designing a new approach. The DRT indicated that the group would have the option to create new tools or to build on and revise existing ones. The DRT selected group members to include clinical representatives from each discipline as well as leadership and support staff representation. The group, known as the redesign team, consisted of a physical therapist, an occupational therapist, a speech–language pathologist, a clinical research educator, a therapy coder, a therapy support specialist, a department leader, and a facilitator. Ad hoc participants could also serve in an advisory capacity.

A project charter provided a structured format for guiding and organizing the project team and tracking progress (Eyler & Kapusta, 2012). The redesign team determined ground rules for the meetings. High attendance at all meetings was required, and a defined plan for how to proceed when members were absent was determined. All members needed to complete the required background reading. The DRT provided background reading (i.e., Axelrod & Axelrod, 2001; Patterson et al., 2008; Pink, 2009; Plsek, 2001; Weed, 1978) at the start of the project, and prework, designed to be completed before the next meeting, was established at each meeting. Roles and responsibilities of the redesign team were established. Opportunities, challenges, and barriers were also determined as a starting point for discussion.

The redesign team used a consensus model for decision making (Susskind, McKearnan, & Thomas-Larmer, 1999) throughout the project and established decision rules at the first meeting. The team chose the consensus model because it had recently been used successfully with another project group at the facility. The redesign team had the authority to implement all decisions reached by consensus without extra administrative approval. The DRT would make decisions if the redesign team could not reach consensus.

This rehabilitation therapy department has a longstanding commitment to the documentation standards and philosophy of the Problem-Oriented Medical Record (POMR) system and SOAP (subjective, objective, assessment, and plan) note format (Feitelberg, 1984; Weed, 1969, 1978). The redesign team was expected to maintain consistency between their decisions and the departmental principles. The four behaviors of the problem-oriented system (thoroughness, reliability, analytic sense, and efficiency; Feitelberg, 1984; Weed, 1969, 1978) would be addressed in the new tool.

The redesign team developed a process for regular communication with rehabilitation therapy staff about the progress of the project. Team members attended staff meetings to either share or gather information at critical points of the project, such as the project start, the audit trial start period, and the postrevision survey completion. Staff were also encouraged to give feedback throughout the redesign process. This feedback was a deliberate effort to increase stakeholders’ connection to, and engagement in, the process to allow them to better understand and shape the outcome.

The redesign team started meeting in December 2010, and the meetings were held weekly for the first 4 mo. Each meeting lasted 2 hr, and the location alternated between two department sites. Meetings were not conducted during the audit trial period, except for one meeting in November 2011 to agree on the process for gathering feedback from the trial. The meetings resumed, and the redesign team met 6 times between February and May 2012 to revise the audit. The redesign team met intermittently until November 2013 to complete the work from the postrevision survey.

Meeting time was used to evaluate the different audit processes, tools, and requirements; develop the survey and analyze the results; determine that one audit tool was going to be developed; develop the questions for the new audit tool; develop the questions to gather feedback on the new audit tool; revise the audit tool based on the feedback; and determine recommendations for the audit process and requirements.

Procedure

Therapy leadership and staff were invited to participate in a confidential online survey in January 2011 before revising the peer audit system. Staff provided ratings about whether the peer audit process was meaningful and clinically relevant; they also rated their level of satisfaction with the peer audit tool and process. Supervisors and staff provided feedback in the same format through the survey and discussions to increase the ease of moving forward and to implement the proposed changes. This process helped with training and coaching, which occurred during staff meetings or on an individual basis.

The redesign team provided updates during staff meetings and by email to prepare staff for the changes that would occur with the enhanced audit. They also provided education about the revised peer audit tool during these updates. A trial of the revised peer audit tool began in April 2011. A questionnaire soliciting feedback on the peer audit tool was distributed in November 2011, and responses were requested by January 2012 to give staff the chance to use the peer audit tool at least once or twice. Changes were made in the second trial as a result of significant feedback on the first trial, and the clinicians were able to see that their feedback was incorporated into the revisions. The revised audit was completed in May 2012. The survey was repeated again in November 2012 and November 2013 after final implementation.

Staff participation was examined as an indicator of satisfaction with the peer audit tool and effectiveness of the changes made to the peer audit process. The redesign team expected a more meaningful peer audit process to increase clinician motivation and participation. Supervisors agreed to track peer audit completion dates for each staff member. The data for the number of peer audits each staff member completed per year and when the peer audits were completed during the year were analyzed before and after revision of the peer audit tool and process.

Results

A single peer audit tool (Supplemental Appendix A; available online at http://otjournal.net; navigate to this article, and click on “Supplemental”) and unified audit process were developed for use across therapy disciplines and care settings. The final version of the tool was implemented in May 2012 and continues to be in use today. The peer audit still includes an observation of a patient evaluation or treatment session, a review of the documentation, and a discussion between the auditor and the clinician being audited about clinical reasoning and therapy outcomes. Technical questions were omitted from the tool and replaced with thoughtful inquiries created to capture the thread of critical thinking and problem solving that occurs through the continuum of care. Examples include

New guidelines and recommendations were also created to accompany the peer audit.

The concept of a group audit was introduced and encouraged as an additional venue for collaboration, teamwork, and mentorship. The group audit does not include the observation but focuses on documentation review and a group discussion. Group audits can be done in a variety of ways to meet the needs of the clinicians at the time. One type of group audit could involve all clinicians in the group reviewing the medical record of the same patient and then having a group discussion about that particular case. A different group could choose to have each clinician in the group review the medical record of different patients with the same diagnosis and then have a group discussion about the similarities and differences in the clinical decision making and documentation.

Clinicians could still choose to complete technical audits and focused audits in addition to this new peer audit process. Supervisors can use a technical audit to monitor changes made in documentation templates, workflow, or regulations to ensure compliance. Clinicians can use focused audits for specific patient populations, specific interventions, or new standards of care. These technical and focused audits help clinicians improve practice in a defined area.

The pre- and postrevision survey results were analyzed. Agree and strongly agree results were grouped together as were disagree and strongly disagree to ensure the survey tool was sensitive enough to detect change. The percentage of positive responses (i.e., agree and strongly agree combined) and the percentage of negative responses (i.e., disagree and strongly disagree combined) were compared from the prerevision survey to the postrevision survey.

Responses to all questions indicated favorable change when comparing pre- and postrevision survey results (Table 1). The question regarding overall satisfaction showed the largest improvement. Favorable responses to this question doubled with an overall increase of 34%. Two of the targeted outcomes of the revised peer audit showed substantial improvement based on the survey results: encouraging critical thinking and promoting meaningful discussion. The smallest change occurred in the question regarding improvement in documentation. This finding may be because the focus of the peer audit revision was on improving the quality of patient care, with the indirect focus on documentation better reflecting patient care.

The survey tool provided opportunity for narrative feedback from participants. Several strong themes were clear in the narrative comments:

The number of peer audits required per year decreased from nine to four to compensate for the increased length and intensity of the new audit. The change in the number of required audits made it difficult to do a direct comparison of completion patterns and rates pre- and postredesign. Overall, there is no observable change in spreading audit activity throughout the year.

Discussion

The objective of the redesign team to review and analyze the previous peer audit process with the aim of designing a new approach was achieved. The redesign team was empowered to rethink the role of the peer audit, putting the emphasis on patient care. The new tool and process shifted the focus away from finding errors in documentation and toward clinical reasoning, using documentation as a reflection of patient care. Therapy leaders wanted the peer audits to result in meaningful discussions that clinicians found clinically relevant, and not just view the audits as an annual performance review requirement. The survey results as well as the comments from clinicians show that the audit changes support this shift of focus.

The redesign team engaged the clinicians in the project by providing opportunities for feedback throughout the process. The feedback gathered from staff was valuable in making additional revisions of the audit tool after the initial revisions were completed. Staff were able to see that their feedback was incorporated into the new revision, which may have reinforced their motivation to continue to improve their clinical skills, including documentation (Axelrod & Axelrod, 2001).

The development of the group audit was consistent with principles of social motivation (Patterson et al., 2008), and the high utilization of the group audit was an unexpected benefit of this project. Staff appear to be finding greater interest in the discussion at the group audit, and the number of individual and group audits has been modified based on staff feedback. Variation exists in how group audits are completed, but these audits do not typically include an observation of a patient evaluation or treatment session. This lack of observation does not seem to be detrimental to the discussion during the group audit. Further research is needed regarding the balance between individual and group audits and the value of each type of peer audit.

Limitations

This case report was based on the experience at one facility. The conclusions are not based on statistical significance, and the results cannot be generalized in the same manner as a research study. The relationship between improved satisfaction with the audit process and changes in clinical practice was not investigated.

Implications for Occupational Therapy Practice

This case report has the following implications for occupational therapy practice:

Conclusion

This case report describes the effort to transform a peer audit process that had low participation rates, clinician satisfaction, and relevance to clinical skills or discussion. A different format was developed that significantly changed the overall perception of peer audits by engaging clinicians in all stages of the redesign. The peer audit has become more valuable and meaningful to clinicians by improving its content and focus on clinical reasoning and discussion (rather than on technical material) as indicated in the survey results. Despite the increased length of the new audit form, clinician perceptions regarding taking time to complete the audit and meet the annual requirement showed favorable improvement in the survey results. These findings suggest that designing an engaging group process facilitated achieving the desired outcomes of this effort by creating a more participative, meaningful, and clinically relevant audit process.

Supplemental Material