Abstract
Keywords
Evidence-based practice (EBP) has moved from an emerging trend to an accepted framework for demonstrating the effectiveness of occupational therapy interventions and improving client outcomes (Upton, Stephens, Williams, & Scurlock-Evans, 2014). Evidence-based practice is generally defined as an integrated approach to clinical decision making that is based on the client’s values and preferences, the clinician’s expertise, and recommendations from the best and most current evidence available (Bennett et al., 2003).
Postprofessional educational opportunities (e.g., face-to-face workshops and journal clubs; Lizarondo, Grimmer-Somers, Kumar, & Crockett, 2012, McCluskey & Lovarini, 2005; McQueen, Miller, Nivision, & Husband, 2006) and formal continuing education (e.g., obtaining an advanced degree) have frequently been cited as viable strategies for increasing occupational therapy practitioners’ skills in applying EBP in real-life clinical scenarios (Bennett et al., 2003; McCluskey & Lovarini, 2005; Thomas & Law, 2013). However, little research has focused primarily on supporting the development of practitioners’ EBP knowledge and skills and implementing this process in practice (see, e.g., Lizarondo et al., 2012; McCluskey & Lovarini, 2005).
School-based occupational therapy practitioners are increasingly challenged to demonstrate positive outcomes for children with and at risk for disabilities. Recent educational reforms, such as the Individuals With Disabilities Education Improvement Act of 2004 (Pub. L. 108–446) and the No Child Left Behind Act of 2001 (Pub. L. 107–110), emphasize the significant role that EBP should play when making clinical and instructional decisions in the schools (Cook & Odom, 2013). Therefore, we conducted a study to investigate the effectiveness of a 17-mo EBP professional development initiative developed by key stakeholders of a special education cooperative in northern Illinois, occupational therapy leadership from the cooperative (Cheryl Huber-Lee and Margret Dess-McGuire), and university faculty from a nearby occupational therapy program (Susan M. Cahill, Brad E. Egan, and Minetta Wallingford).
Method
Research Design
We used a single-group pretest–posttest research design. Participation in the pretest and posttest measures was voluntary. We obtained ethical approval for this study through the university’s institutional review board.
Participants
We recruited occupational therapy practitioners from one special education cooperative in the Chicago metropolitan area. Participants were excluded from the study if they were not affiliated with the cooperative for the entire length of the study or if they did not submit both pretest and posttest responses. All participants provided informed consent.
Measures
Adapted Fresno Test.
We assessed practitioners’ EBP knowledge and skills using the Adapted Fresno Test (AFT), a measure developed by McCluskey and Lovarini (2005) that consists of seven questions about two clinical occupational therapy scenarios. The AFT examines a participant’s ability to write a PICO (population, intervention, comparison, outcome) formula question, document a search strategy, identify an appropriate study design to best answer a clinical question, discuss the interpretation and validity of research studies, and develop parameters to judge the clinical and statistical significance of the findings (McCluskey & Bishop, 2009). Possible scores range from 0 to 156 points (McCluskey & Lovarini, 2005). A higher score reflects better EBP knowledge and skills. The AFT has been found to have good content and construct validity, good interrater reliability, and moderate internal consistency (Cronbach’s α = .74; McCluskey & Bishop, 2009). The AFT has also been found to be a sensitive measure of change in EBP competence after workplace trainings (McCluskey & Bishop, 2009).
Survey.
We assessed the occupational therapy practitioners’ perceptions using a 15-item survey developed by Bennett et al. (2003) to examine the views of occupational therapists on EBP. The survey used a 5-point Likert scale to assess confidence and perceptions related to perceived challenges and barriers to using evidence in practice.
Professional Development Activities
The EBP initiative comprised several activities carried out between September 2012 and February 2014 (17 mo). The activities included in the initiative were the result of a systematic brainstorming session held with practitioners employed at the cooperative at the start of the 2012–2013 academic year. The brainstorming session was led by Cheryl Huber-Lee and guided by the McNellis Compression Planning Process (McNellis, 2003), a systematic way of capturing a group’s ideas and developing action steps. After the brainstorming meeting, we constructed a timeline and developed a series of activities to include in the initiative based on the practitioners’ preferences.
We began the initiative by dividing the occupational therapy practitioners into 8 small work groups of 5 or 6 people and asking them to select a topic of study based on clinical problems previously identified by the cooperative’s occupational therapy department. Over the course of the initiative, each group developed a Best Evidence Statement (Cincinnati Children’s Hospital Medical Center [CCHMC], 2013) with a practice recommendation based on a thorough and systematic review of the literature. Additionally, each group made a PowerPoint presentation describing their findings and shared an electronic collection of articles from their search. Final projects were housed in the department’s resource library. The practitioners also attended three professional development workshops (3 hr each) that took place onsite at the cooperative and were led by authors Cahill, Egan, and Wallingford. The practitioners also had unlimited access to an online self-study module (i.e., review of content, practice opportunities, and quiz) created by Egan and offered on SurveyMonkey (SurveyMonkey, Palo Alto, CA). Throughout the initiative, Cahill, Egan, and Wallingford provided technical assistance to the occupational therapy practitioners. Practitioners sought assistance with developing and revising PICO questions and search strategies, obtaining electronic versions of articles, and synthesizing the literature.
Data Collection
Data were collected at pretest and posttest using SurveyMonkey. Both the AFT and the survey were provided to participants in this format.
Analysis
The pretest and posttest AFT responses were scored using the rubric validated by McCluskey and Bishop (2009). Cahill, Egan, and Wallingford independently scored each of the AFT responses and then met to discuss and reconcile their scoring decisions until 100% agreement was achieved on all AFT items (Peacock & Ward, 2006). The AFT raw scores were entered into IBM SPSS Statistics Version 19.0 (IBM Corporation, Armonk, NY), and a matched-pairs t test was performed with a significance level set at .05. Differences in mean scores (d score; posttest minus pretest) were calculated by hand. Descriptive statistics were calculated for items on the survey.
Results
A total of 29 practitioners completed both the pretest and the posttest. The majority of the participants were occupational therapists with a master’s degree (62.1%, n = 18) who had practiced between 8 and 15 yr (44.8%, n = 13). Table 1 provides participants’ demographic information.
Participant Demographic Information (N = 29)
Adapted Fresno Test Results
The matched-pairs t test (N = 29) revealed a significant difference in scores on the pretest (M = 43.9, SD = 32.67) and the posttest (M = 74.66, SD = 33.99), t(28) = −5.645, p < .001. The point differences (d score) between the pretest and posttest scores for the matched pairs ranged from −17.00 to 104.00 (M = 30.83, SD = 29.28).
Survey Results
Table 2 presents data on the practitioners’ perceptions related to EBP, and Table 3 presents data on the practitioners’ confidence in their EBP skills. Most participants agreed that research was useful in the day-to-day management of their clients and that EBP improved client care. At pretest, fewer than half of the participants were confident in determining the significance of a study’s results and using electronic databases. At posttest, the participants reported higher levels of confidence in determining the significance of a study’s results and using electronic databases. In addition, at posttest more than half of the participants reported confidence in searching the literature and evaluating the validity of a study.
Practitioners’ Perceptions Related to EBP (N = 29)
Note. EBP = evidence-based practice.
Practitioners’ Confidence in Their EBP Skills (N = 29)
Note. EBP = evidence-based practice.
Table 4 presents data on the practitioners’ perceptions related to the barriers they encountered related to EBP. On the pretest, the majority of the participants identified lack of time, lack of available evidence, and limited access to the literature as barriers to implementing EBP. Similar numbers of participants identified these same barriers at posttest.
Practitioners’ Perceptions of Barriers to EBP (N = 29)
Note. EBP = evidence-based practice.
Discussion
The purpose of this study was to examine the effects of a collaborative professional development initiative on school-based occupational therapy practitioners’ perceptions, knowledge, and skills related to EBP. The overall results of this study suggest that the initiative may have been effective in increasing the participants’ EBP knowledge and skills and in maintaining participants’ positive view of evidence as supportive of best practice.
Increasing EBP Knowledge and Skills
The professional development activities associated with the initiative produced significant (p < .001) increases in participants’ EBP knowledge and skills as measured by the AFT, with an average of >30 points difference between pretest and posttest scores. This finding is consistent with other studies examining the effects of targeted professional development activities designed to improve participants’ knowledge and skills as measured by the AFT (Lizarondo et al., 2012; McCluskey & Lovarini, 2005). This study yielded a slightly higher mean point difference (30.83) compared with those of McCluskey and Lovarini (2005; 20.6) and Lizarondo et al. (2012; 22.75), which may suggest that the combination of professional development activities used in this study was more effective in increasing EBP knowledge and skills than the activities studied previously.
Maintaining Participants’ Positive Viewpoints
Participants’ perceptions of their confidence in their EBP skills increased overall. This finding is consistent with those of other studies that examined the effects of focused professional development opportunities on participants’ confidence regarding EBP (Thomas & Law, 2013). The greatest increases in confidence in this study were seen in items related to searching the literature, determining the significance of a study’s results, and using electronic databases. Findings from McCluskey and Lovarini (2005) suggest that critical appraisal continues to be challenging after initial EBP instruction. However, the present initiative included a significant amount of guided practice that occurred in small groups and was related to the creation of Best Evidence Statements (CCHMC, 2013). It is possible that this practice time may have positively influenced participants’ confidence in determining the clinical significance of studies.
Encouraging the Adoption of EBP Behaviors
Changes in practitioners’ knowledge and perceptions have been cited in the literature as important prerequisites to the adoption of EBP behaviors (Forsetlund et al., 2003; McCluskey & Lovarini, 2005; Thomas & Law, 2013). However, major changes at the organizational level are thought to be necessary to encourage practitioners to prioritize EBP and change their everyday work habits and routines (McCluskey & Lovarini, 2005). The current EBP initiative was developed through collaboration between key stakeholders from the cooperative and university faculty. In addition, the occupational therapy practitioners who participated in the initiative provided insights during the early stages of planning that greatly contributed to the design of the professional development activities and to the timeline. The length of time of the initiative (17 mo) and the support from the organization’s administration are two key factors that also likely encouraged positive results.
Limitations
Specific factors related to the organization’s culture (e.g., including the participants in brainstorming the professional development methods) and the occupational therapy department’s leadership likely significantly influenced the results of this study; therefore, these results may not be directly generalizable to other work settings. It is possible, however, that contact with university faculty and work on an EBP project were by themselves sufficient to result in the described changes. Finally, the repeated administration of the AFT may have produced a learning effect in some of the participants.
Implications for Occupational Therapy Practice
The findings of this study have the following implications for occupational therapy practice:
Occupational therapy managers who value EBP should consider collaborating with local universities to offer a series of professional development activities targeted at increasing practitioners’ EBP knowledge and skills. Such professional development activities should be focused on addressing the evidence needs of the specific practice setting.
Professional development activities should be strategically timed and designed to provide the just-right level of challenge to participants.
Didactic instruction combined with opportunities for guided practice should be considered.
Conclusion
Evidence-based practice is expected to play an increasingly central role in school-based occupational therapy practice. The benefits of EBP are widely accepted and critical elements of best practice. The results of this study suggest that a series of professional development activities embedded in a supportive organizational culture can positively influence practitioners’ perceptions of EBP and their EBP knowledge, skills, and behaviors.