Development and Refinement of the American Occupational Therapy Association’s Knowledge Translation Toolkit

Explore the Needs of Toolkit Users

A substantial body of literature supports the critical importance of evaluating the needs of end users in the development of any KT strategy, such as the KT Toolkit (Camden et al., 2015; Powell et al., 2019; Schippke et al., 2017). Our first step toward understanding the needs of occupational therapy practitioners for using EBP was to turn to the literature. A rapid literature search indicated that common barriers to EBP implementation include a lack of time to find and interpret evidence, a lack of resources to support EBP use, difficulties adapting evidence to clients’ unique needs, and insufficient support from administrators (Funabashi et al., 2012; Harding et al., 2014; Juckett et al., 2020; Samuelsson & Wressle, 2015).

To confirm the saliency of these barriers, we then solicited input from occupational therapy practitioners via an online survey. Specifically, we asked the following question: “What barriers make it challenging for you to implement evidence-based assessments/interventions in your clinical practice?” Participants were able to select as many items from the following barrier checklist as they found applied: “lack of resources/equipment/space,” “lack of time to stay updated on evidence,” “lack of administrative support,” “difficulty understanding research findings,” “difficulty accessing research articles and databases,” “lack of confidence implementing evidence-based practices,” “lack of incentive to provide evidence-based care,” “limited relevance to my clients’ needs,” and “other.” Respondents who selected “other” were able to insert narrative comments describing the barriers they experienced relative to EBP implementation. This survey was disseminated through AOTA communication channels (e.g., CommunOT, OT Practice Pulse, and Twitter) and was open to both members and nonmembers from August to September 2019.

Incorporate Frameworks

To further understand the EBP implementation barriers identified by our survey sample, we sought to transform identified barriers to match language from the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009). The CFIR is a taxonomy of 39 constructs that represent multilevel factors influencing the extent to which EBPs can be implemented. For instance, the “lack of time to stay updated on evidence” is a barrier that could be indicative of a lack of “available resources,” a CFIR construct. Categorizing barriers on the basis of CFIR constructs allowed our team to assign common nomenclature to major EBP implementation barriers and informed our selection of KT Toolkit content that could address the barriers perceived by our respondents.

Search for Existing Tools

Given that we categorized our EBP implementation barriers according to the CFIR, we accessed the publicly available CFIR-to-implementation strategy matching tool. This tool allows users to select key implementation barriers (on the basis of CFIR constructs) on an autopopulated spreadsheet and matches those barriers to recommended implementation strategies endorsed by a panel of implementation scientists (Waltz et al., 2019). Further information about the CFIR-to-implementation strategy matching tool can be found at https://cfirguide.org/choosing- strategies.

Data Analysis

Two sources of data were analyzed to prioritize content to be included in the KT Toolkit: (1) online survey data from occupational therapy practitioners and (2) output data from the CFIR-to-implementation strategy matching tool. Survey results were analyzed using descriptive statistics, and any open-ended responses were analyzed using direct content analysis with codes mapped to CFIR constructs to provide a uniform language for EBP implementation barriers. After EBP implementation barriers had been transformed into CFIR terminology (Figure 1), they were inserted into the CFIR-to-implementation strategy matching tool. The top 10 recommended implementation strategies (e.g., “identify and prepare EBP champions”) were then used to inform KT Toolkit content. These strategies were presented to all KT Toolkit team members to ensure they aligned with AOTA priorities.

OPEN IN VIEWER

Figure 1.

Development phase of the Knowledge Translation Toolkit.

Results of the Practitioner Survey

Respondents included 818 occupational therapy practitioners (Table 1). Experience levels ranged from <5 yr (18.0%) to >30 yr (17.7%). Half (50.7%) of practitioners indicated that a master’s degree was their highest level of occupational therapy education, followed by the bachelor’s degree (27.2%). The two most commonly represented practice areas were children and youth (40.6%) and rehabilitation and disability (34.0%).

Although a unanimous agreement was not reached across respondents on the barriers to implementing EBPs (Table 2), the barriers that were most commonly identified included “lack of time to stay updated on evidence,” “lack of resources/equipment/space,” “difficulty accessing research articles and databases,” and “lack of administrative support.” Many of the narrative comments also mapped onto these barriers. For instance, one respondent noted, “[My] employer does not pay employees for extra work put into evidence-based research for interventions (planning, preparing, completing the research all to become better therapists) even though it is a part of our career.” Similarly, respondents described a “lack of resources/equipment/space” as a major factor influencing the use of EBPs. As stated in a narrative comment, “the lack of financial resources to purchase researched assessments, programs, tools, [and] products” served as a barrier to EBP implementation, and respondents also reported limited resources to attend continuing education courses for specific EBPs. For example, one respondent noted that their EBP implementation efforts were limited because of “difficulty accessing training to achieve [a] level of comfort with [a] new treatment approach.” Respondents also described difficulty accessing research articles and databases because of cost and suggested that practitioners should have free access to a broad range of journals.

Even when practitioners were able to identify appropriate EBPs to implement, they reported additional barriers when attempting to adapt the intervention, program, or strategy in their setting and with their individual clients. This outcome resulted from both “difficulty understanding research findings” and “relevance to my clients’ needs.” For example, one respondent reported, “Research is difficult to understand because statistics and research are not emphasized in OT [occupational therapy] school.” Others critiqued the relevance of research findings on the basis of their individual clients’ needs:

The environment [my clients] live in, their attitudes, motivation, and capabilities, all impact the relevance and suitability of implementing interventions with them. Any intervention must also be sustainable. . . . Many clients have more than one health condition which impacts their occupations, and pure, high-level evidence is often not relevant to them and their comorbidities/situation.

Results of the CFIR-to-Implementation Strategy Matching Tool

Findings from our practitioner survey were consistent with prior research and revealed the following major barriers to EBP implementation: (1) lack of resources (e.g., time) and equipment, (2) difficulties accessing research journals and resources, (3) lack of administrative support, and (4) limited relevance to clients’ needs (Samuelsson & Wressle, 2015; Weng et al., 2013). On the basis of these barriers, the CFIR-to-implementation strategy matching tool identified 31 different strategies that could potentially mitigate implementation barriers. The top 10 of these strategies included (1) conducting educational meetings, (2) identifying and preparing EBP champions, (3) capturing and sharing local knowledge, (4) developing educational materials, (5) promoting adaptability, (6) creating learning collaboratives, (7) accessing new funding, (8) assessing for implementation readiness, (9) distributing educational materials, and (10) conducting local consensus discussions.

In an effort to maximize the understandability of these implementation strategies for practitioners, two members of the KT Toolkit team further condensed these strategies into the following four groups: (1) getting started with EBP, (2) changing your own practice, (3) sharing knowledge, and (4) influencing change at your organization. The process of transforming barriers into CFIR nomenclature and condensing implementation strategies into these four groups is summarized in Figure 1.

Involve Experts and Clients in Reviewing Content

Our KT Toolkit development team included members who represented the following practice areas: children and youth, developmental disabilities, productive aging, rehabilitation and disability, and health and wellness. These team members developed and reviewed all KT Toolkit content internally before sharing content with experts in each of these practice areas as well as experts in mental health, work, and industry, all of whom were external to our KT Toolkit team. These experts were selected through a call for volunteers and by collaborating with AOTA volunteer member groups. Input from these experts, and occupational therapy clients, allowed us to gain valuable insight into the practical application of our KT Toolkit resources and helped us identify additional resources we might have overlooked in our initial development phase.

Ensure That Content Is Understandable and Culturally Appropriate

We also leveraged our collaborations with AOTA volunteer groups to recruit occupational therapy stakeholders who could assess the understandability and cultural appropriateness of KT Toolkit content as it applies to their specific practice areas. We ensured that these stakeholders represented diversity in areas such as personal and professional background, ability level, clinical experience, and familiarity with EBP. It was essential for us to gather input about the usability of the KT Toolkit as perceived by practitioners, as well as by administrators and managers, given that the latter are often well positioned to coordinate EBP efforts and quality-improvement initiatives (Birken et al., 2018).

Pilot Test the Toolkit

We refined the KT Toolkit content, in preparation for pilot testing, on the basis of feedback from experts and clients. Our KT Toolkit team members who had access to various practice and geographic areas across the United States led efforts to introduce the revised KT Toolkit to practitioners who had not yet made contributions to its development. In pilot testing, we collected data that represented practitioners’ perceptions of the usefulness of the KT Toolkit and their intention to adopt the KT Toolkit in the future.

OPEN IN VIEWER

Figure 2.

Refinement phase of the Knowledge Translation Toolkit.

Test the Effectiveness of the Toolkit

Although toolkits similar to ours have been received favorably across interdisciplinary fields (Hempel, O’Hanlon, et al., 2019), systematic efforts to evaluate the effectiveness of these toolkits are needed (Yamada et al., 2015). Although pilot testing of the KT Toolkit was useful during the refinement phase, we are in the planning stages of evaluating the KT Toolkit’s effectiveness since it was made publicly available to AOTA members in July 2021. We plan to first track interest in adopting the KT Toolkit as measured by the frequency with which it is accessed. We then anticipate partnering with occupational therapy practice leaders (e.g., supervisors, administrators) to evaluate the KT Toolkit’s effectiveness in influencing practice behaviors and associated client outcomes across multiple settings. As we continue to evaluate KT Toolkit effectiveness, we also plan to further examine the barrier of translating results from clinical trials to complex client populations and to assess the role of EBP champions and clinical experts in addressing this pervasive implementation barrier.