Adapting a Technology-Based Implementation Support Tool for Community Mental Health: Challenges and Lessons Learned

Abstract

With mental health services shifting to community-based settings, community mental health (CMH) organizations are under increasing pressure to deliver effective services. Despite availability of evidence-based interventions, there is a gap between effective mental health practices and the care that is routinely delivered. Bridging this gap requires availability of easily tailorable implementation support tools to assist providers in implementing evidence-based intervention with quality, thereby increasing the likelihood of achieving the desired client outcomes. This study documents the process and lessons learned from exploring the feasibility of adapting such a technology-based tool, Centervention, as the example innovation, for use in CMH settings. Mixed-methods data on core features, innovation–provider fit, and organizational capacity were collected from 44 CMH providers. Lessons learned included the need to augment delivery through technology with more personal interactions, the importance of customizing and integrating the tool with existing technologies, and the need to incorporate a number of strategies to assist with adoption and use of Centervention-like tools in CMH contexts. This study adds to the current body of literature on the adaptation process for technology-based tools and provides information that can guide additional innovations for CMH settings.

Keywords

mental health technical assistance Internet/electronic interventions technology training

Despite availability of mental health (MH) evidence-based interventions (EBIs) designed to promote well-being, there remains a gap between these practices and the care routinely delivered in real-world settings (Levin, Hennessy, & Petrila, 2010). Previous research indicates that many individuals do not receive services that adhere to best practices, leaving them to struggle with mental health issues (Druss, Miller, Rosenheck, Shih, & Bost, 2002; Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004). With community mental health (CMH) organizations (e.g., CMH centers, outpatient clinics) becoming increasingly responsible for addressing the MH crisis (Levin et al., 2010), identifying strategies to accelerate adoption and quality implementation of MH EBIs in these settings is particularly critical (Proctor et al., 2009).

Use of multifaceted implementation supports designed to assist MH providers with delivery of EBIs is one promising approach to bridging the research-to-practice gap (Leeman et al., 2015; Powell, Proctor, & Glass, 2014; Schoenwald & Hoagwood, 2001). Implementation supports include readily available resources and training, data monitoring to ensure EBI adherence and adequate client progress, and cost-effective technical assistance. Access to these supports has been associated with successful implementation of MH EBIs, which in turn increases the likelihood of achieving the desired client outcomes (Durlak & DuPre, 2008; Leeman et al., 2015; Mitchell, Florin, & Stevenson, 2002; Wandersman, Chien, & Katz, 2012).

Although effective, these types of supports are typically time- and resource-intensive for CMH providers. In an attempt to reduce these barriers, recent efforts have focused on use of technology to deliver these supports (Anderson, 2006; Bickman, Kelley, Breda, De Andrade, & Riemer, 2011; Jensen-Doss & Hawley, 2010; Levin et al., 2010). Unlike traditional approaches (e.g., in-person trainings, on-site assistance), digitized implementation support delivery can maximize anytime accessibility, centralize EBI resources, and automate data manipulations.

However, to be scalable, these technology-based implementation support systems need to be adapted to improve compatibility with each novel setting and provider population. Tailoring existing systems is more cost-effective and less time-consuming than developing new digital tools. Adaptation is “the process of making changes to an [innovation] to make it more suitable for a particular population or organization’s capacity without compromising or deleting its core components” (Centers for Disease Control and Prevention, 2010). An adapted innovation (whether a tool, technology, program, or practice) is more likely to a become adopted (Rogers, 1995). Unfortunately, the adaptation process is not well understood nor has it been extensively researched, particularly for technology-based tools designed to facilitate MH EBI implementation efforts (Aarons et al., 2012; Cabassa & Baumann, 2013; Lee, Altschul, & Mowbray, 2008).

Previous research on the adaptation process has been limited in scope, focusing mostly on the tailoring process for one type of innovations, EBIs, and a specific category of adaptations, that is, tailoring for clients of diverse cultural backgrounds (Cabassa & Baumann, 2013; Tu et al., 2014). This project focuses on adaptation of a different type of innovation, a technology designed to support EBI implementation, for a different category of users, providers in CMH settings. Additionally, previous studies tend to document retrospective adaptations, rather than describe a planned process with intent to use the information collected to guide the adaptation (Aarons et al., 2012; Cabassa & Baumann, 2013; Lee et al., 2008). Illustrations of prospective adaptations is crucial to broadening understanding of this process.

Further limiting the utility of current research is an emphasis on the attributes of the intervention, rather than the broader organizational context, as determinants of the adaptation process (Aarons et al., 2012; Bowen et al., 2010; Chor, Wisdom, Olin, Hoagwood, & Horwitz, 2015). Understanding the organizational context is necessary to facilitate fit of the innovation, whether this knowledge results in decisions to modify the innovation to accommodate the setting, removal of contextual barriers, or changes in organizational conditions to support use. This study exemplifies a planned adaptation process, with consideration given to the implementing context, the CMH setting.

Finally, due to the dearth of theoretical frameworks that have been operationalized into practical sets of guidelines for use in real-world adaptation efforts (Aarons et al., 2012; Dolcini et al., 2010), few studies make explicit the process and methods used for adapting an innovation (Carvalho et al., 2013). To our knowledge, there is only one recently published framework, the Contextualized Technology Adaptation Process, that was specifically developed to guide adaptation of technologies to novel health care settings (Lyon et al., 2016). Further clarifying the operationalization of the steps and methods used at different stages of the adaptation process for different types of innovation, including technologies, has been recognized as an important direction for future research. These gaps and limitations highlight the need for further exploration of the adaptation process to accelerate use and scalability of available implementation support technologies for CMH.

The purpose of this article is to contribute to this sparse literature by (1) illustrating an application of the first phase of a planned and purposeful adaption of a digitized implementation support system, Centervention, for CMH, and (2) documenting lessons learned from this initial feasibility exploration. This exploration involved completion of the first two steps of the adaptation process, focused on understanding both the technology user and the technology–context fit (see Method section; Aarons et al., 2012; Lyon et al., 2016). Briefly, Centervention (3C Institute) is an online tool that was originally designed to support implementation of school-based EBIs to promote students’ socioemotional competency (see Method section). Centervention was selected as the example digitized implementation support system for this study for practical reasons. The Centervention purveyor, the 3C Institute, had an interest in understanding whether this tool could be tailored for use in CMH settings. As implementation science researchers, the article authors were interested in exploring the adaptation process for implementation support technologies.

Beyond informing the feasibility of adapting a specific tool, Centervention, this descriptive study serves two important purposes. First, it exemplifies the application and operationalization of a planned adaptation process that could be replicated by purveyors and others with an interest in tailoring their technology-based tools for CMH. Second, it provides valuable insights related to the delivery of implementation supports via technology in CMH settings. To our knowledge, this is one of the first studies examining the planned adaptation of an implementation support digitized tool, with considerations given to the innovation itself, its potential user, and its adoptive setting. Gaining a better understanding of the adaptation process for technology-based support tools is key to facilitating the successful translation of MH research into practice. Digitized implementation support tools that are customized to the needs of CMH providers and settings are more likely to be used, and therefore positively affect the quality of EBI implementation and ultimately the likelihood of achieving positive MH outcomes.

Method

Overview

This adaptation feasibility study was conducted with a convenience sample of 44 MH providers in Missouri. The study data were gathered using a mixed-methods approach to ensure in-depth understanding of the key considerations for adapting a digitized implementation support tool. This analysis used a sequential balanced design for the purpose of elaboration, with the qualitative data set nested in the quantitative study (Palinkas et al., 2011). The quantitative data obtained via surveys produced information on adaptation feasibility around the technology’s core features and perceived fit with the providers’ work and expectations, as well as contextual fit. Follow-up interviews with a subset of 10 participants generated in-depth information about feasibility of adaptation and recommendations for facilitating use of the technology in CMH settings. Analysis of the mixed data resulted in lessons learned around feasibility of adapting the innovation features, perceived fit with CMH providers, and contextual considerations. Data were collected over a 3-month period, following IRB approval.

Operationalizing the Exploration Stage for Adaptation: Study Framework

With the understanding that successful adaptation involves a balance between keeping an innovation’s core elements intact and taking into consideration the realities of the implementing setting, the exploration stage is designed to (1) gain an understanding of how the innovation is perceived by potential users through an evaluation of the unadapted technology (Step 1) and (2) obtain information on the organizational context in which the innovation will be implemented (Step 2). These two steps are aligned with current adaptation frameworks, including the Dynamic Adaptation Process (Aarons et al., 2012) that initially guided the operationalization of the adaptation process for this study, and the Contextualized Technology Adaptation Process that was only recently published (Lyon et al., 2016). The information compiled during this stage should then be used to tailor the technology and conduct usability testing of the adapted technology.

In this study, evaluating the unadapted technology (Step 1) was achieved by assessing the providers’ perceptions of both the innovation’s core features and its perceived work fit (Chor et al., 2015; Marangunić & Granić, 2015). Core features are defined as either “content” (the “what”— the Centervention core features, see below) or “pedagogical” (the “how”—technology-based delivery; Galbraith et al., 2011). Assessment of work fit was based on Rogers’s (1995) perceived innovation characteristics. Complexity (ease of use), perceived usefulness, relative advantage, and compatibility have been identified as the most significant attributes for adoption and use of technology-based innovations (Rogers, 1995; Venkatesh & Bala, 2008).

To better understand the context of CMH settings and their capacity to adopt a support tool like Centervention (Step 2), organizational capacity (i.e., general attributes of an organization and conditions important for implementing a particular innovation) was also assessed (Livet & Wandersman, 2005). With the broader organizational context as a critical precursor of adoption, implementation, and sustainability (Aarons, Hurlburt, & Horwitz, 2011), this information was necessary to further guide the needed adaptations.

Centervention as the Example Innovation

Centervention (3C Institute) was originally developed to support broad scale implementation of MH EBIs by school-based providers (Supplemental Appendix A, available in the online version of this article). Providers have access to four core implementation support components found to facilitate quality EBI delivery (Wandersman et al., 2012; Leeman et al., 2015). These components, tailorable to any MH EBI, include the following: (1) downloadable resources, including program manuals and materials (Resource Center); (2) online trainings, including competency quizzes and continuing education credit; (3) online and live technical assistance, for both EBI implementation and use of Centervention; and (4) quality assurance/improvement tools for adherence tracking (allowing providers to monitor their fidelity to the EBI) and progress monitoring (to track client progress). Thus far, Centervention has been used only in school settings by counselors and teachers in North Carolina, Florida, and Chicago to support quality implementation of EBIs designed to promote the socioemotional competency of students. Its usefulness and utility in predicting EBI adherence and indirectly student socioemotional outcomes have been supported through two previous studies conducted with this user population (Livet, Yannayon, Kocher, & McMillen, 2017; Livet, Yannayon, Sheppard, et al., 2017). Supplemental Appendix B (available in the online version of this article) describes the MH interventions that had originally been incorporated into Centervention for use by school providers.

Participant Sample

CMH participants were recruited through the Missouri Therapy Network, a community of practice platform for therapists in Missouri. To be included in the study, participants had to identify themselves as a CMH provider or administrator and have worked at least a year in their current setting. Of the 44 who were eligible, all completed the study, with 23% (6 providers, 4 administrators) agreeing to participate in an optional follow-up interview. This sample was deemed adequate based on subgroup sampling design recommendations (Onwuegbuzie & Leech, 2007). Sample demographics are summarized in Table 1.

Table 1

Demographics of Participants

Characteristic	n	%
Gender
Female	37	84.1
Male	7	15.9
Age range (years)
21-25	3	6.8
26-35	20	45.4
36-45	7	15.9
46-55	11	25
55+	3	6.8
Race
Caucasian/White	40	93.0
African American	3	67.0
Other	0	0
Education
BA or some college	1	2.3
MA or MS	38	86.4
PhD	5	11.4
Job role
Community mental health provider	35	79.5
Managerial role	9	20.5
Years of experience
1-5	16	36.4
6-10	9	20.5
11-15	2	4.5
16-20	9	20.5
20+	8	18.2
Work setting
Community mental health center/outpatient clinic	27	75.0
Private practice	7	19.5
Residential treatment facility/group home	2	5.6

Data Collection Procedures and Instruments

After consenting to the study, participants were given a written overview of Centervention and its core features and asked to explore and engage with the tool for a 2-week period. They then completed an online survey designed to assess three constructs: feasibility of the tool’s core features in a CMH context, perceived fit with the providers’ work and expectations (e.g., ease of use), and organizational capacity (e.g., openness toward change). Survey concepts and information are included in Supplemental Appendix C (available in the online version of this article). One-hour follow-up phone interviews were conducted by the article authors with those who indicated interest in participating. These semistructured interviews focused on obtaining more in-depth information about (1) perceptions of Centervention and suggestions for potential adaptations, (2) organizational facilitators and barriers to using the tool, and (3) recommendations to facilitate use in CMH settings. To ensure that participants had an opportunity to reflect ahead of time, interview questions were e-mailed at least a week prior to their interview date.

Data Analysis

The quantitative and qualitative results were merged to produce summaries of findings highlighting key learnings (see Results section). This data synthesis was organized around the three domains of interest: core features, perceived work fit, and organizational capacity.

Survey Data

Quantitative survey data were analyzed using SPSS Version 22. Cronbach’s alphas were first computed to assess internal consistency of the survey measures (Supplemental Appendix C). Descriptive statistics were then computed for each scale. The open-ended survey questions were analyzed without predetermined codes. Specific codes (e.g., “readily available assistance”) emerged after the first read of the comments, with these codes being applied during a second read-through by the first author.

Interview Data

Interview data were recorded, transcribed, and analyzed using content analysis (Hsieh & Shannon, 2005). Responses were organized conceptually and coded for patterns. Primary codes included likes, dislikes, and suggested changes related to both core features and perceived fit. Organizational facilitators, barriers, and suggestions for overcoming the barriers were also coded. The materials were then reread to identify emerging themes that became secondary codes (e.g., “attitudes toward technology” as an organizational barrier). A codebook was created as a result of this process, with the third read involving focused coding of transcripts. The coded texts were exported into a matrix allowing for comparison across participants and codes. Analysis was conducted by the first author, with quality assurance checks performed on 20% of the interviews by the second author. There were no significant disagreements between the two authors.

Results

Feasibility of Core Features: Delivery Through Technology Is Insufficient

Participants believed that the Centervention four core support components could be used in CMH settings without any substantive modifications (Table 2). However, while approximately 90% of survey participants indicated their organization felt comfortable around and interested in learning about technology (M = 4.3, SD = 0.83), more than half of the interviewees identified attitudes toward technology as a challenge. Interviewees recommended that delivery through technology be augmented by providing implementation support through “live” channels (e.g., 24/7 IT support).

Table 2

Feasibility of Centervention Core Features for Use in CMH settings: Survey Results

Centervention Component	% Participants Supporting Feature “As Is” (n)	Participant Concerns and Suggestions
Online EBP training	100 (4)	None
Technical assistance
Online (quick tips, Q&As, ask an expert)	100 (44)	None
Peer support (provider tips)	90.9 (40)	Build in functionality to ensure regular use and input of information by peers
Help center^a	88.6 (39)	Ensure that live help is also available
EBP resource center	100 (44)	None
QA/QI
Fidelity/adherence monitoring^a	84.1 (37)	Concerns about Centervention only being able to accommodate manualized EBIs
Progress monitoring^a	93.2 (41)	Concerns about the amount of data entry needed
Reports	100 (44)	None

NOTE: CMH = community mental health; EBP = evidence-based practice; QA/QI = quality assurance/improvement; EBI = evidence-based intervention.

Four of the five comments on the “help center,” four of the seven comments on “fidelity/adherence monitoring,” and one of the three comments on “progress monitoring” were made by participants who reported that the feature could not be applied “as is” to CMH settings because of technical issues (e.g., feature was too slow to load).

Fit With MH Providers’ Work: Efficiency, Integration With IT Infrastructure, and Customization Are Key

On average, based on the survey results summarized in Table 3, respondents agreed that Centervention would be easy to use and useful to enhance job effectiveness. Furthermore, 69% of survey participants reported that advantages of using the tool would exceed drawbacks. When asked about the benefits of Centervention, participants noted the capacity to easily access client progress data in a visually appealing report as the most noteworthy advantage, followed by readily available assistance. The most common drawback was the amount of time it would take to enter the necessary data.

Table 3

Perceived Fit With MH Providers’ Work: Quantitative Survey Results

Variable	n	M (SD)	Frequencies (%)
Variable	n	M (SD)	Strongly Disagree	Disagree	Agree	Strongly Agree
Ease of use	40	3.16 (0.44)	0	5.0	70.0	25.0
Perceived usefulness	38	3.06 (0.43)	0	7.9	76.4	15.7
Compatibility with work	39	2.61 (0.67)	2.6	38.5	46.1	12.8
Variable	n	M (SD)	Drawbacks Would Exceed Benefits		Benefits Would Be About Equal to Drawbacks	Benefits Would Exceed Drawbacks
Relative advantage	42	N/A	4.8		26.2	69

NOTE: MH = mental health.

While the tool was perceived as an “add-on” to other technologies currently in use (e.g., “I think it would probably be a more value-added piece, not necessarily replacement of what we’re currently using”), the need to integrate Centervention with existing IT systems was highlighted by interviewees (e.g., “The biggest thing is being able to tie it in with all of the other required systems that we use”).

Finally, almost 60% of survey participants’ responses agreed that Centervention was compatible with their line of work, while approximately 40% disagreed (see Table 3). Interviewees noted two main reasons for this challenge: (1) the need to focus on direct client services, for which they could bill their time, and (2) a desire for flexibility in providing client services, rather than focusing on fidelity to a manualized EBI. Customizing the tool to the needs of each organization and clearly communicating the purpose of the system were offered as potential solutions.

Organizational Capacity of CMH Settings: Challenges to Consider

Resistance to change: Survey participants described their organizational culture as being mostly “clan-oriented” (family-like, with a focus on mentoring and nurturing) (44%), followed by “hierarchical” (23%), “market” (21%), and “adhocracy” (12%). Only 25% of those surveyed perceived their organization as being open to change (see Table 4). Interviewees attributed this reluctance to fatigue, emphasizing that CMH organizations had undergone a number of transitions over the past few years (e.g., electronic health records). To overcome resistance, interviewees recommended highlighting the benefits (e.g., making one’s job easier, securing future funding) of a system like Centervention.

Inconsistent processes to facilitate innovation implementation: In addition to an inclusive but change-resistant culture, CMH organizations also seemed to have insufficient processes to facilitate adoption and use of innovations. Based on survey results (Table 4), nearly 55% of survey participants rated the item about leadership sharing a common vision for adopting and implementing innovations as “slightly” to “somewhat” true (Table 4). Based on a mean of 3.14, it was also “somewhat” true that respondents’ organizations had teams responsible for reviewing and implementing new practices. Finally, the presence of a plan to ensure accountability during implementation of the innovation was “slightly” to “somewhat” true for the majority of providers. Only 2 of the 10 interviewees described a formalized process for implementing innovations within their agency.

Varied levels of implementation capacity and professional development infrastructure: The existing workforce capacity and the professional development infrastructure were also inconsistently supportive of innovation adoption and implementation. Based on survey results (Table 4), assessment of collective knowledge and skills in CMH settings indicated that knowledge of implementation processes was somewhat lacking. However, survey respondents “somewhat” to “mostly” agreed their organization had formalized staff training processes in place, such as coaching. When asked about the workforce development infrastructure within their agency, half of the interviewees reported it to be rather “bare bones,” while the other half described a well-established training infrastructure. To address variability in the types of in-house supports available, participants recommended offering additional supports tailored to the specific needs of their organization along with the tool. These included formalized training on Centervention, options of training format (e.g., train-the-trainer), a liaison to facilitate communication between the CMH agency and the innovation purveyor, and access to ongoing assistance.

Limited data-driven decision making for improvement purposes: Learning from experience through ongoing and proactive use of data to make improvements was limited. On average, CMH organizations did make use of data “somewhat” to inform their decision making process (Table 4). When both items included in the Learning From Experience scale were examined separately, additional information emerged. Only 21% (N = 43) indicated that it was “completely true” there was a process for collecting intervention-related data, with this information being “reported at least annually.” The remaining 79% either noted that their organization did not have a process (14%) or expressed varied levels of confidence that their agency had some type of process but with the data reported less than annually (65%). Furthermore, only 18% (N = 42) reported that it was “completely true” that these data were used to build action plans to improve current practices. The remaining 82% indicated that these data were either not used in the decision making process (25%) or used to varying degrees but not for improvement purposes (57%). Interviewees shed light on these results by explaining that data were most likely used to make financial decisions or “deal with a request or proposal.” To encourage use of data for improvement purposes, it was recommended that this change be tied to funding.

Variable resources: The resources needed to support an innovation like Centervention were considered insufficient by 54.1% of survey respondents (Table 4). A lack of financial and staffing resources was raised as a potentially significant barrier during the interviews. Interviewees emphasized the need for the tool to be low-cost and to save time. Other recommendations included offering incentives to early adopters and including use of Centervention as a funding requirement. As one interviewee noted, “Our funders dictate what we do, it’s just the reality of the way we work.”

Importance of buy-in and return-on-investment: When asked about buy-in from both management and coworkers, the majority of survey participants believed it was “mostly true” that use of Centervention would be supported by management and “somewhat true” by coworkers (Table 4). Based on interview data, it was reported that buy-in would be limited by financial constraints and return-on-investment. In addition, survey participants’ level of confidence that there would be an advocate for such an innovation in their organization was fairly high (Table 4). Identifying an innovation champion was recommended by interviewees as a strategy to secure buy-in (e.g., “To get someone who is respected, trusted, and has been there for a while would be key”).

Table 4

Organizational Capacity of CMH Settings: Quantitative Survey Results

Variable	n	M (SD)	Frequencies (%)
Variable	n	M (SD)	Adhocracy	Hierarchy	Market	Clan
Organizational culture	44	N/A	12.00	23.00	21.00	44.00

Variable	n	M (SD)	Strongly Disagree	Disagree	Slightly Disagree	Neutral	Slightly Agree	Agree	Strongly Agree
Openness toward organizational change	44	3.90 (1.21)	2.3	6.8	29.5	36.4	13.6	9.1	2.3

Variable	n	M (SD)	Not True	Slightly True	Somewhat True	Mostly True	Completely True
Implementation support processes
Vision	42	2.64 (1.24)	19.1	33.3	21.4	16.7	9.5
Teams	43	3.14 (1.41)	18.5	14.0	23.3	23.3	20.9
Plan	42	2.79 (1.30)	23.8	14.3	31	21.4	9.5
Workforce capacity
Implementation knowledge	34	2.43 (1.41)	35.3	17.6	20.6	11.8	14.7
Professional development	39	3.31 (1.00)	5.1	15.4	38.5	28.2	12.8
Learning from experience	43	3.21 (1.39)	14.0	18.6	7.0	32.6	27.9
Management support	42	3.56 (0.90)	0	9.5	35.7	38.1	16.7
Staff support	42	3.03 (0.80)	0	23.8	50	16.7	9.6

Variable	n	M (SD)	Very Inadequate	Not Quite Adequate	Adequate	Very Adequate
Resource availability	37	2.35 (0.64)	2.7	51.4	40.5	5.4

Variable	n	M (SD)	Not at All	Once in a While	Sometimes	Fairly Often	Frequently, if not Always
Innovation champion	42	3.60 (0.71)	0	7.1	33.3	47.7	11.9

NOTE: CMH = community mental health.

Discussion

To accelerate translation of MH EBIs into practice, scalable tools designed to assist providers with EBI implementation need to be developed and adapted to fit the needs of various CMH organizations. The purpose of this article was to describe the process and lessons learned from exploring the feasibility of adapting an innovative technology-based implementation support tool, Centervention, for use in CMH settings. These findings are relevant for researchers, purveyors and providers alike. First, the study adds to the current body of literature on adaptation in the following ways: It illustrates the adaptation process for a digitized tool (rather than an EBI); it describes use of a purposeful and planful process to inform the adaptation (rather than investigating it retrospectively); it makes explicit the methodology used; and it expands the assessment of adaptation feasibility to include organizational considerations (in addition to innovation attributes and provider work fit). Second, the information generated through this study should be useful to guide tool innovations for CMH settings. As an illustration of the first two steps toward a purposeful adaption, this work highlights a practical methodology for those interested in adapting their technology-based innovation for CMH organizations. It also shares important lessons learned and recommendations that could be considered by those adapting digitized tools for CMHs.

Key Lessons Learned

Live assistance is a must: Technology can maximize EBI adoption and implementation; however, it has not been fully embraced by providers. Previous research suggests providers might be fearful of technology and tend to favor in-person interactions (Garrett et al., 2006). Complementary pedagogical features, such as access to live support, could facilitate adoption and implementation of such tools.

Tools need to be customizable to the specific user needs and integrable with the existing IT infrastructure: The identified challenges included the need to streamline work efficiency, make use of the existing IT infrastructure, and focus on billable hours. Recommendations to offer customization options to meet the specific needs of each organization and to integrate the tool with existing IT systems (e.g., electronic health records) are consistent with previous literature (Lyon & Lewis, 2016). In addition, with many CMH settings using billable hours as a proxy indicator of success, these tools should minimize burden on the user.

A set of strategies to facilitate adoption of technology innovations by CMH organizations need to be designed: CMH organizations were described as sharing an inclusive but change-resistant culture with insufficient capacity, inconsistent processes to support implementation of innovations, and limited use of data to inform improvement efforts. Availability of resources were also variable. Because these contextual variables affect adoption and implementation of an innovation (Chor et al., 2015; Meyers, Durlak, & Wandersman, 2012), it is crucial to design a set of strategies that will engage and assist CMH users with such tools. Of note were recommendations to clearly articulate the benefits and effectiveness of the tool to increase buy-in, obtain support at all organizational levels, provide incentives to stimulate adoption, work with outside change agents to incorporate use of these tools into funding streams, and provide supplemental services to facilitate adoption (e.g., in-person training). These recommendations align with previous research on strategies influencing innovation adoption, such as targeted communication, risk reduction through provision of financial incentives, capitalizing on external pressures (e.g., from funding sources), and augmenting technology-based innovations originally designed to be unsupported with additional assistance (Baron & Graham, 2007; Frambach & Schillewaert, 2002).

Study Limitations and Directions for Future Research

While these results yield important findings, limitations should be noted. First, while triangulation of data through use of mixed methodology served to validate findings, these data were based on a single perspective from each organization. Having multiple stakeholders from each organization participate would ensure accuracy of information. Second, generalizability of findings is limited by a small sample size of highly educated, mostly female Caucasian providers and a focus on one technology innovation. Replicability across user types and technologies will serve to strengthen current findings. Third, participants’ perceptions were based on a targeted exploration of Centervention. While this strategy might be sufficient to inform initial adaptation of the tool, further refinements should involve active use of the technology by CMH providers to support implementation of an EBI. Finally, and more importantly, this study focuses on the first phase of a purposeful adaption, namely, exploring feasibility. Future research should also examine the subsequent phases of the technology adaptation process, such as implementation, refinement, and sustainability (Aarons et al., 2012; Lyon et al., 2016).

Implications and Conclusion

With the MH field moving toward increased accountability through required use of EBIs, CMH providers need convenient, easy-to-use, low-cost tools that support effective EBI implementation in real-world practice. These tools should be easily adapted to new settings and users. The lessons learned from this study provide preliminary insights into the complex process of adapting a technology-based implementation support innovation. Gaining a better understanding of this process should help purveyors and other stakeholders more quickly and effectively tailor these tools to the needs of CMH providers. Implementation supports that are responsive to provider needs have increased usability and, as such, enhanced potential to result in quality implementation of MH EBIs. Quality implementation, in turn, leads to the increased likelihood that these EBIs will have a positive impact on clients’ well-being. Ensuring scalability of digital implementation support tools for CMH providers through effective adaptation is key to bridging the research-to-practice gap and, ultimately, achieving the desired client outcomes.

Footnotes

Acknowledgements

The authors wish to acknowledge the contribution of Stacy McCrary, at Washington University’s Center for Mental Health Services Research, who helped with recruitment of participants through the Missouri Therapy Network. The authors who conducted this study were employees of the 3C Institute at the time of the study.

The tool investigated in this study is commercially available. However, the data collected and described in this article do not impact direct sales, since the focus is on how to adapt the tool to suit the needs of a new group of users.

This research was supported by a grant from the National Institute of Mental Health (2R44MH086983-03).

Supplemental Material

Supplemental Appendices A–C are available in the online version of this article at .

References

Aarons

G. A.

Green

A. E.

Palinkas

L. A.

Self-Brown

Whitaker

D. J.

Lutzker

J. R.

. . . Chaffin

M. J.

(2012). Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science, 7(1), 32.

Aarons

G. A.

Hurlburt

Horwitz

S. M.

(2011). Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38, 4-23.

Anderson

N. B.

(2006). Evidence-based practice in psychology. American Psychologist, 61, 271-285.

Baron

Graham

(2007). Identifying key factors in the adoption of innovative practices. In Carlsen

McFerrin

Price

Weber

Willis

(Eds.), Proceedings of SITE 2007—Society for Information Technology & Teacher Education international conference (pp. 772-776). San Antonio, TX: Association for the Advancement of Computing in Education.

Bickman

Kelley

Breda

De Andrade

Riemer

(2011). Effects of routine feedback to clinicians on youth mental health outcomes: A randomized trial. Psychiatric Services, 62, 1423-1429.

Bowen

S. A. K.

Saunders

R. P.

Richter

D. L.

Hussey

Elder

Lindley

(2010). Assessing levels of adaptation during implementation of evidence-based interventions (EBIs): Introducing the Rogers–Rütten Framework. Health Education & Behavior, 37, 815-830. doi: 10.1177/1090198110366002

Cabassa

L. J.

Baumann

A. A.

(2013). A two-way street: Bridging implementation science and cultural adaptations of mental health treatments. Implementation Science, 8(1), 90.

Carvalho

M. L.

Honeycutt

Escoffery

Glanz

Sabbs

Kegler

M. C.

(2013). Balancing fidelity and adaptation: Implementing evidence-based chronic disease prevention programs. Journal of Public Health Management & Practice, 19, 348-356.

Centers for Disease Control and Prevention. (2010). Promoting science-based approaches: Adaptation guidelines. Retrieved from https://www.acf.hhs.gov/sites/default/files/fysb/prep-making-adaptations-ts.pdf

10.

Chor

K. H. B.

Wisdom

J. P.

Olin

S. C. S.

Hoagwood

K. E.

Horwitz

S. M.

(2015). Measures for predictors of innovation adoption. Administration and Policy in Mental Health and Mental Health Services Research, 42, 545-573.

11.

Dolcini

M. M.

Gandelman

A. A.

Vogan

S. A.

Kong

Leak

T. N.

King

A. J.

. . . O’Leary

(2010). Translating HIV interventions into practice: Community-based organizations’ experiences with the diffusion of effective behavioral interventions (DEBIs). Social Science & Medicine, 71, 1839-1846.

12.

Druss

B. G.

Miller

C. L.

Rosenheck

R. A.

Shih

S. C.

Bost

J. E.

(2002). Mental health care quality under managed care in the United States: A view from the Health Employer Data and Information Set (HEDIS). American Journal of Psychiatry, 159, 860-862.

13.

Durlak

J. A.

DuPre

E. P.

(2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327-350.

14.

Frambach

R. T.

Schillewaert

(2002). Organizational innovation adoption: A multi-level framework of determinants and opportunities for future research. Journal of Business Research, 55, 163-176.

15.

Galbraith

J. S.

Herbst

J. H.

Whittier

D. K.

Jones

P. L.

Smith

B. D.

Uhl

Fisher

H. H.

(2011). Taxonomy for strengthening the identification of core elements for evidence-based behavioral interventions for HIV/AIDS prevention. Health Education Research, 26, 872-885.

16.

Garrett

Brown

C. A.

Hart-Hester

Hamadain

Dixon

Pierce

Rudman

W. J.

(2006). Identifying barriers to the adoption of new technology in rural hospitals: A case report. Perspectives in Health Information Management, 3(9).

17.

Greenhalgh

Robert

Macfarlane

Bate

Kyriakidou

(2004). Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly, 82, 581-629.

18.

Hsieh

H.-F.

Shannon

S. E.

(2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15, 1277-1288.

19.

Jensen-Doss

Hawley

K. M.

(2010). Understanding barriers to evidence-based assessment: Clinician attitudes toward standardized assessment tools. Journal of Clinical Child & Adolescent Psychology, 39, 885-896.

20.

Lee

S. J.

Altschul

Mowbray

C. T.

(2008). Using planned adaptation to implement evidence-based programs with new populations. American Journal of Community Psychology, 41, 290-303.

21.

Leeman

Calancie

Hartman

M. A.

Escoffery

C. T.

Herrmann

A. K.

Tague

L. E.

. . . Samuel-Hodge

(2015). What strategies are used to build practitioners’ capacity to implement community-based interventions and are they effective? A systematic review. Implementation Science, 10, 80.

22.

Levin

B. L.

Hennessy

K. D.

Petrila

(2010). Mental health services: A public health perspective (3rd ed.). New York, NY: Oxford University Press.

23.

Livet

Wandersman

(2005). Organizational functioning: Facilitating effective interventions and increasing the odds of programming success. In Fetterman

D. M.

Wandersman

(Eds.), Empowerment evaluation principles in practice (pp. 123-154). New York, NY: Guilford Press.

24.

Livet

Yannayon

Kocher

McMillen

(2017). Intent to sustain use of a mental health innovation by school providers: What matters most? Advances in School Mental Health Promotion, 10, 127-146.

25.

Livet

Yannayon

M. E.

Sheppard

Kocher

Upright

McMillen

J. S.

(2017). Exploring provider use of a digital implementation support system for school mental health: A pilot study. Administration and Policy in Mental Health and Mental Health Services Research, 45, 362-380. doi:10.1007/s10488-017-0829-7

26.

Lyon

A. R.

Lewis

C. C.

(2016). Designing health information technologies for uptake: Development and implementation of measurement feedback systems in mental health service delivery. Administration and Policy in Mental Health and Mental Health Services Research, 43, 344-349.

27.

Lyon

A. R.

Wasse

J. K.

Ludwig

Zachry

Bruns

E. J.

Unützer

McCauley

(2016). The Contextualized Technology Adaptation Process (CTAP): Optimizing health information technology to improve mental health systems. Administration and Policy in Mental Health and Mental Health Services Research, 43, 394-409.

28.

Marangunić

Granić

(2015). Technology acceptance model: A literature review from 1986 to 2013. Universal Access in the Information Society, 14, 81-95.

29.

Meyers

D. C.

Durlak

J. A.

Wandersman

(2012). The quality implementation framework: A synthesis of critical steps in the implementation process. American Journal of Community Psychology, 50, 462-480.

30.

Mitchell

R. E.

Florin

Stevenson

J. F.

(2002). Supporting community-based prevention and health promotion initiatives: Developing effective technical assistance systems. Health Education & Behavior, 29, 620-639.

31.

Onwuegbuzie

A. J.

Leech

N. L.

(2007). Sampling designs in qualitative research: Making the sampling process more public. The Qualitative Report, 12, 238-254.

32.

Palinkas

L. A.

Aarons

G. A.

Horwitz

Chamberlain

Hurlburt

Landsverk

(2011). Mixed method designs in implementation research. Administration and Policy in Mental Health and Mental Health Services Research, 38, 44-53.

33.

Powell

B. J.

Proctor

E. K.

Glass

J. E.

(2014). A systematic review of strategies for implementing empirically supported mental health interventions. Research on Social Work Practice, 24, 192-212.

34.

Proctor

E. K.

Landsverk

Aarons

Chambers

Glisson

Mittman

(2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 36, 24-34.

35.

Rogers

E. M.

(1995). Diffusion of innovations. New York, NY: Free Press.

36.

Schoenwald

S. K.

Hoagwood

(2001). Effectiveness, transportability, and dissemination of interventions: What matters when? Psychiatric Services, 52, 1190-1197.

37.

S. P.

Chun

Yasui

Kuniyuki

Yip

M. P.

Taylor

Bastani

(2014). Adaptation of an evidence-based intervention to promote colorectal cancer screening: A quasi-experimental study. Implementation Science, 9(1), 85.

38.

Venkatesh

Bala

(2008). Technology acceptance model 3 and a research agenda on interventions. Decision Sciences, 39, 273-315.

39.

Wandersman

Chien

V. H.

Katz

(2012). Toward an evidence-based system for innovation support for implementing innovations with quality: Tools, training, technical assistance, and quality assurance/quality improvement. American Journal of Community Psychology, 50, 445-459.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.28 MB

0.10 MB

0.06 MB