Abstract
Occupational therapy was founded on the principles of humanistic philosophy that emphasize reintegration of people with mental and physical health challenges into the mainstream of life through participation in daily occupations (Cole & Tufano, 2008). Over time, as the medical model became prominent in health care, practitioners focused more on physical disabilities, eventually leading to a decline in the presence of occupational therapy in mental health (Creek, 2014; D’Amico, Jaffe, & Gibson, 2010). In recent years, the American Occupational Therapy Association (AOTA) has been pushing for the profession to reestablish its presence in mental health (AOTA, 2016; Gutman, 2011).
To achieve this objective, occupational therapists need to demonstrate that their interventions require specialized skills that are based on research-supported theoretical principles. Such skills can be demonstrated through competent application of the theoretical principles undergirding interventions, as articulated in the profession’s theoretical conceptual practice models. Therefore, occupational therapy practitioners in mental health need to demonstrate that their interventions are clearly based on (1) the principles of the profession’s theoretical conceptual practice models (Hinojosa, 2013; Ikiugu & Smallfield, 2015), (2) sound research evidence (Gutman, 2011), and (3) occupations that are meaningful to the client (Schell & Gillen, 2014).
Although the profession’s theoretical principles guide clinical decision making, no consensus exists regarding the definition of a theoretical conceptual practice model. In this study, we adopted Kielhofner’s (2009) definition of a theoretical conceptual practice model as consisting of a “solid grounding in practice, theory that clearly addresses a unique practice circumstance or challenge and development of specific practice resources” (p. 62). Some scholars in the profession (see, e.g., Cole & Tufano, 2008) have distinguished between the terms theoretical conceptual practice models and frames of reference. However, Kielhofner (2009) made no such distinction. Others have relied on the Occupational Therapy Practice Framework: Domain and Process (3rd ed.; AOTA, 2014) to guide eclectic theoretical considerations during evaluations and interventions aimed at improving occupational performance through occupation-based practices. The purpose of our meta-analysis was to answer the following question: What is the overall effect of occupational therapy interventions that are based on the profession’s theoretical conceptual practice models in improving occupational performance and well-being among adults with a mental health diagnosis?
Method
Research Design
This study was conducted using a meta-analytic design (Borenstein, Hedges, Higgins, & Rothstein, 2009; Portney & Watkins, 2009). This design was best suited for applying rigorous statistical procedures to estimate the effectiveness of theory-based occupational therapy interventions in improving occupational performance and well-being among clients with mental health conditions.
Definition of Variables
The dependent variables in our analysis were occupational performance and well-being. Occupational performance was defined as the outcome of a dynamic transaction among the client, environmental context, and occupation, manifesting in participation in areas of occupation (activities of daily living, instrumental activities of daily living [IADLs], rest and sleep, education, work, play, leisure, social participation) as defined in the Framework. Well-being was defined as “the total universe of human life domains, including physical, mental, and social aspects” (World Health Organization, 2006, p. 211). Mental health symptoms such as anhedonia, hallucinations, and delusions were some of the indicators of compromised well-being.
Study Sample
Studies included in this meta-analysis were those in which (1) participants were adults (age 18 yr or older); (2) participants had a mental health diagnosis; (3) a randomized controlled trial (RCT) design was used; (4) well-being, occupational performance, or both were the dependent variables; (5) parametric statistics (e.g., t tests in post hoc pairwise comparisons) were used to analyze data; (6) the language of publication was English; (7) the date of publication was between 1995 and June 2016; and (8) a simple majority of the 25 criteria in the Consolidated Standards of Reporting Trials (CONSORT; Schulz, Altman, & Moher, 2010) checklist was attained.
Instruments
Data Extraction Protocol.
We developed a data extraction protocol based on the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA; Moher, Liberati, Tetzlaff, & Altman, 2009) guidelines and the Cochrane Handbook for Systematic Reviews of Interventions (Cochrane Collaboration, 2011) to ensure thoroughness and consistency of the systematic review process (Shamseer et al., 2015). The protocol consisted of the following fields: citation, study design, sample characteristics, occupational therapy intervention (independent variable), theoretical conceptual practice model that was the source of the intervention guidelines, duration of the intervention, dependent variable or variables, experimental and control group means and standard deviations on the dependent variable or variables, post hoc pairwise comparison t values, and p values (Higgins & Deeks, 2011).
Theoretical Conceptual Practice Model Identification Criteria.
Few researchers identified the theoretical conceptual practice model principles that guided the intervention strategies that they investigated. Therefore, we used a framework developed by Ikiugu and Nissen (2016) to determine the theoretical model on which the interventions were likely based, consistent with Kielhofner’s (2009) definition of a theoretical conceptual practice model, as explained earlier. The framework consisted of key constructs for each of the models that Ikiugu and Nissen found are commonly used by occupational therapy practitioners in mental health.
We used these constructs as the criteria for determining which theoretical conceptual practice model was the source of the intervention strategies in the research studies that we reviewed. For example, two key constructs in the Canadian Model of Occupational Performance and Engagement (CMOP–E) are (1) facilitation of client autonomy by encouraging free choice of occupation and (2) empathic sharing of the experienced occupational performance. Therefore, because Buchain, Vizzotto, Henna Neto, and Elkis (2003) investigated interventions that included “free choice of activities” used in therapy and developing “a common activity, as a whole” during group sessions, thus “enabling richer and more varied exchanges between” clients and “between them and the therapist” (p. 27), we concluded that these strategies were likely derived from the CMOP–E.
PRISMA Protocol.
We used the PRISMA protocol (Liberati et al., 2009) to evaluate the risk of bias for the studies included in the meta-analysis. The protocol requires assessment of each study to determine whether it meets the following criteria: concealment of randomization; no early termination of the trial; and blinding of research participants, researchers, and data collectors. The more of these criteria are met, the less bias is attributed to the trial.
CONSORT Checklist.
The CONSORT checklist (Moher et al., 2010) consists of 25 criteria that are used to appraise RCTs for external and internal validity on the basis of randomization status as stated in the study title; summary of the trial design, methods, results, and conclusions in the abstract; statement of specific objectives or hypotheses; list of study participant eligibility criteria; description of the settings and locations where data were gathered; description of how the sample size was determined; and description of the method used to generate the random allocation sequence (Moher et al., 2011). We read each study and indicated the page number on which each criterion on the CONSORT checklist was addressed. If a simple majority of the 25 criteria on the checklist were met, we considered the study to be eligible for inclusion in the meta-analysis.
Procedures
Search Strategies and Study Selection Criteria.
Three student coinvestigators completed a comprehensive search of the literature between September 2015 and January 2016. The databases searched were MEDLINE, CINAHL, PsycINFO, ClinicalKey, Social Work Abstracts, Cochrane Library, PubMed Health, and OTseeker. We used the following search terms in various combinations: occupational therapy, occupational therapy interventions, mental health, theoretical conceptual practice models, occupational performance, and well-being. The three coinvestigators read the titles and abstracts of studies that they considered potentially relevant for inclusion in the sample. References of retrieved articles were hand searched to identify other studies that potentially met the inclusion criteria. The investigators selected the final sample by consensus. The two principal investigators completed an independent search in June 2016 and identified 3 additional studies. The final sample consisted of 11 RCTs (Figure 1 provides the details of the article search and selection). The articles constituting the sample are listed in the references and indicated by asterisks.
Illustration of the systematic search strategy.
Appraisal of Bias and Methodological Quality.
To assess the risk of bias and methodological quality, we evaluated each study using the PRISMA protocol and the CONSORT checklist. Analysis revealed that of the 11 studies, randomization was concealed in 6, researchers or participants were blinded in none, and data collectors were blinded in 4. All 11 RCTs met a majority of the CONSORT criteria.
Data Extraction.
We read each of the articles and extracted data using the data abstraction protocol. We used the theoretical conceptual practice model identification framework (Ikiugu & Nissen, 2016) to determine the theoretical basis for each study’s experimental independent variable.
Statistical Analysis.
We entered the abstracted data into a Comprehensive Meta-Analysis software database (Version 3; Biostat Inc., Englewood, NJ). Hedge’s g was computed as a measure of effect size because it is less biased than Cohen’s d (Borenstein et al., 2009). Because the reviewed studies were conducted in disparate environments and involved participants with a variety of mental health conditions, variability of effect sizes was expected. Therefore, we used a random effects model to compute the summary effect sizes (Borenstein et al., 2009). We tested the data for homogeneity of effect sizes using the Q statistic. Even when the hypothesis of heterogeneity was rejected, however (p > .10), we used the random effects model on the basis of our previously stated rationale. To assess articles for publication bias, we constructed a funnel plot, followed by Duval and Tweedie’s (2000) trim-and-fill method and Egger, Davey Smith, Schneider, and Minder’s (1997) regression intercept test (Borenstein, Hedges, Higgins, & Rothstein, 2004).
Results
In total, 520 people participated in the 11 studies in our sample. Occupational performance was measured for 323 participants, and well-being was measured for 349 participants. A total of 91 outcomes (44 occupational performance outcomes and 47 well-being outcomes) were measured. The Behavioral/Cognitive–Behavioral (B/CB) theoretical conceptual practice model was used to guide intervention in 5 studies; CMOP–E in 3 studies; and the Cognitive Disabilities (CD) model, unspecified client-centered models, and a psychodynamic model in 1 study each. The interpretation of effect sizes was as follows: d > 0.20, small; d > 0.50, medium; and d > 0.80, large (Cohen, 1988).
Efficacy of Theory-Based Interventions in Improving Occupational Performance
A test of homogeneity revealed significant heterogeneity in effect sizes in the 7 studies in which occupational performance was measured, Q(6) = 37.57, p < .001. Analysis for moderators indicated that much of the variability could be attributed to 1 outlier study (N = 42) in which intervention was based on unspecified client-centered theory. The intervention had a small effect favoring the control group (Hedge’s g = −0.06, 95% confidence interval [CI] [−0.20, −0.07], Z = −0.91, p = .36). When the study was removed from analysis, heterogeneity was reduced, Q(5) = 10.05, p = .07. A p > .10 is recommended for the Q test to reject the heterogeneity hypothesis (Kim, Yoo, Jung, Park, & Park, 2012). However, Higgins (2008) argued that “any amount of heterogeneity is acceptable, providing both that the predefined eligibility criteria for the meta-analysis are sound and that the data are correct” (p. 1158).
Because heterogeneity in this case was not substantial (p > .05), our meta-analysis inclusion criteria were sound, and great care was taken to ensure data accuracy, we judged that we could combine the remaining 6 studies in the meta-analysis. The effect of intervention on occupational performance was medium (Hedge’s g = 0.50, 95% CI [0.26, 0.74], Z = 4.05, p < .001; see Supplemental Table 1, available online with this article at https://ajot.aota.org). The funnel plot indicated no publication bias (fail-safe N = 51 studies; Egger’s regression intercept = 0.56, p = .40, one-tailed). This finding was confirmed using Duval and Tweedie’s (2000) trim-and-fill procedure (Figure 2).
Standardized mean differences plotted against standard error of the differences.
Efficacy of Theory-Based Interventions in Improving Well-Being
A test of homogeneity revealed significant heterogeneity in the effect sizes among the 7 studies in which well-being was measured, Q(6) = 537.26, p < .001. Analysis for moderators revealed that much of the variability could be attributed to 2 outlier studies (N = 102) in which unspecified client-centered theory and the psychodynamic model were used to guide intervention strategies. A small effect of intervention was found, based on unspecified client-centered principles, on well-being that favored the control group (Hedge’s g = −0.40, 95% CI [−0.56, −0.23], Z = −4.65, p < .001). The effect of intervention based on the psychodynamic model (N = 60) was very large (Hedge’s g = 3.46, 95% CI [3.18, 3.75], Z = 23.85, p < .001). Removal of the 2 studies reduced heterogeneity to acceptable levels, Q(4) = 7.73, p = .10. Subsequent analysis indicated that there was a small effect of intervention on well-being among study participants (Hedge’s g = 0.46, 95% CI [0.28, 0.65], Z = 4.96, p < .001; Table 1).
The funnel plot revealed that there was a significant publication bias (fail-safe N = 64 studies, Egger’s regression intercept = 3.35, p = .01, one-tailed). Duval and Tweedie’s (2000) procedure revealed that the bias was corrected by trimming and imputing 2 studies, leading to an adjusted Hedge’s g of 0.37 (95% CI [0.16, 0.57]; Figure 3).
Plot of standardized differences in means against standard errors of the differences.
The imputed studies seemed to correspond to the 2 that were removed to address the problem of heterogeneity. When the 2 studies were added back into the analysis, the summary effect was still small (Hedge’s g = 0.49). However, the publication bias was corrected (fail-safe N = 320 studies, Egger’s regression intercept = 12.01, p = .15).
Discussion
The purpose of this meta-analysis was to estimate the effectiveness of occupational therapy interventions based on the profession’s theoretical conceptual practice models in improving occupational performance and well-being among people with mental health challenges. There was indication of a medium effect of interventions in improving occupational performance among treated participants. This finding was consistent with previous research indicating that occupational therapy interventions were effective in improving performance of occupations such as work, parenting, home management and maintenance, and social participation among people with serious mental illness (Arbesman & Logsdon, 2011; Gutman, Kerner, Zombek, Dulek, & Ramsey, 2009; Gutman et al., 2007; Oka et al., 2004). In addition, training clients in social skills, life skills, IADLs, and neurocognitive interventions, coupled with vocational, social, and IADL training, has been found to be effective in enhancing executive functioning and healthy routines among people with mental health disorders (Anzai et al., 2002; Katz & Keren, 2011; Zielinski Grimm et al., 2009).
Our findings also indicate that occupational therapy interventions may have a small effect in improving well-being for people with a mental health diagnosis. This result was consistent with the findings of a systematic review by Gibson, D’Amico, Jaffe, and Arbesman (2011) that occupational therapy was effective in enhancing occupational performance and reducing mental health symptoms, thus improving well-being. Finally, the theoretical principles that informed intervention strategies in the studies constituting our sample were drawn from the B/CB, CMOP–E, CD, Occupational Adaptation (OA), Model of Human Occupation (MOHO), and psychodynamic theoretical conceptual practice models. This result was similar to the finding in previous studies that the B/CB model, MOHO, and CMOP–E were the most frequent sources of intervention strategies in mental health practice (Ikiugu, 2012; Ikiugu & Nissen, 2016; Lee, Taylor, & Kielhofner, 2009). However, unlike in previous studies, the OA and psychodynamic models also featured in our meta-analysis.
Limitations
In most cases, the researchers did not identify the theoretical conceptual practice models on which the interventions that they investigated were based. We often had to use previously developed criteria (Ikiugu & Nissen, 2016) to determine the theoretical basis of interventions. It is possible that we could have made errors in interpreting the interventions investigated in the studies that constituted our study sample. Moreover, the appraisal of bias and methodological quality of the studies in our sample revealed a lack of reporting of preferred criteria. Therefore, the effect sizes reported should be viewed with these limitations in mind.
Considerations for Future Research
In future research, investigators may consider completing content analysis of the studies that do not meet the inclusion criteria for meta-analysis to generate more information about the theoretical conceptual practice models that are generally used to guide interventions in mental health. Such investigations may shed more light on the theoretical reasoning used by occupational therapy clinicians and researchers when they plan therapeutic interventions. Moreover, in future meta-analyses it may be interesting to find out how diagnostic categories modulate the effects of interventions.
Implications for Occupational Therapy Practice
Our findings indicate that occupational therapy interventions based on the following theoretical frameworks may be effective in improving occupational performance and well-being among people with a mental health diagnosis: B/CB, MOHO, CMOP–E, psychodynamic, CD, and OA. However, many times the researchers did not clearly identify the occupational therapy theories that guided their intervention strategies. One interesting finding was that when the theoretical basis of intervention was extremely ambiguous (unspecified client-centeredness), the effect was very small and tended to favor the control group. This finding further underscores the importance of basing interventions on clear occupational therapy theoretical principles. Overall, the results of the study suggested that
Interventions based on theoretical conceptual practice models may lead to improvement in occupational performance and well-being among people with a mental health diagnosis.
Clear identification of specific theoretical conceptual practice models on which interventions are based is often lacking in the published mental health occupational therapy research.
A clearer description of the theoretical basis of interventions (independent variables) in research may provide clinicians with stronger rationales when claiming reimbursement for services.
Conclusion
The purpose of this meta-analysis was to estimate the effectiveness of occupational therapy interventions based on the profession’s theoretical conceptual practice models in improving occupational performance and well-being among people with a mental health diagnosis. Our findings suggest that occupational therapy interventions had a medium effect on improving occupational performance and a small effect on improving well-being among people with a mental health diagnosis. Further research should be conducted to confirm these findings. In addition, occupational therapy practitioners and researchers should clearly and consistently identify and document the theoretical basis and details of their interventions to demonstrate the profession’s unique value in the rehabilitation of people with a mental health diagnosis.
Supplemental Material
Supplementary material for Clinical Effectiveness of Occupational Therapy in Mental Health: A Meta-Analysis
Supplementary material, sj-pdf-1-aot-10.5014_ajot.2017.024588.pdf for Clinical Effectiveness of Occupational Therapy in Mental Health: A Meta-Analysis by Moses N. Ikiugu, Ranelle M. Nissen, Cali Bellar, Alexya Maassen and Katlin Van Peursem in The American Journal of Occupational Therapy
Footnotes
Acknowledgments
The findings of this study were presented at the University of South Dakota School of Health Sciences Research Conference on April 22, 2016, and the University of South Dakota Occupational Therapy Research Symposium on May 4, 2016. They were also presented at the 2017 AOTA Annual Conference & Centennial Celebration on March 30, 2017.
*
Indicates studies included in the meta-analysis.
Note. Each issue of the 2017 volume of the American Journal of Occupational Therapy features a special Centennial Topics section containing several articles related to a specific theme; for this issue, the theme is occupational therapy’s role in mental health. The goal is to help occupational therapy professionals take stock of how far the profession has come and spark interest in the many exciting paths for the future. For more information, see the editorial in the January/February issue, 
.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
