Multifaceted interventions for supporting community participation among adults with disabilities: A systematic review

History of legislation

Numerous U.S. laws have been passed to increase protections and opportunities for people with disabilities to live more independently in the community. Section 504 of the Rehabilitation Act of 1973 was created to increase opportunity for physical and programmatic access in institutions (e.g., universities) that received federal monies. Title VII of the Rehabilitation Act established Centers for Independent Living, and mandated four core services: information and referral, advocacy, peer counseling, and independent living skills training; a fifth core service, transition, was added in the latest version of the Workforce Innovation and Opportunity Act of 2014 (Administration for Community Living [ACL], 2016). The Fair Housing Amendments Act of 1988 allowed tenants to make reasonable modifications to their apartments at their own expense; the Americans with Disabilities Act of 1990 was designed to protect the civil rights of citizens with disabilities; the Olmstead v. L.C. Supreme Court decision (1999) asserted people's right to live in the least restrictive environments.

Despite these enacted laws, many people with disabilities lack personal knowledge, skills, resources, and/or enabling environments that would allow them to take advantage of these rights and resources and to live more independently in the community. Thus, multifaceted interventions for people with disabilities that simultaneously address personal growth, such as job skills, while also building knowledge of one's rights, such as how to request reasonable accommodations, have a unique potential to truly result in increased participation for people with disabilities. Moreover, resulting increased participation in the community by people with disabilities also increases awareness of disability issues among the nondisabled population and possible support for further improvements in policies to facilitate even greater participation.

Rehabilitation research and training center (RRTC)

The University of Kansas, in partnership with the University of Montana, operates a RRTC on Promoting Interventions for Community Living (RRTC/PICL). The purpose of the Center is to examine the effectiveness of interventions to support greater community participation for individuals with physical and co-occurring multiple disabilities and to promote the dissemination and utilization of effective interventions. The RRTC/PICL is charged with developing and promoting interventions that support individuals with disabilities and positively impact community participation outcomes for such individuals. One of the goals of RRTC/PICL is to identify key components of effective evidence-based, multifaceted interventions that support individuals with disabilities in the community through a systematic review of the literature.

Definitions

Definitions related to the intervention(s) that were part of the systematic review of literature are as follows.

Multifaceted interventions—interventions that address two or more individual (changing something about the person—enhancing skills/knowledge, changing behavior, changing perceptions/attitudes) or environmental characteristics (changing something about the people, places, or things in the environments in which the person interacts) in different domains (e.g., social skills, financial management, physical health, mental health, employment, adult learning, health care).

Existing research on disabilities and community participation

Studies of the impact of single-faceted interventions on community participation of adults with disabilities are numerous. An example of a systematic review of a single intervention is a study of the efficacy of using service dogs for people with physical disabilities (Winkle, Crowe, & Hendrix, 2012). Other reports examining the impact of a single intervention on community participation for adults with disabilities include a study of transportation vouchers (Samuel, Lacey, Giertz, Hobden, & LeRoy, 2013), an exercise intervention (Dean et al., 2012), and a cognitive intervention to reduce fear of falling and associated avoidance of activity among older adults with disabilities (Haastregt et al., 2007). However, our theoretical orientation is on reviewing the effectiveness of multifaceted interventions, which are defined as interventions that address two or more aspects for change (Eldh & Wallin, 2015). People with complex and multiple support needs are likely to benefit from more comprehensive, multifaceted interventions that target multiple aspects for change. Our hypothesis is that multifaceted interventions will be more effective, especially when confronted with multiple, complex, and interacting factors affecting outcomes of a complex phenomenon such as community participation by people with a wide range of disabilities.

Very few systematic reviews of multifaceted interventions related to disability issues are in the literature. Most reviews on the topic of disability and community relate to descriptive factors that serve as barriers or facilitators to participation (e.g., White, Gonda, Peterson, & Drum, 2011). In searching The Cochrane Collaborative, The Campbell Collaboration, and NARIC databases, we found no systematic reviews of multifaceted interventions intended to enhance community participation by adults with disabilities of any type. Williams et al. (2007) conducted a systematic review of multifaceted interventions to improve depression care and found strong evidence for the effectiveness of care management for depression; however, we did not use the same definition of “multifaceted intervention” as they did. Williams et al. described “multifaceted interventions” through their inclusionary criteria, indicating that an intervention targeting depression would be included in the review if it included usual care and incorporated “at least one patient-directed element” of the Chronic Care Model, which is a healthcare specific model. Therefore, this definition of a “multifaceted intervention” would not have been applicable to our search for multifaceted interventions to promote the community living and participation of adults with disabilities.

Research questions and logic model

Reviews of multifaceted interventions on disability or other topics are rarely conducted. However, a systematic review was appropriate in this case because we have clear questions and definitions to guide the search for relevant studies. Specifically, the research questions for this systematic review are: (1)

What are the reported community participation outcomes of multifaceted interventions targeted on adults with disabilities?

Review findings will be used to enhance the intervention research of the previously mentioned RRTC/PICL. The figure below demonstrates the role the systematic review will play in future Center activities (Figure 2).

OPEN IN VIEWER

Figure 2

RRTC/PICL logic model. PICL, Promoting Interventions for Community Living; RRTC, Rehabilitation research and training center

We seek to identify components of multifaceted interventions that are effective in facilitating community participation for adults with disabilities, particularly those with severe limitations (e.g., cognition, mobility) that impede their participation in the community. The results of this systematic literature review are intended to inform policy makers in their practical decisions about social and behavioral interventions and public policy regarding funding and services to provide such interventions. In addition, the results have the potential to help health and social work practitioners in the field who work directly with people with disabilities to understand and apply the information regarding multifaceted interventions in their daily work.

Types of studies

The types of intervention studies included are:

RCTs

The team ensured that the research of selected studies is of high quality by using an adapted version of the NTACT Quality Indicator Checklists for Group Experimental studies (see, http://www.transitionta.org/effectivepractices) to assess the methodological quality of the studies to be included in the meta-analysis. Studies that fell in the category of “weak” designs as described on the checklist were not reviewed.

Types of intervention studies that were not included in the review:

Case studies

Because case studies, and single subject designs examine instances of low frequency, such study designs have been excluded. When appropriately designed and executed, case studies and single subject designs can demonstrate strong internal validity. External validity is a more complicated issue for case studies and single subject designs because they require repeated measurements and replication across participants, settings, and researchers to establish an adequate level of external validity (Horner et al., 2005). The purpose of this exclusion is to improve external validity and more closely align with the population our research questions seek explore.

Inclusionary and exclusionary criteria

A priori criteria were developed for the inclusion of studies in this review. Inclusionary criteria were as follows:

Published 2000–2016

The present review seeks to explore populations and interventions that most closely reflect the current literature. For this reason, we only included studies between the years 2000 and 2016. This is to ensure that studies with outdated interventions or dissimilar populations are not included in the review.

We specifically included English-only studies because the concepts and terminology we are using vary from culture to culture. We struggled to clearly define terms in English for our work because concepts like “community-based” and “community participation” differ from culture to culture. Although our research group possesses broad and varied skillsets, often it was difficult to come to agreement among the research team as not all members had the same background and historical knowledge of disability culture in the United States. Considering these challenges, we limited the review to English language studies, which helped to ensure reliability in the application of the inclusionary criteria.

Articles were excluded based on the following criteria:

Articles that were a literature review or compilation of studies

We further applied exclusionary criteria to the inclusionary requirement of a focus on people with disabilities in order to narrow the scope of articles and ensure that the review addressed the populations most likely to be served by our community living and participation research center. Thus, we excluded studies with interventions that were primarily medical in nature for target populations categorized as having the following types of disabilities: obesity, asthma, eating disorders, hoarding, type 2 diabetes, substance abuse/addiction, stroke, cancer, human immunodeficiency virus infection/acquired immune deficiency syndrome (HIV/AIDS), phobias, and other medical issues. Articles featuring these populations often were excluded due to the focus of the intervention (outcome was not related to community participation).

Types of participants

The intervention studies included in the review were required to target persons with a disability/ies for at least a portion of the target sample and report outcomes for the disability sample. Samples in selected articles included adults:

18 years of age or older,

Study samples did not include individuals under 18 years old or who were still participating in a transition program (typically 18–22 years old) in secondary special education.

Types of interventions

The intervention in each study met the following criteria:

Intervention measures one or more community participation outcomes targeted for change

Types of outcome measures

Duration of follow-up

Studies were considered for inclusion in the review if they measured outcomes beyond an immediate posttest. All studies meeting the inclusionary criteria and reporting follow-up data were analyzed for treatment effects for the variables reporting appropriate data for analysis. The same criteria and statistical procedures were used for follow-up conditions as for the primary intervention data. In this manuscript, follow-up data are reported separately from the primary intervention data.

Types of settings

The criteria for types of settings for the interventions were as follows.

Interventions that were applied in a community-based setting were included.

Electronic searches

Publication bias

Publication bias occurs in systematic reviews when the included studies do not properly represent the population of studies in a given field. This can happen when studies are not published, thus not being eligible for the population of studies, due to small or null effects (Rothstein, Sutton, & Borenstein, 2005). Because published studies may be more likely to have large effects than unpublished studies, only including published studies may lead to an overestimation of the treatment effect when synthesizing effect sizes in a meta-analysis.

To investigate the potential impact of publication bias in the present review, we used appropriate measures such as the funnel plot in combination with Egger's regression test for funnel plot asymmetry (Egger, Smith, Schneider, & Minder, 1997; Light & Pillemer, 2004).

Selection and screening of studies

After the results from all three searches were combined, the abstracts of the articles were exported to EndNote. A team of three coders screened studies for inclusion/exclusion at three stages, Stage 1: abstract, Stage 2: full-text for initial criteria, and Stage 3: full-text for methodology. We selected 15 studies that met criteria for inclusion in the study and were of “acceptable” (13) or “high” (2) methodological quality for inclusion in the meta-analysis.

Data extraction and management

At least two reviewers independently extracted data for all eligible studies and then reviewed and compared the data for accurate entry. They used a coding sheet to gather data from each article. When two or more study samples provided sufficient information to permit effect size calculations, we conducted random-effects meta-analyses to synthesize effects across studies.

Assessment of risk of bias in included studies

In the final full-text review stage, we assessed each included study for methodological quality using as the risk of bias for such characteristics as reporting, internal validity bias, selection bias, external validity bias, and attrition bias using an adapted version of the NTACT Quality Indicator Checklists for Group Experimental studies.

Measures of treatment effect

We made all calculations using the metafor and robumeta packages in R to synthesize, compute, compare, and determine variation in effect sizes and treatment effects across the reviewed studies (Fisher, Tipton, & Zhipeng, 2017; Viechtbauer, 2010). The calculation of effect sizes account for the variety of measures used to assess intervention effects by calculating and analyzing all data using the standardized mean difference converted to Hedges' g. For those studies reporting F test, t test, or p values rather than means and standard deviations, effect sizes are calculated using appropriate conversion conventions in R.

In order to provide a substantive interpretation of the intervention effects and retain effect size independence, the effect sizes from the individual studies are aggregated within and across studies using only same or conceptually similar outcomes. For example, for studies reporting hours worked, salary, job tenure, these data are classified as employment outcomes with the combined treatment effect on employment aggregated across other studies reporting employment outcomes. Categories of measured community participation outcomes include employment, quality of life, mental health, adult learning, activities of daily living, autonomy, independent living, social skills, community activities, and housing. Different outcomes categories unrelated to the community participation outcomes of interest were not analyzed.

Unit of analysis issues

The unit of analysis was determined based on the presentation of outcome data. For those studies presenting participant (e.g,. student, patient, client, consumer) data the unit of analysis was at the individual level. For those studies presenting at a group level (e.g., center, state) the level of analysis was conducted at the group level.

Dealing with missing data

Attrition was calculated for each study and an evaluation was conducted to assess overall quality of the study. No included study was eliminated from the analysis due to missing data.

Assessment of heterogeneity

Heterogeneity analysis was conducted for participant, intervention, and outcome characteristics. In light of the fact that multiple effect sizes may be attributable to sampling error, a random effects model and the associated inverse variance weight at the 95% confidence level was used for all analysis. The random effects model provides for an assumption of population variation from which the sample is drawn and calculates the impact of the effect size by estimating the parameters of that population.

Assessment of reporting bias

The assessment of reporting bias was accomplished via an assigned NTACT rating as high quality, acceptable quality, or did not meet quality. The rating of high quality required that all items be positively accounted for in the text, while acceptable quality required that items 1–6, 9, 10, 12, 16, and 17 be present. A rating of did not meet quality required that one or more of the acceptable quality items be absent.

Data synthesis

Data extracted from all included studies reported continuous outcomes, the effect size data was synthesized using outcome means and standard deviations when available. For those studies reporting F test, t test, or p values rather than means and standard deviations, effect sizes were calculated using appropriate conversion conventions provided by the metafor package in R (Viechtbauer, 2010). No studies reported dichotomous data. When two or more studies using a common group comparison (e.g., T vs. C or T₁ vs. T₂) were reported a meta-analysis was conducted as provided by the metafor package in R (Viechtbauer, 2010). The meta-analysis used a correlated effects model with small-sample size correction for analysis of effect sizes across studies aggregated on the basis of a common outcome. To account for potential issues of dependency within studies, Robust Variance Estimation was used to estimate the standard errors.

Subgroup analysis and investigation of heterogeneity

Subgroup analysis was planned for (a) types of direct access to community participation, (b) dimension of community participation, (c) length of intervention, (d) place of intervention, (e) type of outcome measure, and (f) disability type. However, because of an insufficient number of studies, analysis was only conducted for employment and length of intervention.

Sensitivity analysis

A one-removed procedure was conducted as a sensitivity analysis to assess the strength of a single study effect size on the overall effect size for those outcomes in which multiple studies contributed to a common outcome. No other sensitivity analysis was appropriate due to the small number of studies or represented in the measured outcomes.

Results of the search

The electronic databases searches yielded 4,738 potentially relevant documents for review. During the review, we collected an additional five articles identified through ancestral searching/specific references to relevant articles during the review of full texts for a total of 4,743.

These documents were filtered for duplicates while conducting the searches, and we used Endnote (a bibliographic management software) to sort and identify further duplicates prior to beginning the review of the search results. Of these 4,743 documents we reviewed, an additional 18 were duplicate hits, which we eliminated from further consideration. We reviewed the titles and abstracts of the 4,743 documents to determine potential relevance, excluding 4,556 due to irrelevance to the review, leaving 186 articles on which to examine the body of the text to determine inclusion in the review. At the initial review stage, two researchers reviewed 12% of the manuscripts by title and abstract to check for alignment among researchers in application of the inclusionary criteria and to resolve any uncertain determinations.

Next, we obtained and reviewed 186 full-text documents, meeting weekly to discuss the articles and come to consensus on the determination of inclusion in the meta-analysis. Of the 186, 37 full-text documents were determined to have content appropriate for the review. We then reviewed the remaining 37 manuscripts for methodological quality. We excluded 22 studies due to ineligible or poor-quality study design. Fifteen studies (peer-reviewed manuscripts) met all eligibility criteria and were included in the meta-analysis. Figure 3 illustrates the flow of studies through the systematic review process.

OPEN IN VIEWER

Figure 3

Prisma flow-chart

Included studies

An examination of the full texts of 186 potentially relevant studies resulted in 15 studies being included in the meta-analysis (see Table 1), representing 3,296 treatment participants and 3,782 control or comparison participants. All included studies employed a multifaceted intervention with participants with disabilities. We conducted a meta-analysis of the impact of multifaceted interventions on the following community participation outcomes: employment, quality of life, mental health, adult learning, activities of daily living, autonomy, independent living, social skills, community activities, and housing.

Excluded studies

Studies that were excluded from the analysis may have employed interventions that had multiple components (i.e., multifaceted), but the interventions did not focus on two different domains. There were 149 excluded studies due to ineligible interventions or ineligible study design. Among studies with ineligible designs, single population studies without comparison groups were common. We also excluded studies because the participants did not meet criteria either due to age (e.g., younger than 18) or the described disability was excluded from our defined population (e.g., obesity, diabetes, hoarding, phobias, substance abuse, suicidal tendencies, HIV/AIDS, arthritis, stroke with no other reported disability).

Publication bias

Examining the funnel plot in Figure 4 we see that studies, identified by a single point in the plot, are generally evenly distributed while roughly forming the funnel shape. This is an initial indicator that it is likely publication bias is not significantly impacting the overall results of this systematic review.

OPEN IN VIEWER

Figure 4

Funnel plot

Results from the Egger's regression test further confirm this notion (t = 0.724, df = 13, p = .482). With a nonsignificant result from the Egger's regression test aligning with the visualization of the funnel plot, we are confident that the results of this systematic review are not significantly impacted by publication bias.

Assessment of heterogeneity

In addition to the overall effect size, we report two measures of heterogeneity to assess the level of consistency across studies included in the meta-analysis. First, we report I ², which measures the percentage of between-study variance attributable to true heterogeneity and not sampling error (Borenstein, 2009; Higgins & Thompson, 2002). Next, we report τ ², which is a measure of the dispersion of true effect sizes between studies (Borenstein, 2009).

The I ² (I ² = 92.01%) indicates that 92.01% of the between-study variance is due to true heterogeneity. According to Higgins et al. an I ² of 25.00 is considered low, 50.00 moderate, and 75.00 high (Higgins, Thompson, Deeks, & Altman, 2003). Because of the high degree of heterogeneity in the present meta-analysis, it follows that moderator variables impact the result. Similar to what was drawn from the I ² result, the τ ² (τ ² = 0.182) indicates a high degree of heterogeneity.

Thus, the following analyses and results are presented as a refined assessment of treatment effects as independent measures for design, participant, treatment, and outcome characteristics for each of the observed outcomes. These analyses are organized by outcome according to the design, and treatment classifications described earlier in this paper.

As we discuss the studies analysed below, we use “Experimental” to refer to the populations that are receiving the multifaceted intervention. We use “Control” or “Comparison,” as appropriate, to refer to the populations to which the outcomes of the Experimental group receiving the multifaceted intervention is compared.

Employment outcome

Activities of daily living (ADL) outcome

A total of three studies used an RCT Exp versus Ctl design (Birk et al., 2004; Onor et al., 2007; Szanton et al., 2011) to assess the treatment effects for ADL outcomes. Results of this meta-analysis revealed an overall nonsignificant treatment effect (g = 0.422, 95% CI, [−1.201, 0.358], p = .289) (Table 6).

Similarly, a single study (Kurz et al., 2009) used a QED Exp versus Comp design to assess treatment effects associated with the ADL outcomes for two groups (mild cognitive impairment and dementia). The resulting treatment effect failed to reach statistical significance (g = −0.184, 95% CI [−0.948, 0.579], p = .636) for either of the two disability groups separately or combined.

Mental health

Mental health outcomes were assessed by three studies (Birk et al., 2004; Kurz et al., 2009; Onor et al., 2007) evaluating the treatment effects for mental health outcomes with an aging population of adults. Birk and Onor compared a multifaceted approach with a no-treatment control group, while Kurz utilized a multifaceted approach compared to a single-faceted program. Results of this analysis yielded a combined g value of −0.181, (95% CI [1.325,−0.964], p = .757. To assess the potential impact of the design, a one-study analysis indicated that when Kurz was removed from the aggregation, the resulting effects still did not reach significance for the multifaceted group (g = −0.738, 95% CI [−1.389, −0.087], p = .026) and indicates that the comparison group performed better than the multifaceted group on the measures of mental health.

Adult learning

A single study (Gutman et al., 2009) assessed the treatment effects on outcomes of adult learning for social, task, and interpersonal skill development for participants with psychiatric disabilities using an RCT Exp versus Comp design. The analysis of the treatment effects for this group yielded a positive statistically significant effect (g = 1.144, 95% CI [0.806, 1.483], p = .001) in favor of the multifaceted intervention over an alternative single-faceted approach.

Quality of life

Two studies evaluated quality of life for people who have aging-related disabilities (Szanton et al., 2011) and who acquired a TBI (Twamley et al., 2014) to assess treatment effects associated with participant assessment of quality of life outcomes. Szanton's data were based on an Exp versus Ctl design and yielded a significant treatment effect for QOL measures (g = 0.693, 95% CI [0.240, 0.854], p = .003) in favor of the multifaceted treatment. Twamley's data were based on an Exp versus Comp design and yielded a nonsignificant treatment effect (g = 0.195, 95% CI [−0.464, 0.854], p = .561) for the QOL outcomes. Due to the qualitative difference in design, no aggregation of effect sizes was appropriate.

Autonomy

Birk et al. (2004) assessed treatment effects on an autonomy outcome measure using an RCT Exp versus Ctl design. Results of this analysis yielded a g value of 0.850, 95% CI [0.023, 1.723;], p = .056) indicating a nonsignificant treatment effect.

Independent living, social skills, community activities, and housing

Fleming et al. (2009) was the sole study to assess independent living, social skills and community activities for a group of participants with TBI. Data were drawn from a QED Exp versus Comp design with a self-report questionnaire used to assess each of the three outcomes. Results of this analysis revealed a significant treatment effect for the comparison groups in the independent living (p = .017) and community activities (p = .022) outcomes and a nonsignificant treatment effect (p = .522) in favor of the comparison group in the social skills outcome as shown in Table 7.

Tsemberis and Eisenberg (2000) analysed data collected from two different programs serving a homeless mentally ill population in New York City. Tsemberis used an QED Exp versus Comp design in which one program, Linear Residential Treatment, was a public-funded program designed to provide permanent housing for clients. Pathways Supported Housing was a nonprofit agency that provided services to individuals who were unable to obtain housing through the Linear Residential program. Using a survival analysis, Tsemberis determined that five years later 88% of the Pathways participants and 47% of the Linear participants retained permanent housing. Results of the comparison between the two groups for retention of permanent housing was found to be statistically significant in favor of the Pathways Supported Housing group (g = 0.228, 95% CI [0.092, 0.363], p = .001).

Employment

Results from those studies reporting on intervention programs to improve employment outcomes, five studies (Bell et al., 2008; Gimm et al., 2011; McGurk et al., 2007; Mirza & Hammel, 2009; Tsang et al., 2009) reported results based on a true experimental design. The meta-analysis of these studies yielded a nonsignificant intervention impact (p = .085). A single study (Cook et al., 2005) also reported results of an intervention impact comparing a multifaceted intervention with a single-faceted intervention and reported a significantly positive effect for the experimental group (p = .004). No overall conclusions based on these data can be offered due to the small number of studies representing the outcome of employment.

One of the design issues that emerged was whether the absence of an overall treatment effect could be due to the method of analysis. The issue of concern was that the more complete analytical method, ITT, has been described as a more accurate representation of the “real world” of intervention in that participants in many settings dropout, do not complete the post assessment, or are otherwise unaccountable for a complete measure of the intervention effect. Alternatively, the TOT approach identifies the treatment effect for participants who complete the entire intervention program including all assessment measures. This distinction can be viewed partly as statement of practicality verses reality, and as such becomes a matter of the nature of the application of the findings.

Those studies using an ITT approach to the data analysis yielded a nonsignificant effect while the studies using a TOT revealed a statistically significant intervention effect for the multifaceted intervention approach (p = .001) and a significant difference between the two methods of analysis (p = .001). These data suggest the possibility that the less restrictive TOT (n = 3) method of analysis may introduce an inherent positive bias resulting in an inflated true intervention effect while the ITT (n = 2) approach may more accurately reflect the overall effectiveness of an intervention. In either case, as applied to these data, no definitive interpretation can be drawn due to the small number of studies represented in each group.

A second variable of interest was the length of treatment as a measure of intervention effectiveness. Across the 15 studies the range of the weeks of intervention was 4–105 weeks and were categorized into three groupings: 1–10, 11–20, and 21+ weeks. When this range was applied to the employment related data presented in Table 2, the only time frames of 1–10 and 21+ weeks were represented. Results of the aggregation of studies representing each of these time frames indicates that the shorter intervention time period results in significantly larger program effect in favor of the multifaceted group while the longer intervention produced a small statistically nonsignificant effect (0.390). These data should be cautiously interpreted due to the small number of studies in each time frame as well as the wide variation in the number of weeks represented in the longer time frame. That is, while the 1–10 week studies showed a significantly positive effect, there are no intervening time frames to compare as a measure of intervention effect deterioration between the two time frames. That is, without an intervening measure the trend of the treatment effect is unknown. It might be that the degradation of the intervention effect begins at 21+ weeks.

An additional three studies providing data regarding employment outcomes were included in the review but analyzed separately. These studies used comparison of a multifaceted intervention with a single-faceted intervention rather than an untreated control group comparison. Two studies used a RCT design (Ferguson et al., 2012; Fleming et al., 2009) for the comparison while Cook et al. (2005) used a QED design. Results of the analysis of these studies suggested that the multifaceted intervention did not result in a significantly positive advantage when compared to the single-faceted interventions.

Adult learning

Adult learning (Gutman et al., 2009) outcomes were found to demonstrate a positive effect when using a QED design comparing a multifaceted with a single-faceted intervention. The findings supported the multifaceted group over the comparison group (g = 1.144); however, without a no-treatment control group, no conclusion can be drawn as to the independent effectiveness of the multifaceted intervention. The further potentially confounding factor of different designs makes any confirming statement of treatment effect inappropriate.

Activities of daily living

Three studies (Birk et al., 2004; Onor et al., 2007; Szanton et al., 2011) evaluated multifaceted interventions’ impact on ADL compared to a nontreated control group. Additionally, Kurz et al. (2009) reported a QED designed study reporting a comparison between multifaceted and single-faceted programs. With the results of all four studies yielding statistically nonsignificant findings in favor of the multifaceted participants, either the usefulness of the outcomes measures or the multifaceted intervention itself are potentially suspect. As stated earlier, these data too must be interpreted with caution due to the small number of studies represented in the analysis.

Mental health

The analysis of the effect of a multifaceted approach for mental health outcomes was represented by three studies, (Birk et al., 2004; Kurz et al., 2009; Onor et al., 2007). The results also yielded a lack of support for the multifaceted approach. Due to the small number of studies available, no conclusion or explanation can be offered for the treatment effects related to mental health.

Quality of life

Quality of Life outcomes were assessed by two studies using different designs for an aging disabled population (Szanton et al., 2011) and a population with TBI (Twamley et al., 2014). Szanton et al. (2011) found a statistically significant impact for quality of life measures (p = .003), while the Twamley et al. (2014) study yielded a nonsignificant outcome (p = .561). Due to design differences, the aggregation of these data was not conducted. Due to the small number of studies and the design differences, no substantive conclusions can be drawn.

Autonomy

Outcomes associated with measures of autonomy were reported by Birk et al. (2004). Results of this RTC study yielded a large but nonsignificant effect (g = 0.850, p = .056). While these results approach statistical significance, with data from only a single study, no conclusion or explanation can be drawn.

Independent living, social skills, community activities, and housing

A single study assessed the effects of independent living, social skills, and community activities (Fleming et al., 2009). None of these outcomes provided support for the multifaceted intervention.

Tsemberis and Eisenberg (2000) reported on an assessment of the effects of a multifaceted intervention compared to single-faceted intervention on housing retention. While a significant effect was found, no conclusions can be drawn from a single study.