Abstract
Introduction
The purpose of this study was to assess the effectiveness of a housing transition program for homeless shelter residents with chronic mental illness and substance use.
Method
A two-group controlled study design was used to assess the program, with 10 participants in an intervention group and 10 in a control group. Goal attainment scaling and quality of life ratings were used pre- and post-intervention to determine if a statistically significant difference existed between groups at post-intervention.
Results
A statistically significant difference existed between intervention and control group goal attainment scaling scores (U = 9.50, p < .03, d = 1.34), and on quality of life scores (U = 10.50, p < .04, d = 1.30) at post-intervention. At a 6-month follow-up, 57.14% of intervention group participants had transitioned into supportive housing, while only 25% of control group participants had transitioned.
Conclusion
Findings suggest that intervention participants made greater progress toward desired housing goals and reported higher quality of life ratings than controls at post-intervention. This study provides support for the effectiveness of a housing program for homeless adults with chronic mental illness and substance use histories to achieve housing goals and higher quality of life ratings.
Introduction
Homelessness is an epidemic international problem. In January 2015, 564,708 people were homeless on any given night in the United States (US) and an estimated 1.6 million used homeless shelters that year (US Department of Housing and Urban Development, 2015). In that same year, 85,460 British adults were reported homeless and applied for government housing assistance (Department for Communities and Local Government, 2016). Similarly, it was estimated that over 235,000 Canadians experienced homelessness in 2014, with approximately 35,000 homeless on any given night (Gaetz et al., 2014).
In the US, approximately one-third of homeless adults have severe chronic mental illness (for example, schizophrenia, bipolar disorder, major depression) (Treatment Advocacy Center, 2014) and two-thirds have a substance use disorder (Office of National Drug Control Policy, n.d.). Homeless adults with mental illness and substance use disorders are more likely to remain homeless and lose housing compared to those without these diagnoses (Substance Abuse and Mental Health Services Administration, 2016). Many self-care and home management skills that are typically learned during the transition from adolescence to adulthood (for example, bathing, hygiene, clothing care, apartment cleaning and organization, meal preparation, and money management) are not acquired by this homeless sub-group, since the onset of mental illness commonly occurs at this life stage and may impact the attainment of daily life skills. Further barriers to the attainment of daily life skills include sporadic periods of childhood and adolescent homelessness and foster care placements (Dworsky et al., 2013). If such skills were learned before or after early adulthood, they have often deteriorated as a result of untreated mental illness and periods of homelessness, hospitalization, or incarceration (Tsemberis et al., 2012).
In the US, homeless adults typically obtain shelter placements lasting several months to 1 year. During this time, case managers help residents to complete applications for federal and state subsidized permanent supportive housing (Tsemberis et al., 2012). Occupational therapists commonly do not provide services in either the shelter or supportive housing environment. Although supportive housing involves the provision of weekly case management and as-needed medical services, residents are expected to live fairly independently.
Many homeless adults with chronic mental illness and substance use histories proceed through the shelter system and receive supportive housing without acquiring the skills needed to manage physical and mental health disabilities, and safely function in and maintain apartment living (Greenwood et al., 2013). Once in supportive housing, the lack of home management and health maintenance skills can lead to problems such as hoarding, unsafe use of stoves and appliances causing fire safety hazards, failure to take prescribed medication resulting in illness events and rehospitalizations, and inability to budget subsidy payments for food and self-care (Khandor et al., 2011; Raphael-Greenfield and Gutman, 2015). Although some occupational therapists have been involved in the development of living skills with this population (Boisvert et al., 2008; Gutman et al., 2004; Helfrich and Fogg, 2007; Helfrich et al., 2006), there remains a critical need for therapists in the US to develop and assess interventions to help homeless adults with mental illness live optimally in supportive housing.
Description of SMART (Supporting Many to Achieve Residential Transition) Program modules.
Each of the six SMART modules is based on hybrid education theories and multimodal learning. Hybrid education involves the use of both digital materials and instructor delivery of educational content (Biddix et al., 2015). We selected hybrid education because it is paced at the learner’s processing speed and allows the learner to integrate audiovisual learning, hands-on practice, and consultation with a face-to-face facilitator. Because many chronically homeless adults with mental health and substance use histories have cognitive problems and limited education (Hyman et al., 2011), we believed that hybrid education would best facilitate learning. Multimodal learning involves the use of visual, aural, and kinesthetic materials to enhance the educational process (Canazza and Foresti, 2013). Each SMART module uses digital screen images overlaid with printed words and verbal narration to support comprehension. Screen text was designed to be easily readable, with accompanying images that reflect similar demographics to those of program participants. Each 1-hour module can be paused so that a facilitator can engage participants in hands-on functional activities that allow the practice of skills presented in the modules; modules can be used with groups or individuals. The SMART Program has been manualized, and facilitator scripts, interventionist training procedures, fidelity checklists, and intervention guidelines were developed for all six modules. Although occupational therapists have been involved in the development and assessment of home management skills for homeless adults (Boisvert et al., 2008; Gutman et al., 2004; Helfrich and Fogg, 2007; Helfrich et al., 2006), we are unaware of the creation of digitally based programs combining multimodal and hybrid learning formats with hands-on practice for this population.
The purpose of this study was to assess program effectiveness with an intervention and control group selected from a convenience sample. Our research question asked whether participants who attended the SMART Program would have higher transition rates to supportive housing, attain greater progress toward apartment living goals, and report higher quality of life compared to controls who did not attend the program.
Method
Research design
This study assessed program effectiveness using a two-group, controlled trial in which one group attended the SMART Program and received treatment-as-usual, while the control group received only treatment-as-usual. The program was implemented over 6 weeks, using twice-weekly 1-hour sessions. Approval for program assessment was obtained by the Columbia University Medical Center (CUMC) institutional review board and shelter agency, and all participants provided written consent.
Participants
A convenience sample of 20 homeless adult men was recruited from a large New York City shelter over a 1-week period. Participant mental health diagnosis, ethnicity, and level of education were obtained from self-report and clinical chart audit by a case manager. Inclusion criteria were English speaking, interest, and availability to attend the program. The first 10 participants who met the inclusion criteria were enrolled. Control participants (n = 10) were enrolled based on interest in program participation but were unable to attend due to conflicting schedules. Both control and intervention arms received treatment-as-usual.
Instruments
Examples of participant SMART (Supporting Many to Achieve Residential Transition) Program goal topics.
Example of a participant goal attainment scale where 0 is the expected outcome level, −2 is the most unfavorable outcome, and +2 is the most favorable outcome.
The Manchester Short Assessment of Quality of Life Scale (MANSA) was designed for community-based mental health populations and is a 16-item, seven-point Likert self-report scale (1 = could not be worse, 7 = could not be better) that can be completed in 10 minutes (Priebe et al., 1999). The scale has convergent validity with the Lancashire Quality of Life Scale (r = 0.83, p < 0.05) and moderate internal consistency with a Cronbach’s alpha of 0.74, p < 0.001. In this study the MANSA was completed by participants at pre- and post-intervention.
Intervention
Intervention description
The SMART Program was implemented as a 6-week, twice-weekly program; each of the six modules was delivered in 1-hour sessions over 2 days. We limited program participation to a convenience sample of 10 shelter residents, who were the first to consent to participation. This sample size was based on educational literature suggesting that small group sizes best facilitate learning, particularly when students have learning difficulties (Berkeley et al., 2011). Sessions occurred in the shelter’s occupational therapy room, which was equipped with a small simulated apartment. Program materials were delivered through a DVD format that could be paused at specific intervals to allow for hands-on practice of apartment living skills. Each module was manualized to enhance intervention fidelity and consisted of a facilitator instruction manual, appendix of practice skills and activities, facilitator training procedures, and fidelity checklists.
Facilitator
The facilitator, who was external to the research team, was an occupational therapist with 25 years of experience in mental health practice and was assisted by two graduate occupational therapy students carrying out Level II fieldwork. The facilitator received two hours of training in program implementation by the first author, along with weekly 1-hour consultations before and after each session. During the study, the facilitator provided intervention to SMART Program participants only and was blinded to group assignment.
Intervention fidelity
To assess intervention fidelity, a nine-item fidelity checklist was used to ensure that the facilitator implemented modules as intended. The checklist was completed by the first author once per week over the 6-week program to rate the facilitator’s congruence with manualized intervention guidelines. The facilitator averaged a 95% agreement with manualized instructions over the 6-week intervention.
Treatment-as-usual
Both control and intervention group participants received treatment-as-usual. Treatment-as-usual was provided by shelter staff (employees and volunteers of the shelter) and consisted of weekly case management, psychiatry, occupational therapy, recreational therapy, and medical and dental care. Occupational therapy consisted of group and individual sessions addressing resident housing goals (such as meal preparation, resume writing, self-care); cooking, exercise, current events, and music groups were run weekly. Treatment-as-usual occupational therapy services were provided by CUMC doctoral occupational therapy students and faculty (who were shelter volunteers and external to the research team).
Data collection
In the week before intervention, the second author (an occupational therapist) met individually with control and intervention group participants to conjointly develop and rank three personalized goals addressing the shelter transition and apartment maintenance process. At this time, the MANSA was also completed by each participant. One week post-intervention, the second author again met with each participant to determine progress made for each GAS. Participants completed post-intervention MANSA scales at this time. The second author was blinded to participant group assignment. At 6-month follow-up, the second author contacted each participant’s case manager to determine whether and when participants transitioned from the shelter to supportive apartment living.
Data analysis
T scores were used to determine if a difference existed between control and intervention groups on GAS scores at post-intervention. Using the formula below, pre- and post-intervention GAS scores were converted to T scores:
The T score is a standardized score with a mean of 50 and a standard deviation of 10. The standardization of T scores allows their comparison to a larger population. When T scores equal 50, they indicate a raw score of 0 on the original GAS (the participant’s expected level of performance). When T scores are greater than 50, they indicate that the participant performed above his or her expected level. When T scores are below 50 they indicate that the participant regressed in performance, below his or her expected level. The conversion of raw scores to T scores allows clearer interpretation of the participant’s overall score when multiple, separate goals are involved. T scores also allow the participant’s weighting of each goal to be factored into the final outcome score (Kiresuk and Sherman, 1968; Ottenbacher and Cusick, 1990, 1993). Once T scores were calculated for all participants, a Mann Whitney U test was used to determine if a statistically significant difference existed between control and intervention groups at post-intervention (Portney and Watkins, 2015).
A Mann Whitney U test was also used to determine if a statistically significant difference existed between control and intervention group MANSA scores at post-intervention (Portney and Watkins, 2015).
Data were analyzed using SPSS version 23 and significance level was set at α < 0.05. A power analysis suggested that with a sample size of 10 per group with a standard deviation of 5, we were powered to detect between-group differences of 6.6 points on GAS T scores with a power of 0.8 at α < 0.05.
Results
Intervention and control group participant breakdown of demographics.
GED: General Educational Development exam.
At pre-intervention, all participant-selected GAS scores were set at −1 (the participant’s present performance level). A Mann Whitney U test found no statistically significant difference between pre-intervention MANSA scores for both the intervention (M = 39.14, SD = 9.45) and control groups (M = 38.62, SD = 7.06), suggesting that there was no difference in their quality of life ratings.
Participant goal attainment scale (GAS) and T scores.
Participant Manchester Short Assessment of Quality of Life Scale (MANSA) quality of life scores.
At a six-month follow-up, four (57.14%) intervention group participants had transitioned into supportive housing while only two (25%) of the control group participants had transitioned. A case manager reported that, at the 6-month follow-up, problems encountered by the intervention group participants who had transitioned into supportive housing included loneliness and anxiety. In addition to loneliness and anxiety, control group participants who had transitioned were also reported to have problems with hoarding and medication compliance.
Discussion
Our research question asked whether a group of homeless men with chronic mental illness and substance use histories who received the SMART Program intervention would have higher transition rates to supportive housing, attain greater progress toward apartment living skills, and report higher quality of life compared to control group participants who did not attend the program. Our data analysis found statistically significant differences between control and intervention group GAS T scores and MANSA scores, suggesting that the intervention group participants made greater progress toward apartment living skill goals and reported higher quality of life at intervention end. At a six-month follow-up, four (57.14%) intervention and two (25%) control group participants had transitioned to supportive housing, suggesting that SMART Program attendance may have helped intervention group participants better learn needed apartment living skills, report greater life satisfaction, and achieve supportive housing transition. Although both groups received treatment-as-usual, it appears that the addition of the SMART Program helped intervention group participants to make greater progress compared to controls.
Limitations
One study limitation was a small sample size and a 25% attrition rate. Our attrition rate was similar to or lower than other intervention studies with homeless adults having mental health and substance use histories (Hobden et al., 2011). Although this population, once sheltered, commonly receives medical and psychiatric health care, medication compliance is fragmentary and is often not required for shelter placement. Reduction of substance use is encouraged but is also difficult to achieve, particularly when substances such as K2 (synthetic cannabinoids) have become epidemic international problems driven by ready availability and inexpensiveness (Aikins et al., 2015). An acknowledged limitation of this study involved our inability to collect information about participants’ substance use.
A power analysis suggested that our sample size of 20 participants was sufficient to detect between-group differences at 6.6 points on GAS T scores—a power level that decreased after a 25% attrition rate. Yet, despite our small sample, our analysis still found a statistically significant difference between control and intervention group GAS T scores and MANSA scores at intervention end, suggesting that it was likely the SMART Program intervention that caused observed differences. Although we deliberately limited group size to 10 participants to enhance learning in a sample of adults with limited education and cognitive problems, we should have enrolled a minimum of 15 participants per group to achieve a study sample size of 20 after attrition.
Another limitation was a lack of randomization. We chose not to use a randomized waitlist control group because we were unable to run the SMART Program within 6 months of study completion. Although there was considerable interest in attending the SMART Program, we found that many residents were unable to attend the program at a regularly scheduled time. We therefore enrolled the first 10 residents who were both interested and able to attend. Residents who were interested but could not attend due to scheduling conflicts agreed to serve as controls. The lack of randomization limits the generalizability of the findings; however, all participants—both control and intervention—shared similar demographic variables which mirrored US national statistics on homeless adults with chronic mental illness and substance use disorders (US Department of Housing and Urban Development, 2015).
A final limitation involved the inability to collect outcome measures, other than transition rate, at a 6-month follow-up. Once shelter residents transitioned to supportive housing they were relocated at distances that were difficult to access without additional data collectors. As a result we were unable to collect 6-month post-intervention GAS and MANSA scores, which may have provided information about the SMART Program’s effectiveness over time.
Future research
Future research should involve larger sample sizes, randomization, and 6-month and 1-year follow-up measures. Future research should determine if the SMART Program would benefit formerly homeless adults now residing in supportive apartment living who have difficulty with community participation, and apartment, money, and chronic illness management. The SMART Program is currently being expanded to include 12 new modules addressing supportive education and employment for homeless adults who have an interest in and are able to resume student/worker roles. These modules will be assessed for effectiveness and satisfaction in future research. Access to all SMART modules will be available at no cost on the CUMC Occupational Therapy Program’s website.
Conclusion
This study found that a supportive housing program for homeless adults with chronic mental illness and substance use histories helped participants achieve housing goals and higher quality of life scores when compared to a control group who did not attend the program. Although occupational therapists in the US are increasingly providing services to this population, the effectiveness of our services must be documented to ensure service reimbursement and secure our role in this community-based arena.
Key findings
A large sub-group of the homeless population has chronic mental illness and substance use disorders that have contributed to a gap in knowledge about independent apartment living skills.
What the study has added
Occupational therapy-developed housing programs for adults with histories of homelessness, mental illness, and substance use can help this group attain needed skills in preparation for the transition to supportive housing.
Footnotes
Acknowledgments
We thank Lee Ann Westover, Joseph White, Etta Graham, and Lauren Zavier for their assistance in this work.
Research ethics
This study was approved by Columbia University Medical Center’s institutional review board and the ethics committee of the shelter agency. All participants provided consent.
Declaration of conflicting interests
The authors declare that there is no conflict of interest.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
