Mental Health Evidence in the American Journal of Occupational Therapy

Effectiveness Studies

Two of the seven articles addressed effectiveness of occupational therapy intervention in mental health practice (Chan, Tsang, & Li, 2009; Gutman, Kerner, Zombek, Dulek, & Ramsey, 2009). The study by Gutman et al. (2009) used a quasi-experimental pretest–posttest design with random assignment to an experimental and control group to examine the effectiveness of supported education for adults with psychiatric disabilities compared with treatment as usual from the clients’ home mental health facility. Quantitative and qualitative data were collected at preprogram and postprogram points and follow-ups of 1 and 6 mo, addressing targeted behaviors, skills, and outcomes of study participants. As a result of its design, this study achieved a ranking as Level II evidence. The findings demonstrated the efficacy of providing skills training directly related to the participants’ identified roles of student or worker. Such skills included reading, studying, time management, math and writing skills, computer and Internet skills, library skills, public speaking, stress management, and professional behaviors and social skills. This study also measured occupational therapy students’ comfort level working with clients with mental illness. Results indicated that students’ comfort level with clients increased from the first session to the final session 6 wk later.

Chan and colleagues (2009) presented a case report that discussed the benefits of their integrated supported employment treatment program for an individual with severe mental illness. They found that this person used learned skills with support and sustained employment for significantly longer duration than any of the individual’s previous employment efforts. A case report is considered Level V evidence by AOTA because of its descriptive nature and limited generalizability.

Instrument Testing to Establish Reliability and Validity for Occupational Therapy Assessments

Two articles (Rieke & Anderson, 2009; Quake-Rapp, Miller, Ananthan, & Chiu, 2008) focused on methodological research that was descriptive in nature and addressed the use and development of assessment tools. Rieke and Anderson (2009) sought to establish the discriminant validity of an assessment tool to measure sensory processing of people with obsessive–compulsive disorder (OCD). The scores of the sample with OCD were compared with the adult age standardization group scores. They found that people with OCD scored differently from the standardization tables in the Adolescent/Adult Sensory Profiles manual, but the study did not demonstrate whether this finding has clinical significance.

Quake-Rapp and colleagues (2008) discussed the development, use, and validity of a behavioral observation assessment used with people with severe mental illness across activities in natural environments. The purpose of the study was to identify the incidence of maladaptive behavior in youths in a community support training program. The authors used a single cohort of 30 adolescents with four trained observers collecting data. Although the authors targeted measuring adolescent behavior as the purpose of the study, this article primarily described the data collection method and the interrater reliability of the trained observers’ use of the tool developed to collect the data. The authors found that trained observers achieved high interrater reliability using partial-interval time sampling for high-frequency maladaptive behaviors related to compliance and tasks but had inconsistencies and lower reliability for recording of less frequent behaviors of violent episodes and sexual inappropriateness. Quake-Rapp and colleagues discussed the need for valid and reliable occupation-based behavioral observations for assessment across settings with the ability of recording direct observations of behavior for assessment and reevaluation purposes.

Basic Research: Correlation and Descriptive Studies

Three studies (Magasi & Hammel, 2009; Mann et al., 2008; Zimolag & Krupa, 2009) contributed to basic research by providing descriptions of populations and population needs for wellness and community participation. The study conducted by Magasi and Hammel (2009) was identified as a qualitative study through its methodology and outcomes and so did not receive an evidence rating in accordance with the evidence table used for this review. Zimolag and Krupa (2009) and Mann et al. (2008) were identified as descriptive studies that did not measure intervention outcomes and did not receive evidence ratings.

Magasi and Hammel (2009) conducted an ethnographic study of women with disabilities with secondary diagnoses of a mental illness, ages 38–59 living in nursing homes. They identified five themes related to social justice, social support, personal control, political economy, and strategies of resistance connected to these women’s lived experiences of the nursing home environment. They concluded that it was important for occupational therapists to be advocates and liaisons of infusing nursing homes with occupation and choice and providing awareness of legislation regarding least restrictive environments and a range of community living options by providing information to clients and families.

Mann et al. (2008) conducted a post hoc review of data accrued from a larger 3-yr study that described an elderly population (≥ age 60) with depression with demographic data regarding marital status, ability and disability, and use of assistive technology. This observational descriptive study found that the severity of depression decreased over 3 yr despite increased physical disability with no causal relationship identified. The authors also found that elderly people increased acquisition of assistive devices as they aged and suggested that future research, through a larger study, should investigate whether depression and use of assistive devices were correlated.

Zimolag and Krupa (2009) conducted a descriptive study using survey data related to pet ownership by people with chronic mental illness. They found that people with chronic mental illness who owned and cared for pets had better perceptions of their social and psychological community integration than nonpet owners. No differences for physical community integration were noted between pet and non–pet owners with chronic mental illness.

Studies That Demonstrate Linkages Between Occupational Engagement and Health

None of the reviewed articles effectively evaluated the links between occupational engagement, participation, and health factors. Although the study by Mann et al. (2008) attempted to do so, the clarity of relationship among health variables was confounded by the multitude of variables mined for possibilities. Zimolag and Krupa (2009) attempted to relate pet ownership with increased community integration and personal well-being, with mixed results. Clients with pets tended to have higher Global Assessment of Functioning Scale scores and perceived well-being, but there was no difference in community integration between pet owners and non–pet owners with mental illness. The articles by Gutman et al. (2009) and Chan et al. (2009) supported occupational participation and role development in the contexts of work and education; however, they did not examine specific health factors in relationship to occupational engagement. Magasi and Hammel (2009), while exploring perceptions of six women with disabilities living in nursing homes, did not link health factors with occupational engagement but elicited further information related to social support and occupational justice. These studies demonstrated the benefits of engagement in educational, work, social, pet care, and daily living occupations to perceptions of wellness and community integration of clients as well as the effects of hospitalization on loss of occupations and engagement.

Meeting the Challenge of the Centennial Vision for Mental Health Practice

The number of published articles addressing mental health practice was very limited. The level of research evidence, with one exception, would also be considered weak, and there was only a limited direct relationship or relevance between the research described and clinical practice with clients with mental illness. No efficacy studies or systematic reviews regarding occupational therapy and mental health were published in AJOT during this time. The limited number of articles published reflects the current dearth of mental health evidence supporting practice published in AJOT.

Self-reflection is often a challenging and difficult task. In mental health practice, we often ask our clients and patients to reflect on their behavior to determine their priorities, make behavioral changes, or simply attend to the feedback that surrounds them. In a similar way, the editor of AJOT has provided us with a moment for self-reflection and the opportunity to examine the research generated by occupational therapy scholars and published in the national association’s journal.

We cannot overinterpret the finding that only 3.9% (seven articles) of all the articles published in AJOT during 2008–2009 addressed mental health practice. We have only the results and not an explanation as to why there were not more articles. There are probably many reasons for this low number, such as limited number of submissions; alternative opportunities to publish in journals with a wider, more diverse readership; or differing editorial standards. Regardless of the explanation, this finding reflects the great need for additional effort on the part of researchers and clinicians practicing in mental health to address questions and provide evidence that the successful participation in occupation is an important component of maintaining mental health and a means to regain function after an exacerbation of a mental illness. If, as stated in the Centennial Vision, we, as a profession, are going to reclaim the practice area of mental health, we must provide high-quality evidence that demonstrates the effectiveness and importance of occupational therapy interventions. A representative part of this research should be found in our national journal. A national association’s journal is considered its flagship journal, with its best evidence reviewed and published. Health professionals and occupational therapists should desire to subscribe to the journal for evidence related to occupational therapy, its research, and decision choices for client care.

The nature of the research reviewed in this article suggests the great distance we must travel if we are to achieve the goals of the Centennial Vision. More important, only one article in this review focused on the use or acquisition of occupations or functional skills by people with mental illness (Gutman et al., 2009). The remaining studies were only tangentially related to mental health practice, often solely because people with a mental health diagnosis were included in the study. Moreover, the research reviewed for this article does not address ways to support or change occupational performance or the importance of an occupational therapy perspective.

Another important point to consider is the strength of the evidence generated. The article by Gutman and colleagues (2009) was the only study to achieve an evidence rating of Level II. The other effectiveness study was a case report that qualifies for an evidence rating at Level V, which is considered a very low level of evidence. Five of the seven articles, one of which was a qualitative study, did not achieve an evidence rating. Although the studies were descriptive and contribute to developing questions for future research, they were not studies of efficacy or effectiveness of occupational therapy interventions and, as such, provide little support for the benefits from occupational therapy. Most systematic reviews would not generally include evidence at this level because of its limited ability to provide support for decision making. Likewise, the studies provide limited support for the effectiveness of occupational therapy mental health practice.

Note that the goal of this review is not to disparage the limited evidence that has been produced but rather to answer the question, “What is the state of affairs regarding research published in AJOT relative to the Centennial Vision and mental health practice?” On the basis of the evidence available, we must conclude that with AJOT’s current publication record, we may not be achieving the goals set out in the Centennial Vision.

Occupational therapy research in mental health is being produced by occupational therapists in countries around the world, including Canada, Australia, Spain, Korea, Britain, and China. It is also being published in journals specific to mental health, such as Occupational Therapy in Mental Health and Psychiatric Rehabilitation Journal. A growing body of research supports treatment interventions within the scope of mental health occupational therapy practice as discussed in the Occupational Therapy Practice Framework (AOTA, 2008; Gibson et al., in press). However, this research is being carried out by mental health professionals other than occupational therapists. Occasionally, such studies will define or view occupational therapy in a way that is reflective of practice from more than >30 years ago (Anzai et al., 2002; Kopelowicz, Liberman, Wallace, Aguirre, & Mintz, 2006; Liberman et al., 1998). The view of psychosocial occupational therapy as only traditional arts and crafts activities is very limiting and not representative of current practice (Chan, Lee, & Chan, 2007; Hayes, Halford, & Varghese, 1992). This view can be addressed only when we as occupational therapists assume the responsibility for defining occupational therapy and demonstrating its effectiveness through research.

Directions for Future Research

Any expectation for future research in the mental health practice arena must be linked with the policies of occupational therapy practice and education and individual behavior. We cannot expect therapists to practice in mental health without concerted efforts to recruit and train people interested in this practice area. Our education programs cannot erode, dilute, or disparage mental health education and must strengthen the research training of our students. Simultaneously, because AOTA once supported licensure across the 50 states, it is necessary that it actively support the designation of occupational therapists as qualified mental health practitioners. It is important that we, as members of the association, band together with all our numbers to orchestrate efforts state by state if not nationally to once again gain recognition as licensed professionals who provide beneficial services to clients with mental illness. In concert, we need to let reimbursers and policymakers know that we are well trained and provide cost-effective, efficacious interventions to address the issues of mental health related to illness, growth and development, life transitions, wellness, aging, and all aspects of promoting a healthy society.

Practitioners also have responsibilities. We must ask questions about our practice, seek evidence, and create treatment programs based on that evidence. We must contribute to research, and when we do, we must see that occupational therapy is represented appropriately. Finally, we must see ourselves as researchers and clinicians. It is important for those of us in daily contact with clients to develop research agendas and to determine what aspects of practice we can support through our questioning and demand for evidence.

Although occupational therapy research is lacking in all areas, the two major research gaps in mental health practice are occupational therapy intervention effectiveness and efficacy. Intervention effectiveness and efficacy are most important because these are the areas that reimbursement sources and policymakers look at for allocating funds. Parity and access to care are still impediments for clients with mental illness. Only those services that are efficacious and effective for promoting improved health and community reintegration will warrant reimbursement, because accountability now relates to outcomes. All occupational therapy practitioners, clinicians, researchers, and educators need to collaborate and talk to generate research questions and best methods for data collection and analysis. These questions may focus on evaluating program outcomes relative to time, intensity, and quality of intervention methods, or we may want to consider the evaluation tools used and whether these tools are the best methods of measurement for our clients’ performance and our intervention programs. Along with evaluating mental health practice in typical mental health contexts, we need to remember that mental health remains part of our holistic practice philosophy regardless of clients’ diagnoses and investigate the mental health aspects of occupational therapy intervention in all contexts. It is essential that we support each other in these processes.

In conclusion, we thank and commend the seven groups of authors for their research and professional commitment to the sharing of their mental health knowledge and practice and meeting the charge of the Centennial Vision. We are encouraged that occupational therapists have taken up the gauntlet and are venturing into the realm of research in support of practice. Finally, we have hope that all of us will take up the professional responsibility by answering the call to evaluate and evolve occupational therapy practice powerfully, artfully, compassionately, scientifically, and with evidence.