The effects of stress management interventions among police officers and recruits

1.1 PRIOR REVIEWS

The lack of quality data describing the effectiveness of stress management interventions for police officers was highlighted most recently by Penalba, McGuire and Leite (2009), who conducted a Cochrane review to assess the effectiveness of psychosocial interventions given to law enforcement personnel (police officers and military police) to prevent psychological disorders. Randomized and quasi-randomized controlled trial studies of psychosocial interventions were included. Of the 10 studies included in the review, the authors indicated that only five reported data could be used for analysis. None of the studies fully met the review quality criteria. Many were missing pertinent data and the data showing effectiveness was minimal, leading the authors to conclude that the studies were generally of low quality. The authors also concluded that performing a meta-analysis was not possible, and that there was insufficient evidence to indicate whether psychosocial interventions given to law enforcement to prevent psychological disorders were effective.

A systematic review investigating the effectiveness of stress management programs among other worker populations found similar results. van Wyk and Pillay-Van Wyk (2010) conducted a Cochrane review to examine the effects of staff support interventions focused on coping with work stress, preventing burnout, and improving job satisfaction among healthcare workers who were primarily nurses. Three preventive interventions were examined: (1) support groups; (2) stress management training; and (3) management interventions provided to support staff. The authors concluded that most of the 10 primary studies were methodologically weak and low quality. Overall, they found that insufficient evidence exists to show whether stress management interventions reduced work stress, prevented burnout, or reduced staff absenteeism. Limited evidence showed that stress management interventions improved staff moral and job satisfaction. The authors suggest that more rigorous studies are needed.

In an early review of stress management interventions aimed at individual and organizational factors, Newman and Beehr (1979) concluded that the basis for intervention efficacy was “professional opinions” (p. 35) and few research approaches were rigorous. More recent reviews have found mixed results. van der Klink, Blonk, Schene, and Dijk (2001) conducted a meta-analysis to examine the effects of four types of stress management interventions: cognitive-behavioral interventions, relaxation techniques, multimodal programs, and organization-focused interventions. They found a moderate effect for cognitive-behavioral and multimodal interventions, and a small effect for relaxation techniques. The effect for organization-focused interventions was not significant. Significant effects were found for psychological outcomes, complaints, and quality of work life. The authors concluded that stress management interventions are effective.

van der Hek and Plomp (1997) updated a review of stress management interventions initially conducted by DeFrank and Cooper (1987). They examined studies published between 1987 and 1994 and concluded “it is still impossible to determine which specific interventions or techniques are most effective and should be recommended” (p. 140). When significant effects were found, it was difficult to assess which components on the intervention produced the effects. The basis for these conclusions was the heterogeneity among studies which made it difficult to compare studies, not a lack of sufficient studies.

Richardson and Rothstein (2008) updated a systematic review originally conducted by van der Klink, Blonk, Schene, and Dijk (2001). They investigated stress management interventions among teachers, nurses, factory workers, maintenance workers and social service workers, among others found medium to large effects with a sample size of 2,847. In general, their results support van der Klink et al.'s but they also found that weaker studies did not change the effects and larger effects resulted when only true experiments were included in the analysis.

2.1 REVIEW QUESTION

What are the effects of officer stress management interventions on stress outcomes?

To answer this question we: (a) examined the conceptual differences in interventions given to police officers and recruits; (b) examined variations in approaches to interventions given to police officers and recruits; (c) synthesized the body of knowledge on sources and types of stress for police and recruits, and the strengths and limitations of the interventions; (d) discovered reasons for conflicting intervention effects (e.g., different curricula, different outcome measures, different research methods); and synthesized what is known about the impact of different study designs, research methods, interventions, and data analysis procedures on outcomes.

3.1 CRITERIA FOR INCLUSION AND EXCLUSION OF STUDIES IN THE REVIEW

The criteria for inclusion of retrieved studies focused on population characteristics and sampling strategies, interventions, study methods and designs, data analysis and outcome results.

(1)

The population includes veteran police officers, police recruits, and/or civilian (non-sworn) police personnel. Law enforcement organizations provide stress management training to recruits during academy training, whereas veteran police officers receive in service training (Finn & Tomz, 1997). The most obvious distinction between veteran police officers and recruits is the amount of work experience they possess, although other differences can also be noted. These include age, marital status, parental status and rank. These statuses are a source of stress. Because interventions may be given to veteran police and recruits together, in addition to including civilian law enforcement personnel, these three populations are included in the review. Civilians will be included because some stress management interventions include both sworn and non-sworn law enforcement personnel. Studies with a mixed population of police officers and non-police officers (e.g., teachers, nurses, firefighters) will be included in the review only if the findings can be extricated for police officers, police recruits, and/or civilian (non-sworn) police personnel.

3.2 SEARCH STRATEGY FOR IDENTIFICATION OF RELEVANT STUDIES

A variety of search methods were used to identify studies. These methods included: (1) searching electronic databases; (2) handsearching relevant journals, books, and conference proceedings; (3) searching Internet websites; (4) visually scanning reference lists from relevant studies; (5) contacting organizations and authors who have knowledge of police stress management and development program evaluations; and (6) citation searching.

We used the above methods to search for “grey literature” (e.g., book chapters, government reports, doctoral dissertations and master's theses, conferences proceedings, and organizations that provide stress management and development programs to police officers) in order to reduce publication bias.

The following electronic databases were searched:

The following Boolean search strategy was employed - police and ((stress or suicide or “substance abuse” or “alcohol abuse”) and (management or prevention or awareness or debriefing or development)) and (training or program*). After identifying studies, the studies were imported to the bibliographic software Refworks. Refworks was used to manage and share the studies among the review team, and create the American Psychological Association (APA) bibliographic style.

We searched 35 databases. The number of hit results ranged from zero to over 11,000. A low number of hits was sometimes indicative of a lack of relevant sources but was also sometimes the result of a lack of sophistication in the search engine in terms of processing a complex Boolean search strategy. An increase in hits was sometimes possible if the search strategy was simplified to “police and stress”, but such simplification often led to articles addressing police officers dealing with members of the public suffering from stress. A high number of hits also revealed a lack of ability in the search engine to process a complex Boolean search: in this case articles with any of the individual terms entered into the search were found, with the result that very few were related to the review topic. The key to fruitful searching in any of the databases was identifying relevant subject descriptors supplied by the database publisher. For example, Academic Search Complete uses the term stress management while BIOSIS Previews uses the term stress reduction: while the difference may seem minor, using the wrong terminology in a given database can significantly lessen the number and quality of hits.

As part of the search strategy, the review team contacted organizations and authors who perhaps have knowledge of police stress management and development program evaluations. Contacts with organizations and authors did not result in studies that could be used in the review.

3.3 DESCRIPTION OF METHODS USED IN THE COMPONENT STUDIES

While RCT designs are the preferred designs to assess intervention effectiveness, these designs are sometimes impractical in police settings particularly among police recruits who are required to receive the same training (and interventions). Observational designs (in which natural changes in interventions are examined (CRD, 2009)) and quasi-experimental designs (designs that do not use random assignment to control and experimental conditions) although weaker in external validity, are most likely to be used to assess stress management intervention effects among police officers and recruits.

The samples most likely used in the component studies will be volunteers who were randomly assigned to control and experimental conditions. It is likely that few studies will consist of samples randomly recruited from police departmental employee sampling frames.

Self report data are used most often to assess outcomes. A wide variety of outcome measures have been used to investigate stress management interventions for police officers and recruits. Measurement strategies include knowledge and attitudes about the effects of stress; alcohol use and abuse; and levels of anxiety and depression, for example.

3.4 DETAILS OF STUDY CODING CATEGORIES

The coding protocol (coding form, codebook) was piloted prior to beginning the primary data collection from retrieved studies. The pilot test also assisted the reviewers with testing procedures to resolve disagreements regarding relevance decisions. The coding form noted publication characteristics, sampling and population characteristics, intervention, measurement, design, and data analysis characteristics. In some instances these characteristics were specific to police officers and recruits, such as officers' rank.

3.5 STATISTICAL PROCEDURES AND CONVENTIONS

The meta-analysis was performed using the computer software program Comprehensive Meta-Analysis Version 2.2.050 (Borenstein et al. 2009a). For studies reporting multiple outcomes and time points these were not treated as independent. Effect size estimates may lack statistical independence because different effect size estimates may be calculated on the same participants using different outcome measures; effect sizes may be calculated by comparing different interventions within a single control group or different control groups within a single intervention group; different samples may be used in the same study to calculate an effect size for each sample; or a series of studies may be conducted by the same research team (Hedges, 1990). To address these issues we separated out the effect sizes by outcome type (psychological, behavioral and physiological) and analyzed effect sizes separately for different outcomes types. Among studies that contained multiple outcomes, the outcomes were averaged. Effect sizes were not averaged across different outcome types. Effect sizes were calculated using reported means, standard deviations and sample sizes, and reported Cohen's d and t-test results.

Averaging the effect sizes across studies may result in biased average effect estimates and incorrect sampling errors if the effect sizes from different studies vary in precision. Therefore each effect size calculated for a study was weighted proportional to its precision, giving more weight to studies with more precision and larger samples. The standardized measure of effect Hedge's g was used to estimate the effect size. This measure also provides standardization among the variables included in the primary studies (Lipsey & Wilson, 2001).

3.6 TREATMENT OF QUALITATIVE RESEARCH

The qualitative findings obtained from retrieved studies will be used to interpret the quantitative results and to provide contextual information (CDR, 2009). Studies that utilize qualitative methods (focus groups, interviews) will be included in the systematic review only if these studies meet the other inclusion criteria, and are included in a study providing quantitative data. In other words, mixed methods studies that meet the inclusion criteria will be included in this systematic review. In this way consistency will be maintained and the qualitative data will provide contextual information to the quantitative data.

4.1 DESCRIPTION OF ELIGIBLE STUDIES

Our search and contacts with organizations and authors who may have had knowledge of police stress management and development program evaluations occurred over a 9 month period from August 1, 2009 through May 31, 2010. The last search was conducted on May 31, 2010. Retrieved studies were published between 1984 and 2008. The total number of titles and/or abstracts identified from the search strategy and screened was 678. Of these titles and abstracts, 25 studies were retrieved. Upon closer examination, 13 of these 25 studies were excluded due to lack of insufficient data to perform the analysis. Studies were excluded due to the lack of either pre-test or posttest data, or the lack of separate outcome data for police officers. Table 1 provides examples of characteristics among the 13 excluded studies.

Table 2 provides a description of the 12 primary studies included in the systematic review.

The studies were published between 1984 and 2008. The studies included in the analysis comprised a total sample size of 906 participants. One study included a comparison group in addition to the experimental and control group (Coulson, 1987). The comparison group was excluded from the analysis. Among the 10 studies reporting gender, 401 participants were male and 91were female. Among the 11 studies reporting age, the average age was 34.48 (SD = 3.57). As Table 2 also shows, three studies reported racial/ethnic data. The years of police experience officers possessed were reported among 7 studies. Years of police experience excluded samples of police recruits. The average years of police experience was 10.77 years (SD = 4.00).

Only 3 studies reported the rank of the officers which were: below the rank of sergeant; police officer, detective and administrative officers; and deputy chiefs, commanders, lieutenants, sergeants, and police officers. Seven studies contained samples comprised only of police officers; two studies reported a sample of police officers and civilian personnel; one reported a sample of police officers and recruits while another included a sample of police officers and significant others, and one study included a sample of police recruits.

As Table 2 also shows, the types of interventions provided were quite varied. Among the 12 studies, 10 interventions were described: stress management program; stress reduction program; stress inoculation; brief interventions; counseling; nutrition and physical conditioning; circuit weight training; Eye Movement Desensitization and Reprocessing (EMDR), visual-motor behavior rehearsal (VMBR), and a writing intervention. Eleven of the primary studies provided a citation for the prior work establishing the basis for the intervention.

One study described the development and testing of a unique HeartMath stress and emotional self-management technique (McCraty & Tomasino, 1999). One study investigated an intervention as a treatment for police officers diagnosed with PTSD (Gersons, Carlier, Lamberts, & van der Kolk, 2000), and was the only study which required that study participants experience stress prior to or as a condition to participating in the study. The remaining 10 studies investigated interventions intended to either reduce stress outcomes or assess outcomes. As such, it was difficult to assess whether the interventions were intended as preventative interventions.

The length of the interventions ranged from a minimum of 1 week to a maximum of 16 weeks. When measured in minutes and hours, the duration of the interventions ranged from 35 minutes to 24 hours. The average length of the interventions was 10.95 hours (SD = 7.33).

4.2 OVERALL MEAN EFFECT ACROSS STUDIES

Effect sizes were calculated from 8 published studies, 3 unpublished doctoral dissertations, and 1 unpublished report. The effect size was grouped by outcome type (psychological, behavioral or physiological) with separate analyses for each outcome. Multiple effect sizes for an outcome type were averaged. Table 3 reports the standard difference (Hedge's g) and 95% confidence intervals under the random effects model among behavioral outcomes. A total of 30 effects were combined. As the table shows, across the 3 studies, the effects ranged from -0.695 to 0.194. The mean effect was -0.176. These represent a very large negative effect to a very small positive effect. Table 4 shows the random effects model among physiological outcomes. Twenty-nine effects were combined across 2 studies. The effects ranged from 0.030 to 0.324, or near zero to medium effects. The mean effect was 0.196.

Table 5 presents the results of the random effects model among psychological outcomes. A total of 162 effects were combined to yield effect sizes that ranged from -0.485 to 0.975, a moderately large negative effect to a very large positive effect. The mean effect was 0.038.

4.3 HETEROGENEITY OF EFFECTS

The Q statistic was calculated to test the null hypothesis that all studies share a common effect size and to examine whether the heterogeneity among the 12 studies was statistically significant. As Table 6 shows, the Q value was statistically significant for psychological outcomes (p = 0.035). The Q values were not statistically significant for the physiological and behavioral outcomes. As such, we also computed I². These results are also shown in Table 6. I² was zero for physiological outcomes, suggesting little meaningful variability across effect sizes. I² was 47.23 among psychological outcomes suggesting that 47.23% of the variability among the effect sizes was the result of heterogeneity between the studies. Similarly I² was 61.69 among behavioral outcomes suggesting that 61.69% of the variability among the effect sizes was also the result of heterogeneity between the studies. However, these results should be interpreted with caution because both the Q and I² statistics have limited statistical power due to the low number of studies included in the analysis (Borenstein et al., 2009b).

4.4 ANALYSIS OF MODERATOR EFFECTS

Moderator analyses were performed among the 12 primary studies using the psychological outcomes only. Seven moderator variables were theorized to influence stress management intervention outcomes. These include: (1) length of stress management intervention - six studies provided a one day intervention and six studies provided a two day intervention. The length of the interventions was coded as either one day or two days in duration. The review team reached consensus to code the length of the interventions as either one day or two days in duration believing that an intervention described as lasting 4 weeks, although the actual intervention was only 15 minutes per day, could be misleading; (2) type of stress management intervention - two studies provided a physical exercise intervention and ten studies provided a psychosocial educational intervention; (3) population (recruit, veteran police officer, civilian) - seven studies included samples comprised only of police officers; one included a sample of police recruits; one included a sample of police officers and significant others, two studies reported a sample of police officers and civilian personnel; and one study reported a sample of police officers and recruits. The later three studies were coded as police officers and others; (4) gender - three studies included male samples and seven studies included both males and females among the samples. One study did not include females in the final analysis due to attrition and one did not report the number of males and females after attrition. Only samples that participated in all time points were included in the present analysis; (5) years of police experience - years of police experience was coded as less than 10 years of experience, or more than 10 years. Three studies reported less than 10 years of police experience and four studies reported more than 10 years of experience; (6) random assignment to conditions - two studies used a control group without random assignment and 10 studies reported random assignment. One study used a control group but did not report how the group was obtained; and (7) attrition - five studies did not report the attrition rate among the sample and seven studies reported attrition among the sample. Among the seven studies reporting attrition, the dropouts were excluded from the final analysis.

Because only three studies reported the rank of the officers which were: below the rank of sergeant; police officer, detective and administrative officers; and deputy chiefs, commanders, lieutenants, sergeants, and police officers, the rank of the officer was not included as a moderator variable.

4.5 SENSITIVITY ANALYSIS

A sensitivity analysis was performed to examine the impact of each study on the mean effect. The analysis was performed in several ways. First, it was run with all studies included except the first study, then with all studies except the second study, and so forth. A second approach was performed with larger studies removed, then with smaller studies removed. Finally, the analysis was performed with the 7 moderator variables removed to assess the impact that these factors might have on the mean effect. Each analysis resulted in a zero effect.

4.6 SUMMARY OF QUALITATIVE DATA

Of the 12 studies included in the review, only one study (Richmond, Kehoe, Hailstone, Wodak, & Uebel-Yan, 1999) included a focus group component to explore attitudes toward alcohol consumption. The sample consisted of 43 participants of both genders and different police ranks, who were randomly assigned to 5 focus groups. The findings of the focus group discussion provided some contextual information regarding the lack of significant improvement in reducing excessive alcohol consumption, smoking and stress symptoms among the police officers who participated in the study's intervention. These factors included the lack of trust held by police officers toward the law enforcement organization's involvement in health issues, the perception that alcohol consumption was a private matter unless it interfered with job performance, and the stressors inherent in the work environment were conducive to the use of alcohol consumption as a coping mechanism and a central part of the police culture for bonding among officers.

Another study (Wilson, et al, 2001) conducted clinical and exit interviews with police officers and their significant others. The clinical interviews focused on the job stress inherent in police work, organizational issues, and personal issues. Officers and their significant others were asked to describe one stressor for each area. Officers reported stressors such as inadequate salary, excessive paperwork and working overtime. These stressors formed the basis for the stress management interventions provided to the experimental and control groups. The exit interviews were also conducted with each couple focused on whether police officers and their significant others participated in interventions after completing the stress management intervention as well as any recommendations for providing the stress management intervention differently in the future, among others. Most of the qualitative data obtained from the exit interviews was not reported in the study although officers' significant others reported observing positive changes in their officer spouses following participation in EMDR, officers who received the EMDR stress management intervention referred colleagues to the intervention and many of the significant others expressed a desire to have participated in the EMDR intervention.

4.7 PUBLICATION BIAS

Only including published studies in the analysis can result in a file drawer problem. Omitting unpublished studies in a meta-analysis can result in findings that are misleading (Glass, 1981), and overestimate the true effect size (Borenstein et al., 2009b). Moreover, to ignore dissertations based on assumptions that they lack rigor is unwarranted (Glass, 1981), and to assume that peer reviewed journal publications result in high quality studies can be misleading (Borenstein et al., 2009b).

To assess publication bias, the funnel plot by precision proposed by Borenstein et al (2009b) was used. The funnel plot is displayed in Figure 1. The results show that the two large studies are situated near the top of the funnel plot and centered around the mean, whereas the 10 smaller studies are situated near the bottom and are more dispersed around the mean effect size. Of these 12 primary studies, eight (67%) were published studies, and four (33%) were either an unpublished report or unpublished doctoral dissertations.