Safety interventions for the prevention of accidents at work: A systematic review

Types of studies not eligible for inclusion

1.

Studies measuring effects only after the intervention is applied

Primary outcomes

The primary outcomes of interest included: 1.

Incidence of accidental work injuries causing physical harm

We also included safety behavior and relevant injury risk factors as primary outcomes, as they are considered proxies for accidents at work (Laitinen et al., 2003; Laitinen & Päivärinta, 2010). Outcomes of mental or psychological harm resulting from an accident were excluded from the review.

Secondary outcomes

The secondary outcomes of interest included changes in knowledge and attitudes as well as changes in workplace norms, climate, or culture

Accidents at work are relatively rare events, from a statistical perspective. Safety-related behavior and workplace risk factors are, consequently, often used in the evaluation of safety interventions. In cases where we might identify changes in intermediate measures, such as knowledge levels, safety behaviors or climate, as well as in injury rates, this would add significantly to the knowledge of work injury prevention efforts.

Outcome measures

(NOTE: Effect measures are described more specifically in Section 4.3.5)

The primary outcomes of safety interventions are typically measured as dichotomous (binary) data on an individual basis (e.g., an individual is either injured or not in the timeframe of interest). However, on the group or organizational level, injury incidence rates are of primary interest. The rate may be expressed in many ways typically taking the form of the number of injuries/population size in a given time period. Cases of work disability and number of lost working days can be expressed similarly. It is not uncommon for these rates to be calculated based on a denominator of person-time, such as hours of work. Prevalence rates (%) are acceptable, given a stable work population over time. Changes in a relevant injury risk factor or safety behavior may also be measured as a binary variable (exposed—yes or no, or safe/correct behavior—yes or no) or as a percentage of improvement.

Measures of knowledge, attitudes, climate, and culture are typically expressed continuously where data can take any value in a specified range, for example, a safety climate scale. Measurement of safety attitudes as well as climate and culture in the context of safety intervention research are evolving constructs (Flin et al., 2000; Kines et al., 2011; Seo et al., 2004). We expect to see them measured in a variety of ways depending on the context of the work and the specific research questions. Consequently, we did not restrict our assessments to any pre-specified scales or indices.

The duration of follow-up was extracted for each study, allowing us to examine outcomes across a variety of time points. We assumed that the lag from intervention to effect would vary depending on the main types of intervention. Usually, interventions directed at the individual level would have a shorter lag than interventions directed at the group or organizational level, as for the latter there is usually a more complex relationship between the intervention and the outcome, and thus requiring a longer timeframe for implementation. Also we assume that there is a differential effect of interventions over time.

For all types of safety interventions we examined outcomes at the following time points: post-test (immediately after the intervention ends); short-term (up to 12 months); medium-term (from 1 to 3 years); and long-term (with a follow-up longer than 3 years). For example, we usually expect a longer time frame before legislation and enforcement will effect on accidents and that there may be differential effect at different time points.

Studies using RCTs or controlled before and after designs

Effect size statistics was used for the various types of safety interventions. In addition, a narrative analysis of studies was conducted to capture relevant knowledge of the effects of safety interventions from studies not included in meta-analysis. We reported these studies in accordance to intervention characteristics, intervention components and contextual factors (Lehtola et al., 2008), to evaluate the summarized effect of the various types of safety interventions.

Studies using serial measurements

Rather than beginning our assessment of studies using serial measures with a fully pre-specified assignment of categories of studies and quality criteria, we were more interested in how investigators sought to improve quasi-experimental or nonexperimental intervention studies across a broad range of occupational safety intervention efforts designed to prevent fatal and nonfatal work injuries. To make this presentation more transparent, we reported separately on these studies that were not combined in meta-analyses, in a manner focused on intervention characteristics and contextual factors (Lehtola et al., 2008).

For this report we included intervention evaluations with at least three measures before and three after the intervention, as well as studies with at least three measures, where one should be a premeasure, if the investigator described an approach that improved the internal validity of the study. We also included serial measures even if the interventions did not occur at a precise moment in time. Simple BA studies were not included in the analysis.

We took a grounded approach to this review activity to more clearly describe how investigators have used studies with serial measures in workplace injury intervention evaluations. This included attention to ways in which potential threats to validity were addressed. Each report was read by two members of the review team and data were independently extracted. Impressions were then compared and discrepancies discussed to reach consensus. In rare circumstances a third opinion was sought.

We attempted to capture descriptions of why investigators believe a given intervention should work, based on theoretical constructs or the epidemiology of the injury that is aimed to be prevented, and when that should be the case. We looked at the time period of the intervention itself and the length of follow-up, how the fidelity and/or adoption of the intervention were measured or described, and the analytical approaches and tools used. We also extracted contextual information provided by investigators to justify an approach and/or help in the interpretation of findings.

Approaches taken by investigators that addressed internal validity concerns in these nonexperimental designs and improved study quality were noted. Using the information from this text review we classified the study designs into categories that captured whether comparisons or other analytical strategies were employed. Furthermore, we noted additional design or analytical features used in the overall inferential process that were considered to strengthen the work.

Along with assessment of bias (as described earlier), each reviewer assessed whether a reasonable inferential process was described by the investigators, that included, but was not limited to, statistical inference. Ways the evaluation could have been further improved were noted and consideration was given as to whether the research questions could have been addressed using an experimental approach.

Overall characteristics of included studies

Section 9.1 presents the characteristics of each of the 100 included original studies. The included studies are mainly representing the western societies, as only few studies came from Africa and Asia (Table 1).

Study design

Of the 100 original studies included in the analyses 16 studies were using randomized controlled design (RCT), 30 were CBA studies, and 54 studies used serial measures (ITS).

The included RCT studies were randomized at group level, some using stratified or pair matched random samples and others simple random sampling. The 30 included CBA studies used various forms of control, such as historical controls and matched controls. The included 54 studies with serial measures (ITS) cover several different approaches. Simple serial measures taken from one group over time were more commonly used (n = 24). Other approaches included serial measures with internal comparison (n = 12), serial measures with external comparison (n = 10), serial measures with stratified analyses (by job, type of injury, focus of intervention, etc.) (n = 2), and serial cross-sectional measures (n = 2). Finally, we found hybrid designs using combinations of the above (n = 4).

The 100 studies used for the analysis of effectiveness of safety interventions, included 120 safety interventions in total (Table 2).

In the following we refer to these 120 safety interventions, if we are not specifically referring to the included number of original studies.

Participants, types of work, and types of injuries

The 120 safety interventions included 31,971,908 individuals in 59 safety interventions, 417,693 groups/firms in 35 safety interventions, and 15,505 injuries in 17 safety interventions. Out of the 59 safety interventions, two were evaluating national prevention measures which alone accounted for the 31,667,110 million individuals. The remaining nine safety interventions used other types of measures, such as safety exposures, safety observations, gloves, or claim rates.

Tables 10–15 (Section 12) provides an overview of the nature of included safety interventions using RCT, CBA, or serial measures (ITS), by type of safety intervention, including information on Participants, Intervention characteristics, Comparisons conditions, Outcome measures and Study design (PICOS).

Half of the 120 included safety interventions evaluated “all types of injuries” in the workplace (n = 61). The two largest specific types of injuries assessed were needlestick injuries (n = 20) and overexertion injuries (n = 20), such as dislocation, sprains, and strains, mainly in hospitals or health care institutions. Overexertion injuries were related to acute exposure in the workplace, such as the transfer or support of patients or clients. Safety interventions to prevent overexertion injuries were mainly studied in human health and social work activities, such as health care institutions. Five studies evaluated the effect of interventions on eye injuries. The effect of safety interventions on fatal injuries were only evaluated in four studies (Bulzacchelli et al., 2007; Derr et al., 2001; Menendez et al., 2012; Suruda et al., 2002), of which two were in the construction industry.

More than one-third of the 120 included safety interventions evaluated “all types of accidents” in the workplace (n = 46). The two largest specific types of accidents are contact with sharp or pointed materials or tools (n = 23) and overexertion of the musculoskeletal system (n = 19), mainly in hospitals or health care institutions. Collision and other horizontal impact on body (n = 10) were a type of accident involving transport and vehicles, and were mainly seen in transport and storage (n = 7). Few safety interventions evaluated assault or violence at work (n = 4), where persons were exposed to assaults from patients, prisoners or customers in retail.

The main groups of participants in the included safety interventions came from health and social activities (n = 35), manufacturing (n = 22), agriculture (n = 14), transport and storage (n = 11), and construction (n = 10). These groups made up 77% of the included safety interventions. These five sectors are generally considered as high-risk sectors. Mining and quarrying industry is also a high-risk sector, but only three of the included studies evaluated safety interventions in this sector.

Most of the included safety interventions used firms as the analytical unit, sometimes worksites, such as in the construction sector, or organizational units. Nearly half of the safety interventions were evaluated in large firms or public institutions with more than 250 employees (n = 53). Micro firms were mainly from the agricultural sector, evaluating safety interventions on smaller (family) farms (n = 5). For several safety interventions the company sizes were mixed, often evaluated with an ITS design, for example, as in the evaluation of legislation and enforcement. For about 9% of the safety interventions the size of enterprise was unclear (n = 11).

The theoretical basis or rationale for the interventions

This review used a conceptual model of safety interventions including five specific types of safety interventions and combinations of these (multifaceted safety interventions), thus providing six main types of approaches to reduce accidents at work (see chapter 2.3). The safety interventions directed at the individual level included attitude and beliefs modifications (n = 11), behavior modifications (n = 6), and modifications of physiological strengths and resistance (n = 5).

The interventions directed at the group or organizational level included culture, climate and normative changes (n = 11), and structural modifications (n = 51), where the latter included legislative changes and enforcement (n = 23), engineering controls (n = 19), and administrative controls (n = 2). Only two safety interventions included economic incentives. A large group of safety interventions investigated the combined effect of two or more safety interventions (n = 36), where the largest group involved safety interventions investigating the combined effects of components across individual and organizational levels (n = 21). In one study the type of safety intervention evaluated was unclear (Lanoie, 1992).

Most safety interventions provided some information about the rationale for the interventions, some more explicit than others. However, only a smaller subset of the studies provided a more integrated theoretical basis for why the safety intervention should work, and which was reflected in the design and measurements of intermediate outcomes.

The studies on attitudinal approaches used a theoretical or conceptual basis for the safety intervention to a limited extent. Most studies referred to common sense (Adams, 2013; Gadomski et al., 2006; Johnson & Owoaje, 2012) or following recommendations from authorities (Wang, 2003). One exception was Gregersen's (1996) study of drivers in transport, who used the insights from Kurt Lewin's (1947) theoretical and practical research on how to change habits by use of group discussions.

Evaluation of safety interventions based on changes in individual strengths and physiological readiness used references to physiological theory and principles, such as Physical Readiness Training (PRT) (Knapik et al., 2003), or knowledge about how connective tissues in tendons, ligaments, joints etc., become shortened and dense when not exercised, then limiting the range of motion, and thus risk of overexertion of muscles and tendons (Hilyer et al., 1990; Leffer, 2010).

In the 1980s and early 1990s we saw early work evaluating behavioral approaches to safety in manufacturing in the UK, Finland, and the US (Cooper et al., 1994; Fellner & Sulzer-Azaroff, 1984; Saari, 1998; Sulzer-Azaroff & de Santamaria, 1980; Sulzer-Azaroff et al., 1990). The reported safety interventions had follow-ups lasting from 4 months up to 3 years, with only one safety intervention lasting 36 month (Saari & Näsänen, 1989). Other variations of behavior based theories were referred to in the included safety interventions (Bena et al., 2009; Daltroy et al., 1997; Quintana, 1999). Jinnah et al. (2014) used a 16-item instrument to measure farm safety behavior of fathers and youth, guided by the TPB (Ajzen, 2012). Cheng and Chan (2009) combined behavior based theories with attitudinal changes guided by the linear KAP model. Other variants of the behavior based theories included social marketing approaches, as well as Rogers's theory of diffusion of innovation (Rogers, 1995), which was used by Chapman et al. (2011) to increase awareness and encourage adoption of safer dairy farming work practices in a 7-year follow-up study.

After more than 20 years of behavioral safety research, safety climate interventions attempt better integration with other domains of management research. From a theoretical standpoint, safety climate created a link to important constructs of management theory, for example, leadership (Zohar, 2002). Safety climate theory or components of safety climate were investigated in a number of studies (Cooper et al., 1994; Cunningham & Austin, 2007; Fellner & Sulzer-Azaroff, 1984; Kines et al., 2010; Moore-Ede et al., 2004; Sulzer-Azaroff et al., 1990; Zohar & Luria, 2002, 2003).

In the early 1990s studies focusing on reducing needlestick and/or sharps injuries appeared in correspondence to growing concern about the AIDS epidemic and later hepatitis concerns (Birnbaum, 1993; Lawrence et al., 1997; Zafar et al., 1997); these have continued (Cunningham & Austin, 2007; Phillips, 2012; Reddy & Emery, 2001; Rogues et al., 2004; Smollen, 2004; Sossai et al., 2010; Whitby et al., 2008). The latest of these is a report on effectiveness of the US Needlestick Protection Act passed in 2000. The rationale for the prevention of needlestick injuries is predominantly various forms of engineering controls, initiated by policies of the health care system or legislation (Grimmond et al., 2010; Jensen et al., 1997; Lawrence et al., 1997; Prunet et al., 2008; Reddy & Emery, 2001; Rogues et al., 2004; Smollen, 2004; Sossai et al., 2010; van der Molen et al., 2011; Whitby et al., 2008). Also teaching and educational efforts (Mehrdad et al., 2013; Wang, 2003), counseling approaches (van der Molen et al., 2011), and goal setting and feedback (Cunningham & Austin, 2007) were used to improve knowledge and attitudes and in turn safer behavior in handling needles. And finally these efforts were also combined in multifaceted approaches to prevention (Gershon, 1999; Srikrajang et al., 2005; Valls et al., 2007; Zafar et al., 1997). In these studies, investigators began making use of ongoing injury surveillance systems to address intervention evaluation.

By the 2000s studies focused on preventing injuries from lifting (transfer of patients) in health care appeared using engineering controls (Alamgir, 2008; Schoenfisch et al., 2013) or multifaceted safety interventions (Black, 2011; Chhokar et al., 2005; Evanoff et al., 1999; Fujishiro et al., 2005; Garg, 1999; Martin et al., 2009; Park et al., 2009; Passfield, 2003).

Structural modifications include varied approaches that change the physical, organizational, or regulatory environment. A common feature of the structural approaches is that environmental factors are changed, often over longer time periods or permanently, consequently with more profound effect. One type of structural modification is engineering control, for example, introduction of machine safeguards, walkways, elimination of hazardous substances or materials, and other changes in the physical environment that directly influences individuals' safety without necessarily affecting their behavior.

Preference for engineering control is based on the public health hierarchy of hazard control (Herrick & Dement, 1994; Lingard & Holmes, 2001). This approach follows the basic tenet of industrial hygiene regarding control of health hazards in working environments; it has been applied to the control of physical hazards responsible for energy transfer and subsequent accidents and injuries in the workplace as well (Castillo et al., 2006). Emphasis in this model is given to the most effective and efficient preventive measures that eliminate risk at the source of the hazard. Lower tiered approaches in the hierarchy control risk through barriers or use of personal protective equipment or training and education efforts. Of note, engineering controls typically focus on control of a specific hazard, in marked contrast to safety climate or culture approaches that often do not, but rather address safety more broadly.

Other types of approaches in this group are based on simple linear models, where a chain of multiple events culminate in an injury. Safety prevention is then directed at removing one or more of the elements involved in this chain of events, to prevent the occurrence of the injury. One such model is Heinrich's “Domino theory” (Heinrich, 1931), which has had a tremendous effect on practical safety interventions, and still is much in use despite numerous pitfalls that have been described (Johnson, 2011; Manuele, 2014). A further development of these models is the complex linear models, such as the BowTie model and the Swiss cheese model (Reason, 1997), that illustrate that even though there are several barriers between hazards and accidents there can be flaws (as holes in the slices of a cheese) in these barriers that co-incidentally can be aligned and then result in accidents.

Another type of structural modification is social control, which introduces coercive power or incentives for people or organizations to change behavior. This is related to compliance with rules and regulations on a non-voluntary basis, for example, by use of enforcement and legal sanctions, as well as compliance on a voluntary basis, for example, by use of marketing, economic incentives, reputations, benchmarking, and insurance-related benefits for low risk companies. Regulation may serve as a potentially powerful institutional force to promote the adoption of occupational health and safety policies and practices (Chambers et al., 2013). The basic idea is that such instruments provide an incentive for companies or people at work to adhere to certain (legal) standards, either due to the risk of penalties in case of noncompliance, or because a benefit can be achieved in exchange for an appropriate behavior (Rothschild, 2000).

Currently, reports of the effect of legislation as a structural approach to occupational safety are conflicting, which is not surprising given the complexities involved in such evaluations for the most part. While effect variation may reflect differences in the actual implementation of the legislative approaches to prevention, a number of other factors may also be of potential importance. These include the very nature—or strength—of the legislation, and thus the requirements it is intended to impose. For example, a call for training would be expected to have a different effect than a requirement for safer equipment that removed a dangerous exposure.

In early 2000s we also began seeing studies focused on assessing effectiveness of efforts to reduce fatalities and injuries in the high-risk construction industry, including teaching, coaching, feedback and climate interventions (Bena et al., 2009; Cheng & Chan, 2009; Derr et al., 2001; Kines et al., 2010; Lipscomb et al., 2003; Suruda et al., 2002), evaluation of regulatory acts (Derr et al., 2001; Farina et al., 2013; Lipscomb et al., 2003; Suruda et al., 2002), and multifaceted approaches (Lipscomb et al., 2008, 2010; Spangenberg et al., 2002). The first study using serial measures to assess legislative effectiveness in this industry was published in 2001 by Derr. We found no studies evaluating engineering controls in the construction industry.

Comparison conditions

Most of the controlled safety intervention studies (RCT and CBA) used treatment as usual (TAU) as the main type of comparison condition (n = 46). However, the content of TAU varied across studies, reflecting the contextual conditions and how work and prevention are managed in particular settings (Andrée Löfholm et al., 2013). In most cases there were already some activities going on at a workplace, which were directed at the prevention of a certain problem, for example, needlestick injuries or overexertion injuries. Therefore, comparison conditions were in most cases usual activities (TAU), in only few cases (n = 5) no other intervention (Hilyer et al., 1990; Jinnah et al., 2014; Johnson & Owoaje, 2012; Quintana, 1999), and alternative types of safety interventions (n = 5) (Adams, 2013; Forst, 2004; Parker et al., 2009; Rautiainen et al., 2004). For the latter it could be cases where safety glasses were provided and only the educational components were different and thus evaluated (Adams, 2013). In some cases, it was difficult to evaluate the precise comparison condition, for example, when a historical cohort group was used (Peate et al., 2007).

Types of outcomes

The main types of outcomes used for evaluation of effects of safety interventions were injuries (80%), and to a less degree “risk or behaviors” (18%). Four studies (2%) investigated the effect on fatal injuries, 92 studies (44%) investigated the effect on nonfatal injuries. The remaining reported safety interventions (55%) mixed fatal and nonfatal injuries in their evaluation of effects.

Selection sample bias

For the dimension selection sample bias three domains were assessed that each can introduce sample bias: sequence generation, allocation concealment and equivalent groups. Both randomly and non-randomly, selection can introduce bias in estimation of effects of safety interventions.

Performance bias

For most of the safety interventions (n = 114) we judged a high RoB that the participants were not blinded, including all safety interventions using RCT study design. Only in few cases we assessed low RoB that the participants knew about the safety intervention. Clearly, the social context of work makes it very difficult to blind participants. There are also ethical reasons for informing participants of the safety interventions they are involved in.

Detection bias

Detection bias is concerned with systematic differences between groups in relation to how outcomes are determined, including blinding of outcome assessors and the statistical analysis. Variation in reporting of accidents over time is a concern in evaluating the effects of safety intervention, as safety interventions could influence reporting propensity in the intervention group. We considered this a problem in several studies (Cheng & Chan, 2009; Mehrdad et al., 2013; Rasmussen et al., 2006). In Wang (2003), students reported their behavior and their number of injuries. This is likely to cause underreporting of accidents in both groups.

Blinding of outcome assessors was judged as a high RoB in the following studies (Forst, 2004; Harms-Ringdahl, 1987; Hilyer et al., 1990; Kines et al., 2010; Mattila & Hyoedynamaa, 1988; Mehrdad et al., 2013). In cases where the intervention, data collection and management of data were done by the same researchers, we considered detection bias as a high RoB.

Some studies were judged at high RoB as appropriate methods were not used to take into account the appropriate unit of analysis (Black, 2011; Cheng & Chan, 2009), or censoring time to event data (Evanoff et al., 1999).

Attrition bias

As safety intervention studies often require participants to change the way they usually work, this can result in their dropping out. Consequently, the characteristics of the intervention group can change, if there are systematic differences between dropouts and completers. Some studies provided useful information on employee turnover, dropouts and percentage of completers, but if this information is not available it can be difficult to judge the attrition bias, and there is a risk of underestimating the RoB.

If loss to follow-up was not taken into account in the analysis we considered this a high RoB, which was the case for two RCT studies (Kines et al., 2013; van der Molen et al., 2011) and unclear in one RCT study (Zohar, 2002). In four CBA studies we considered this a high RoB (Black, 2011; Evanoff et al., 1999; Forst, 2004; Mehrdad et al., 2013), and unclear for two studies (Hilyer et al., 1990; Ray et al., 1997). Three studies did not use intention to treat analysis, and we did not have data to correct this (Evanoff et al., 1999; Kines et al., 2013; van der Molen et al., 2011).

Reporting bias

Selective reporting of outcome data and results can be difficult to assess if there is no protocol published beforehand. Only in recent years, have investigators even considered publishing workplace study protocols in advance. We checked that pre-specified primary outcomes were reported. We also checked whether the pre-specified severity of accidents or the pre-specified types of accidents were considered. We identified four CBA studies (Haviland et al., 2012; Mattila & Hyoedynamaa, 1988; Parker et al., 2009; Wang, 2003) and one RCT study (Adams, 2013), where we judged high RoB caused by selective reporting of study outcomes. In one of these (Parker et al., 2009) not all results from the intervention companies were presented in the estimation of effects.

Fidelity of intervention

For assessment of the fidelity of intervention we judged whether the target population was given the safety intervention and/or the target population adopted the safety intervention (implementation of intervention). If one of these two criteria were not met we judged the fidelity of intervention as high RoB. We judged high RoB for 25% of the studies due to the intervention not being clearly provided and/or adopted by the target population. In 30% of the studies this information was unclear from the study report.

In some cases we were informed that the intervention groups received the intervention, but no information on whether it was adopted (Daltroy et al., 1997; Kim et al., 2004; Mehrdad et al., 2013; Peate et al., 2007; Zohar, 2002). In other cases we had no information on the degree of the actual implementation, for example, use of eyewear (Forst, 2004); whether firefighters maintained exercises throughout the intervention period (Hilyer et al., 1990); enforcement was not documented (López-Ruiz et al., 2013); whether older non-safe devices were removed in engineering approaches (Valls et al., 2007); or whether provided equipment were in fact being used (Black, 2011; Santaweesuk et al., 2014). In one case improvement of safety knowledge was measured, but the intervention was about change in behavior, and this was not documented (Wang, 2003). Following this we judged the fidelity of interventions as unclear RoB. In a study of the influence of weight loss on reduction of work injuries, there was no documentation on whether weight loss was maintained in the follow-up year (Morgan, 2012), which we considered a high RoB.

Intervention rationale

For the assessment of the intervention rationale we evaluated whether it has been clearly stated why the intervention should work, including a reasonably description of the theoretical or conceptual elements. We also accepted plain arguments for why an intervention should work, if it was reasonably clear. If one of these two criteria were not met we judged the intervention rationale as high RoB. We only judged high RoB for 8% of the studies where it was not clearly stated why the intervention should work. In 3% of the studies this information was unclear from the study report. This means that most studies have some reasonably description of why the intervention should work. However, in 30% of the safety interventions neither a theoretical nor a conceptual approach were used as a background for the intervention. In some cases the effect was taken for granted, as in the case of the effect of legislative efforts (Derr et al., 2001; Farina et al., 2013; Haviland et al., 2012; Levine et al., 2012).

Other potential sources of bias

In half of the safety interventions risk of other potential sources of bias was judged to be high. In one study we judged that the “Hawthorne effect” was a RoB, as strong involvement of researchers could have influenced outcomes (Cheng & Chan, 2009). We did not consider self-report a bias in itself, but in some cases we judged that it was likely that the intervention contributed to an increased reporting of cases of injuries in the intervention group (Daltroy et al., 1997), or that outcome data, such as self-reported injuries (Gadomski et al., 2006; Rasmussen et al., 2003; Rautiainen et al., 2004) were not a reliable measure, or that different seasons for data collection in the control and intervention groups were a threat (farming), even though statistical adjustments for time at risk were made (Rasmussen et al., 2003). Knowledge of intervention in control group (bleed over) can bias results toward null, in particular when the control and intervention groups are in the same location (Harms-Ringdahl, 1987; van der Molen, 2011). In some other cases, we judged that the intervention likely influenced self-reported outcome data (Mattila & Hyoedynamaa, 1988), including self-reported intermediate measures (Wang, 2003). In the latter case, students reported their behavior and their injuries, and we judged that this is likely to have caused underreporting of both. In one study, we judged that economic incentives for participation likely influenced the reporting of injuries (Rautiainen et al., 2004). In a study on effects of health education on the riding habits of commercial motorcyclists in Uyo, southern Nigeria (Johnson & Owoaje, 2012), we judged a high RoB, as self-reported riding behavior in face-to-face interviews is likely to cause underreporting of cases.

In Knapik et al. (2003) we considered it a high RoB that the control group had more than double the number of running miles compared to the intervention group, as an increase in running miles would increase the risk of traumatic overuse injuries, and this was not taken into consideration in the statistical model.

Another important aspect was potential for regression-to-the-mean, as intervention activities usually were initiated as a result of high injury rates. We judged high RoB due to regression to the mean in four studies (Black, 2011; Kim et al., 2004; Levine et al., 2012; López-Ruiz et al., 2013).

RCTs

The RoB assessments for the 20 included RCT safety interventions are shown in Figure 3.

OPEN IN VIEWER

Figure 3

Risk of bias graph in percentage for each item across all RCT studies, based on review authors' evaluation of each study. RCT, randomized controlled trial

The most common RoB identified was “other potential sources of risk of bias,” where nearly half of the studies were judged to have high RoB (Adams, 2013; Cheng & Chan, 2009; Daltroy et al., 1997; Morgan, 2012; Parker et al., 2009; Rasmussen et al., 2003; Rautiainen et al., 2004; Srikrajang et al., 2005; van der Molen et al., 2011), rather than those sources of bias that were specifically listed and assessed. Examples of other risks of bias are: that intervention has contributed to increase reporting of claims in the intervention groups; bleed-over between groups; the Hawthorne effect, external validity concerns, and selection bias caused by the intervention. RoB for the 20 RCT study designs is shown in Figure 4.

OPEN IN VIEWER

Figure 4

Risk of bias for each study using randomized controlled trials (RCT) study design

Controlled before and after (CBA) study designs

As safety interventions using CBA study designs do not use randomization, they will in general be assessed with higher RoB compared to RCT designs. However, in a few of the studies we judged that a non-randomized selection was not likely to be significantly compromised (Gregersen, 1996; Hilyer et al., 1990; Johnson & Owoaje, 2012; Levine et al., 2012; Peate et al., 2007; Santaweesuk et al., 2014). In 10 out of the 39 CBA safety interventions we judged that there was a high RoB due to nonequivalent groups, some due to differences in injury rates at baseline (Black, 2011; Carrivick et al., 2002; Harms Ringdahl, 1987; Lopez-Ruiz et al., 2013), or because the historical control group had unknown risk at baseline (Peate et al., 2007) (Figure 5).

OPEN IN VIEWER

Figure 5

Risk of bias (RoB) for Controlled Before and After (CBA) study designs: RoB graph in percentage for each item across all CBA studies, based on review authors' evaluation of each study

Risk of attrition was only judged to be high in four cases (Black, 2011; Evanoff et al., 1999; Forst, 2004; Mehrdad et al., 2013), where loss to follow-up were not accounted for in analyses, or where high turnover questioned the representation of the sample (Black, 2011). Half of the CBA studies suffered from other potential sources of bias, where regression to the mean was an important RoB for some studies (Black, 2011; Harms Ringdahl, 1987; Kim et al., 2004; Lopez-Ruiz et al., 2013), whereas others were likely to have serious risk of reporting failures (Mehrdad et al., 2013; Rasmussen et al., 2006). RoB for each CBA study design is shown in Figure 6.

OPEN IN VIEWER

Figure 6

Risk of bias for each study using Controlled Before-After (CBA) design

Serial measures (ITS) studies

The more common concerns regarding systematic error, or bias, in the studies with serial measures included maturation and possibilities that the intervention affected reporting of outcomes. Based solely on longer observation, potential maturation risks were not unexpected. However, some investigators effectively countered this issue through the use of internal and/or external comparisons or through the analytical methods chosen. Based on our assessments of RoB and any other validity concerns, an overall appraisal was made and studies of lower quality were noted.

Lower quality studies for which we considered the overall inferential process was problematic included (Birnbaum, 1993; Chhokar et al., 2005; Cooper et al., 1994; Fellner & Sulzer-Azaroff, 1984; Kuehl et al., 2013; Reddy & Emery, 2001; Saari & Näsänen, 1989; Sossai et al., 2010; Sulzer-Azaroff et al., 1990; Sulzer-Azaroff & de Santamaria, 1980; Zafar et al., 1997). While we had concerns about aspects of the Cunningham & Austin study (2007), it was noted that it was presented as a pilot study of an intervention focused on behavior of teams in operating rooms. It is of note that the studies considered to be of lower quality were more likely to have included older work that was published before 2000; 7 of 10 (70%) studies with serial measures published before 2000, but only 5 of 32 (16%) were published from 2000 forward. The lower quality studies were often small and used informal internal comparisons rather than direct statistical adjustment of outcome data.

Studies using serial measures were commonly used in the evaluation of legislative interventions. Blinding of participants and prevention of knowledge of the legislative interventions is not possible. Furthermore, based solely on length of the observation periods (four years and up to more than 30 years) all of the studies using serial measures were theoretically at high risk of being influenced by other changes. However, in large part, the investigators had tailored their approach/analyses to compensate for concerns. High risk of maturation effects was assessed for three studies (Derr et al., 2001; Lopez-Ruiz et al., 2014; Suruda et al., 2002). While both Derr et al. (2001) and Suruda et al. (2002) studies in the US construction industry observed declining fatalities, it is difficult to clearly assign attribution to the legislative effects. Construction injury and fatality rates were declining in general in the study periods, and the investigators made only informal comparison to other fatality patterns in the industry. Lopez-Ruiz et al. (2014) reported several potential factors that could have influenced traffic injury patterns that would not have been addressed through their use of an internal control with non-traffic work-related injuries.

Given the use of the secondary data for retrospective analyses in the ITS studies, concern regarding the intervention affecting outcome data is minimized. It is still possible that concerns about penalties, particularly if they increased after the legislation went into effect, could influence reporting by employers. Little information was available in the study reports about the enforcement of the legislative efforts. No concerns were identified regarding selective outcome reporting (which is hard to identify). No concerns were noted about specification of the point of intervention; investigators who did not use time periods immediately following the intervention explained their approach clearly.

Aside from these more common concerns regarding bias in ITS studies, several other concerns regarding bias or threats to validity were noted as well (see Figures 7 and 8).

OPEN IN VIEWER

Figure 7

Risk of bias (RoB) for Serial measures (ITS) study designs: RoB graph in percentage for each item across all serial measures, based on review authors' evaluation of each study

OPEN IN VIEWER

Figure 8

Risk of bias (RoB) for each SERIAL MEASURES (ITS) STUDIES

Safety campaigns (Type 1.1.1, Supporting Information Appendix 12.3)

The basic rationale in safety campaigns is that peoples' behaviors are changed through providing them with the right information or knowledge about the hazards in the workplace, and the consequences these can have on their safety at work. This is to be accomplished by use of persuasive messages in posters, pamphlets, videos, or spots on television and the like. We retrieved two studies evaluating safety campaigns, where both studies were using a CBA design. We judged that meta-analysis was not possible for these safety interventions, as the interventions were different in nature and also the outcome measures used (injuries and behavior).

The studies came from the agriculture (Forst, 2004), and transport and storage (Gregersen et al., 1996) sectors. Safety information in the included studies was provided through various means. In neither of the two studies the campaigns were restricted to media sources, but were more intensive. For example, the campaigns were directed at specific target groups such as drivers in transport and workers on farms and the workers were addressed directly face-to-face with safety information.

The agricultural campaign elements were very specific and related to actual work on the farms where the evaluation took place, and evaluated the effect of safety campaigns focused on eye injuries faced by Latino farm workers in southeastern Michigan and northern Illinois (Forst, 2004). Together with visits among Latino workers and provision of campaign materials in English and Spanish, a variety of tinted and clear safety glasses were also provided to the Latino farm workers. The study found an effect in the short term of safety campaigns to change the behavior and risk of the Latino workers with a SMD of −0.62, 95% CI: −1.13, −0.11. However the study had several limitations (see Figure 8, RoB of included RCT and CBA studies) (Comparison A1, chapter 11.1.1.).

In the study among drivers at the Swedish tele-operator company, “Televerket” (Gregersen et al., 1996), the drivers were targeted at meetings with very specific messages relating to actual work and seasonal problems and road conditions. For example, during the autumn, drivers were informed about driving in darkness, stopping distances and warning. A spring meeting included awareness of other road users, as well as company specific problems such as loading tools and equipment. The safety information was provided by specially trained employees from within the company, during working hours. A number of video films with road safety themes were shown and campaign material and pamphlets were handed out to the drivers. A nonsignificant increase in accidents was reported from this intervention at medium-term follow-up (OR: 1.39, 95% CI: 0.96 to −2.01) (Comparison A2, chapter 11.1.1.).

The evidence base was not strong enough to conclude on safety campaigns.

Counseling approaches (two-way approaches) (Type 1.1.2, Supporting Information Appendix 12.3)

We retrieved four safety interventions, three with RCT study designs (Adams, 2013; Gadomski et al., 2006; van der Molen et al., 2011) and one with CBA study design (Gregersen, 1996), that used counseling approaches. The included studies used group discussions (counseling), alone or in combinations, to increase knowledge and changes in attitudes and beliefs. Counseling and group discussions are a two-way communication process and considered to be an approach with more interaction and involvement than a one-way educational approach. We conducted a meta-analysis to evaluate the overall effect of the counseling approach (Comparison A3 and A4, in chapter 11.1.1.).

One RCT study in health wards in hospitals indicated no significant effect of group discussions among health care workers (HCW) (van der Molen et al., 2011). They used a 1-h interactive Power Point slide presentation beyond usual information given, where participants of the wards were informed, and where information among participants was exchanged about the causes, consequences and prevention of needlestick injuries on their ward. The study showed a non-significant effect on injuries at short-term follow-up (OR: 0.45, 95% CI: 0.15–1.31) of this short intervention effort.

Adams (2013) compared an enhanced interactive education package to a standard education already being provided. The enhanced education consisted of the initial standard education plus education in the form of short street-plays and messages; group motivational sessions; and individual counseling provided by health workers to those who were not regularly using protective eyewear. These sessions occurred over 1–2 h every week in the 1st month, twice in the second and 3rd month, and so on, and in total 11 educational sessions were conducted. Compared to the van der Molen et al. (2011) study above, the doses provided is much stronger. However, there was no significant effect of this safety intervention at short-term follow-up (OR: 0.80, 95% CI: 0.40–1.60).

Another study assessed whether active dissemination and in-depth communication and counseling on the North American Guidelines for Children's Agricultural Tasks (NAGCAT) reduced childhood agricultural injuries (Gadomski et al., 2006). The intervention also included in-depth communication between educators and intervention group participants. The core set of guidelines included a chart of recommended ages for tractor operation by size of tractor and task, and guidelines on driving a farm tractor with no implement attached, and several other specific guidelines. Farm visits were conducted and a parent resource booklet containing 52 guidelines was provided to the farm families. Several booster information interventions followed these farm visits. Possible co-interventions were also assessed and this was taken into account in the analysis. The length of follow-up was 2 years and 10 months (medium-term). The study showed a significant effect on injuries (OR: 0.52, 95% CI: 0.29–0.93).

In the Swedish “Televerket study” each driver participated in three meetings of small groups of 8–15 drivers, discussing problems of road safety and what to do about them. Each meeting lasted approximately 1 h. The method used an approach that has been used in Japan in a study of bus drivers (Misumi, 1978, 1982) which showed that accident involvement decreased sharply following group discussions. The intervention arm was compared with a matched control group.

The discussions were led by drivers from their own working unit who had attended a special introduction course. The company had agreed to follow-up on suggestions from the drivers about measures that should be implemented by the company. There was a significant effect of this intervention at medium-term follow-up (OR: 0.53, 95% CI: 0.36–0.77). We had some concern that reporting of accidents may have been influenced by the intervention. There may have been an additional component to the counseling as such, as there seemed to be strong management involvement, and further incentives to the work of the groups as measures would subsequently have been implemented by the company. However, this was not clear from the manuscript, and we decided to classify this intervention as counseling approaches and not multifaceted.

The counseling interventions are homogeneous, though with small effect sizes favoring the intervention. The high level of homogeneity in these interventions may be explained by quite similar interventions focusing on personal protective devices in primarily dynamic work settings. The meta-analyses showed limited evidence for no effect at short-term follow-up (OR: 0.67, 95% CI: 0.38–1.21; I ² = 0%). We judged limited evidence for an effect of intensive and interactive counseling approaches including group discussions at medium-term follow-up (Meta-analyses in Comparison A3, in chapter 11.1.1.).

Teaching and educational approaches (Type 1.1.3, Supporting Information Appendix 12.3)

Teaching and education, as means for modifying attitudes and beliefs, are common approaches to the prevention of accidents at work. Usually this is a one-way information approach. Three CBA studies (Johnson & Owoaje, 2012; Mehrdad et al., 2013; Wang, 2003) and two serial measures studies (Bena et al., 2009; Mujuru et al., 2009) were included.

Wang (2003) evaluated the effect of a 60-min lecture and a 20-min video on the prevention of needlestick injuries among student nurses in Changsha, China. A non-significant effect was seen of this intervention (OR: 0.29, 95% CI: 0.07–1.20), after we took the design effect into account. The study had several limitations such as attrition (22 out of 56 students lost to follow-up), intervention fidelity (intermediate outcome indicate no effect of intervention), and that behavioral intervention might have influenced self-reported injuries to the teacher. In another study by Mehrdad et al. (2013) in Iran the focus was also the effect of teaching and educational activities on needlestick injuries, showing a non-significant increase in injuries in the intervention group (OR: 1.08, 95% CI: 0.77–1.51). Serious RoB was also judged for this study (Figure 6), including lack of intervention fidelity and detection bias, and we also considered that the intervention was likely to have influenced reporting of injuries.

The third CBA study evaluating teaching and educational approaches (Johnson & Owoaje, 2012) assessed the effect of health education on the riding habits of commercial motorcyclists in southern Nigeria. The motorcyclists in the intervention group received safety education with health and safety lectures and interactive sessions, and the controls had no intervention (placebo treatment, received HIV health education). No significant effect of the intervention was observed (OR: 0.93, 95% CI: 0.46–1.85) (See Comparison A5 for the three CBA studies in chapter 11.1.1.).

These studies of teaching and education safety interventions are moderately homogeneous, though with no effect at short-term follow up. The diverse settings (Iran, China, and Nigeria), variations in the work context (health establishments and mail delivery by means of motorcycle), and one study with alternative control group, are likely to be sources of heterogeneity. We conducted a meta-analysis for the effect of these three teaching and education safety interventions at short-term follow up that showed no significant effect (OR: 0.90, 95% CI: 0.56–1.46; I ² = 37%) (Comparison A5 in chapter 11.1.1.).

In the safety interventions using serial measures, there were observed minimal (Bena et al., 2009) or no impact (Mujuru et al., 2009), in the absence of other intervention efforts, for safety teaching and educational approaches (Comparison B1, chapter 11.1.2.). One study in the construction trades aimed to decrease overall injury rates through education (Bena et al., 2009); only 29% of the workforce were able to be recruited to participate on this large Italian construction project. By use of a time-series regression model to assess the effect at medium-term follow-up, the study demonstrated a very modest and non-significant effect. Mujuru et al. (2009) evaluated the effect at long-term follow-up (8 year) of a single session that included a 14-min video focused on prevention of causes of fatalities in logging (US), specifically struck by injuries and caught in, under or between events. No significant effects were seen in these studies.

These latter two safety interventions using serial measures represent different intensities and doses of training. However, the analyses showed limited evidence for no effect of the safety interventions, and thus supported the meta-analysis of the controlled studies. We conclude that there is limited evidence for no effect of teaching and educational approaches, without any other simultaneous intervention efforts (Comparison B1 in chapter 11.1.2.).

Summary of the effect of attitude and beliefs modification

No or modest effects were found across the various means directed at attitude and belief modification as an approach to reducing injuries or improve safety and behavior. Thus, it is somewhat uncertain whether attitude and belief modification as a safety intervention approach, by itself, has any effect on work injuries. There seems to be limited evidence for no effect of the KAP model, as a one-way communication strategy in a workplace setting.

There could be seen some promise in the group discussions, as the nature of this approach may represent stronger interaction and involvement than one-way educational approaches or campaigns. Future work should distinguish between campaigns disseminated through media and other channels of influence, and attitude modifications that take place “face-to-face” in group discussions; the latter approach among Swedish telephone installers was the only study that showed a significant positive effect on the number of work-related road traffic accidents at medium term follow-up.

We conclude that the link between attitude modification and accidents seems to be uncertain, and that the level of evidence is limited for no effect of one-way communications in preventing accidents. However, attitude shaping as a one-way approach may be relevant in the context of other types of safety interventions, and will also be present as components in the multifaceted safety interventions analyzed below. Counseling approaches representing two-way communication showed limited evidence for a strong effect at medium-term follow-up, but no effect at short-term follow-up.

Safety training

A study to reduce cut injuries among young and inexperienced workers (Banco et al., 1997) used training in use of case-cutters. Each employee attended a 15-min training session specific for the type of cutter being used over a 4-week period. The content of the training sessions included instruction and practice of opening and closing of case cutters, changing blades, top, tray-pack, and mid-cut cutting techniques, blade disposal, cutter storage, and re-supply of case cutters and blades. The study demonstrated a modest (small) non-significant change in reducing injuries (OR: 0.91, 95% CI: 0.39–2.12) (Comparison A6, in chapter 11.1.1.).

Gregersen's (1996) study on Swedish telephone repair workers showed an effect at medium-term follow-up (2 years) of training on the reduction of accidents (OR: 0.60, 95% CI: 0.43–0.83). The training was comprised of three blocks: maneuvering training, skid training and “commentary driving.” Apart from the skid training, where a skid simulation car was used, the training was carried out in real traffic. Each of the blocks lasted 2 h and 30 min, and thus the whole training program lasted nearly 8 h. A special driver training team from the National Society for Road Safety provided the instructors for all the drivers (Comparison A7, in chapter 11.1.1.).

Individual goal setting and feedback/coaching

A study (Cheng & Chan, 2009) of the effect of individual job coaching and use of a job-specific occupational health education program (protection motivation theory), on prevention of work-related musculoskeletal back injury among construction workers, did not give any significant effect (OR: 0.37, 95% CI: 0.09–1.56). This study also had several shortcomings in the design and statistical analyses, and is thus of low quality (Comparison A8, in chapter 11.1.1.).

Jinnah et al. (2014) reported on interventions to improve tractor safety as part of a larger youth-on-farm safety intervention, where efforts focused on knowledge transfer, active learning, and skill building. A 16-item survey was used to measure farm safety behavior of fathers and youth (mainly to prevent tractor overturns), guided by the TPB(Ajzen, 2012). Youth 10–19 years of age were randomly assigned to either one of two intervention groups (parent-led or staff-led), or a control group; Training and information surrounding tractor safety including use of Roll-Over Protection Structure (ROPS) were provided to the farms, and either staff (intervention arm one) or fathers (intervention arm two) provided individual feedback to the youths.

We evaluated the effect of the interventions by measuring the SMD of youth's unsafe behaviors for the intervention farms compared to the control farms (Comparison A8, for outcome 8.2: risk and behavior, in chapter 11.1.1.). For the parent-lead intervention there was a significant effect (SMD: −0.60, 95% CI: −0.12, −1.08), and a non-significant effect for the staff lead intervention (SMD: −0.41, 95% CI: −0.85, 0.04).

A controlled trial of a training program, equal to the so-called “back schools,” among 4000 US postal workers, was evaluated to see whether it reduced injuries (Daltroy, 1999). This behavior based intervention included education through classroom sessions, safety training and individual safety coaching, and did not provide any significant effect (OR: 1.11, 95% CI: 0.90, 1.37) (Comparison A9, in chapter 11.1.1.).

Quintana (1999) evaluated the effect of task-delineated safety (TDS)—a BBS management scheme—on slip, trip, and fall (STF) accidents. The outcome was the number of hazards observed which may induce in a STF accidental injury. The TDS approach combined behavior based principles of clear task delineation, individual accountability, and continuous performance feedback. They reported a reduced risk of hazards, but it was not possible to estimate a point estimate on the basis of the data provided (Comparison A10, in chapter 11.1.1.).

Summary of the effect of behavioral modifications (Type 1.2.0., Supporting Information Appendix 12.3)

We judged that the included behavioral safety interventions were too diverse in types of interventions and settings to allow for meta-analytic estimates. The effects of the behavioral safety interventions point in different directions, where interventions both favor control and intervention.

Neither of the two RCT studies aiming to reduce low back injuries related to lifting and handling of goods in construction and in postal service work provided any significant effect of the behavioral approach on injuries. The four CBA studies pointed in different directions. The one study arm in Gregersen (1996) using training of drivers had an effect on road traffic accidents, whereas the Banco et al. (1997) study among young and inexperienced workers did not reduce injuries.

Apart from Gregersen's (1996) study of drivers, the statistical power is not strong in the safety interventions evaluating safety training. The level of evidence is limited for no effect at short-term follow-up and moderate effect at medium-term follow-up. We conclude that the included studies indicate limited evidence for an effect of safety training. The level of evidence is moderate for little or no effect of individual feedback or coaching at medium-term follow up, whereas there is limited evidence for no effect at long-term follow-up.

Summary of the effect of safety interventions aimed at physiological modifications (Type 1.3.0., Supporting Information Appendix 12.3)

Overall, we judged the level of evidence to be limited for safety interventions targeting strength and endurance of human physiology, and that these safety interventions tend to have no effect on reducing injuries at work. While one RCT study (Morgan, 2012) reported a positive effect of personal fitness in the form of weight loss, we judged the evidence of the relationship between weight loss and injuries to be unfounded. Further work in this area should explicitly define the mechanisms through which such interventions are intended to work, and when the effects may be expected to be seen, for which types of accidents.

The first of the three main types of safety interventions reported above, were directed at the individual level, whereas the following approaches, are directed at the group or organizational level.

Summary of the effect of climate, norms and culture (Type 2.1.0., Supporting Information Appendix 12.3)

There are a few high-quality studies in this group of safety interventions, which indicate an effect on behavior and accidents. However, taken as a whole, the effect pattern in the included studies is inconsistent, and in particular there is no clear demonstration of the relationship between the intermediary measures (climate scores, safety index scores, and behavior), and reduction in injuries in all of the studies. Of those studies we considered higher quality, only Zohar (2002) used occupational micro-accidents as outcome (minor accidents—see above). We conclude that there is limited evidence that leadership based safety interventions have from little to moderate effect.

There was limited evidence that safety interventions using classic goal setting and feedback methods directed at group level have no effect at short term follow-up.

Legislative changes and enforcement (Type 2.2.1, Supporting Information Appendix 12.3)

Structural interventions such as regulation and enforcement may serve as a potentially powerful institutional force to promote the adoption of occupational health and safety policies and practices. The basic idea is that changes in laws and regulations provide coercive power or incentives for people or organizations to change behavior, and stick to certain (legal) standards that can impose effects on large working populations, as they are compulsory.

Fifteen studies were identified that sought to evaluate the effects of legislative intervention on accident at work through regulation and/or enforcement activities. The majority were from the US; (Bulzacchelli et al., 2007; Casteel et al., 2009; Derr et al., 2001; Foley et al., 2012; Haviland et al., 2012; Levine et al., 2012; Lipscomb et al., 2003; Marlenga, 2006; Monforton & Windsor, 2010; Phillips, 2012; Suruda et al., 2002) two were from Spain (Benavides et al., 2009; Lopez-Ruiz et al., 2014); and there were single studies from Italy (Farina et al., 2013) and Canada (Hogg-Johnson et al., 2012).

All but four of the studies were classified as serial measures (ITS) (Benavides et al., 2009; Bulzacchelli et al., 2007; Casteel et al., 2009; Derr et al., 2001; Farina et al., 2013; Foley et al., 2012; Lipscomb et al., 2003; Lopez-Ruiz et al., 2014; Marlenga, 2006; Monforton & Windsor, 2010; Phillips, 2012; Suruda et al., 2002). One involved a CBA study design (Haviland et al., 2012), one was a simulated RCT (Levine et al., 2012), and one was an RCT (Hogg-Johnson et al., 2012). All of the studies that focused on legislative efforts, or rule making, used ITS study designs; while three of the five studies exploring effects of activities designed to increase compliance with existing regulations such as consultation, inspections or enforcement—included the more experimental designs (Haviland et al., 2012; Hogg-Johnson et al., 2012; Levine et al., 2012).

Investigators typically provided important contextual detail regarding the intervention that was used in guiding the analytical approach and/or in the interpretation of findings (Table 14, structural safety interventions).

In the following, the effects of legislation and the effects of enforcement/compliance efforts will be treated separately.

Effects of enforcement (Type 2.2.7, Supporting Information Appendix 12.3)

Seven studies, representing 12 interventions, assessed effects of inspections and enforcement of laws and regulations (Benavides et al., 2009; Farina et al., 2013; Foley et al., 2012; Haviland et al., 2012; Hogg-Johnson et al., 2012; Levine et al., 2012; Lopez-Ruiz et al., 2014). Four of the six studies assessing enforcement efforts demonstrated declining patterns of injury rates consistent with the enforcement efforts. Further information on the studies can be seen in Table 14 on the nature of included Studies Evaluating structural modifications, under the subheading 2.2.1 legislative change and 2.2.7 enforcement of laws and regulations. Funnel plots for study effects can be seen in chapter 11.1.1. (Comparisons: A15–A18).

Enforcement efforts targeted overall and serious injuries in construction in Italy (Farina et al., 2013); overall reportable injuries in manufacturing in the US state of Pennsylvania and Ontario, Canada (Haviland et al., 2012; Hogg-Johnson et al., 2012); and injuries in mixed industries in the states of Washington and Pennsylvania in the US (Foley et al., 2012; Levine et al., 2012). The observation period for these studies ranged from 21 months to 12 years (mean 7.8 years, median 10 years) with follow-up periods after the intervention from 21 months to 6 years (mean 4.2 years, median 4 years).

The Hogg-Johnson study (2012), the only RCT, reported no significant effects of enforcement (OR: 0.99, 95% CI: 0.90– 1.09) for non-fatal injuries compared to no intervention (Comparisons: A16, in chapter 11.1.1.).

The Levine study (2012), a simulated RCT (study design classified as CBA), compared inspections to no intervention for all types of injuries at short-time follow-up (OR: 0.86, 95% CI: 0.77–0.95) and long-term follow-up (but data not included because we could not rule out risk of repeated measures). The Levine study did not find any significant effect at medium term follow-up (but data not included, for same reason as above). For the Haviland et al. (2012) study we took the sum of year one and two for complaint inspections (based on workers' complaints about a hazard) with or without penalty, to compare with programmed inspections (based on industry and firm hazardousness), and to have the same time points for the follow-up (Comparisons: A16, in chapter 11.1.1.).

We calculated the standard error from the P values given in the article according to the methods described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins & Green, 2008). In addition, Haviland et al. (2012) study did not find significant effects of complaint inspections at medium-time follow-up, with citations or penalties imposed compared to no inspections/penalties (OR: 0.99, 95% CI: 0.89–1.05). However, the programmed inspections showed a modest effect at medium-time follow-up, with citations or penalties imposed compared to no inspections/penalties (OR: 0.95, 95% CI: 0.91–0.98). Businesses without inspections served as the control, and they excluded multi-site organizations to ensure linkage between facility, intervention, and injury, and thus strengthened the internal validity.

Foley et al. (2012) incorporated a mix of internal and external comparisons and stratified analyses that facilitated interpretation. Foley compared companies (accounts) that had enforcement activities and companies that had consultation activities, respectively, to companies without any consultation or enforcement. Foley found that inspections with or without citations were more effective than consultations, although with no or little effect. This was the case for both fixed-site industries (OR: 0.96, 95% CI: 0.92–0.99), such as manufacturing, and non-fixed site industries (OR: 0.97, 95% CI: 0.95–0.99), such as construction work. However, Foley did not find any significant effect of consultations, either on fixed-site industries (OR: 0.97, 95% CI: 0.93–1.01) or on non-fixed sites (OR: 0.92, 95% CI: 0.85–1.00) (Comparisons: A17, in chapter 11.1.1.).

Three ITS studies were included (with eight study arms in total) (Comparison: B6); Farina et al. (2013) report was based on external comparisons for which greater detail would have been helpful. Farina et al. (2013) investigated the formalized programs to implement minimum health and safety requirements in construction work, comprising inspection, training and information plans, and found a modest effect (RR: 0.81, 95% CI: 0.67–0.99).

Benavides et al. (2009) reported decreasing injury rates but the declines could not be attributed to the “Preferential Action Plans” (PAP) that focused on regional manufacturing and private industry. Lopez-Ruiz et al. (2014) reported mixed outcomes of the effectiveness of the “Penal Point System” and “Reformed Spanish penal code” initiatives on traffic-related occupational injuries. It was also difficult in this study to attribute observed patterns to the roadway safety interventions, and furthermore, the two interventions were treated as if they were independent, even though they overlapped in time. It may be that company internal policies and practices are more important in managing employee working conditions, thereby reducing road traffic accidents, as was seen in the study by Gregersen (1996) (Comparisons: A4, in chapter 11.1.1.).

Effects of economic incentives (Type 2.2.2, Supporting Information Appendix 12.3)

We retrieved one controlled before and after study (Gregersen, 1996), and one with serial measures (Rautiainen et al., 2005). Marketing and economic incentives can be seen as a voluntary exchange, for example, insurance related benefits for low risk companies (Rautiainen et al., 2005), or economic incentives for group level behavior (Gregersen, 1996). The basic idea is that such instruments provide an incentive for companies or groups of people at work to stick to certain (legal) standards, as a benefit can be achieved in exchange for appropriate (more safe) behavior.

Another approach used for behavior change is the use of bonus systems. In another study arm of Gregersen's work (1996) the effect of a group-based bonus system on the reduction of traffic-related accidents at work among drivers was assessed. The group-based reward system included the whole working unit that resulted in peer pressure in each group (and among groups). At the outset, the group was given a money level, based on the number of vehicles (SEK 200 per vehicle). An average group with 30 cars started with SEK 6000. For each accident caused by a driver in the group, the amount of money was reduced by SEK 100 or 200 depending on seriousness—for all drivers in the group. After 1 year, the remaining money was given to the drivers for a group activity such as a party, a pleasure trip or buying something together, such as physical training equipment. This group level bonus system showed a significant effect on reduction of work related road traffic accidents (OR: 0.72, 95% CI: 0.53–0.96). However, we cannot fully rule out underreporting here, as it is likely that the peer pressure of the group will work both on the driving behavior and on the reporting behavior (Comparison A19).

A single study from Finland (Rautiainen et al., 2005), which took advantage of a natural experiment to assess effects of providing economic incentives (type 2.2.2) to farmers for lower injury claim rates, reported decreased rates of workers' compensation injury claims, but not for injuries with 30 or more days of work disability. The pattern of decreased claims by severity levels was observed in each category with up to 29 lost days (0 days, 16.3% decline; 1–6 days, 14.1% decline; 7–13 days, 19.5% decline; and 14–29 days, 8.4% decline), and is consistent with some underreporting. However, the authors pointed out that while underreporting would be expected to be greatest in the zero lost days' category, this was not the case. This would seem to indicate that underreporting cannot explain the full estimated effect of the intervention; however, no effects were observed on more serious events (Comparison B7, chapter 11.1.2.).

Effects of engineering controls (Type 2.2.4, Supporting Information Appendix 12.3)

We included 17 studies evaluating engineering controls, three RCT study designs (Jensen et al., 1997; Prunet et al., 2008; van der Molen et al., 2011), four CBA designs (Banco et al., 1997; Bell, 2002; Grimmond et al., 2010; Harms-Ringdahl, 1987), and 10 studies using serial measures (Alamgir, 2008; Briggs et al., 2003; Lawrence et al., 1997; Prezant et al., 1999; Reddy & Emery, 2001; Rogues et al., 2004; Schoenfisch et al., 2013; Smollen, 2004; Sossai et al., 2010; van der Molen et al., 2011; Whitby et al., 2008). A common characteristic of safety interventions based on engineering control is the focus on elimination or substitution of the hazardous substances or materials at the source. Another strategy is to separate the persons from the hazardous conditions by introduction of machine safeguards, safer hand tools, walkways, or other changes in the physical environment, which reduce safety's dependence on human behavior. Even though the studies evaluated different approaches to engineering controls in different contexts, they all built on the same basic principle of directly influencing workers' safety by eliminating hazards or separating the workers from the hazards, thus reducing dependence on human behavior (Comparison A21–A24 and Comparison B8–B10).

Jensen et al. (1997) found that double gloving reduced the rate of perforation of glove barriers during abdominal surgery, and thereby the number of episodes in which transmission of disease from patient to surgeon would be possible. The outcome measure was perforations of glove barriers. This classical approach to engineering control, where the double gloving provided a barrier between disease and surgeon, resulted in significant reduction in the risk of exposure to bodily fluids and thus the risk of transmission of diseases from patients to surgeon (OR: 0.33, 95% CI: 0.21–0.51) (Comparison A21, chapter 11.1.1.).

van der Molen et al. (2011) evaluated the effect of the introduction of injection safety needles provided by a commercial supplier (another arm of the study evaluated the effect of group discussions). The existing injection needles were replaced by the new injection needles with the safety device. A non-significant 28% reduction in injuries was observed with this intervention (OR: 0.72, 95% CI: 0.29–1.83).

Grimmond et al. (2010) investigated whether enhanced container engineering increases disposal safety and reduces injuries in health care. Sharps safety devices were developed following the principle that enhanced container engineering increases disposal safety by reducing dependence on human behavior. This resulted in a relatively large reduction in container-associated sharps injuries at short-term follow-up (OR: 0.18, 95% CI: 0.11–0.29). The reduction in sharps injuries in general was also reduced, but at a more modest rate (OR: 0.85, 95% CI: 0.74–0.99).

Banco et al. (1997) evaluated the effect of a new and safer case cutter on injuries among young unexperienced workers, compared to the use of a traditional case cutter. The results showed a reduction of accidents at short-term follow-up, but it was not significant (OR: 0.51, 95% CI: 0.19–1.37) (Comparison A22, chapter 11.1.1.).

Evidence of effect, as well as lack of effect, was demonstrated for engineering controls in four studies using serial measures that were conducted in healthcare. Smollen et al. (2004) found evidence that the introduction of a safety butterfly device (Type 2.2.4) resulted in a large decrease in needlestick injury rates (RR: 0.20, 95% CI: 0.07–0.55) that was sustained over 1.5 years of follow-up (medium-term). Similarly, Whitby et al. (2008) found a 51% reduction of needlestick injuries following replacement of conventional needles with safety devices (RR: 0.49, 95% CI: 0.43–0.56), at medium-term follow-up. In both cases the older devices were removed after the staff was trained in use of the new safety device. Reddy and Emery (2001) evaluated the effect of medium-term availability of engineering controls on needlestick injuries among HCW. The engineering controls was a hospital-wide implementation of safety syringes and needleless-intravenous systems in an 800 bed hospital in Texas. However, HCW also had access to older devices, which may be the reason that the effect was more modest, compared to the results of Smollen (2004) and Whitby et al. (2008) (RR: 0.78, 95% CI: 0.65–0.93).

Analyses by Rogues et al. (2004) demonstrate evidence of an effect (OR: 0.62, 95% CI: 0.51–0.75) consistent with other reports of multifaceted safety interventions to prevent needlestick injuries using a campaign, training, and technological modifications (engineering controls) to reduce exposure. The intervention required action on the part of workers to trigger a sheath to cover needles' post-phlebotomy, but old devices were removed from the workplace, guaranteeing use of the newer devices. While injury rates for all needlestick injuries were declining during this period, and compliance with universal precautions could contribute to the reduction in injuries, the decrease of the incidence of needle-related injuries not related to phlebotomy was not significant. The analyses could have been improved through a more rigorous approach to analyses that formally used non-phlebotomy needlestick injuries as an internal control (Comparison B9, chapter 11.1.2.).

In the same vein, Prunet et al. (2008) evaluated the effect of respectively passive (automatically triggered) and active safety catheters compared to conventional (non-safety) catheters among health care personal. The two types of catheters should protect the HCW from contamination from the patients, by separating the HCW from the bodily fluids. The outcome was cases of blood splashes, but no information on number of staff were provided, and thus it was not possible to calculate effect sizes of the two interventions. However, the results indicated a better protection with safety catheters, with lower exposure to HCW, and the passive safety catheter provided better protection against splashes to the environment than the two other types. Automatically triggered passive safety triggers provided better protection in general (Comparison A22, for outcome 22.2: Risk and behavior, chapter 11.1.1.).

Bell (2002) evaluated whether West Virginia (WV) logging companies experienced a reduction in injuries after beginning to use feller-bunchers, which are tree-cutting machines, which replace some of the more hazardous work done with a chainsaw during harvesting operations in forestry work. This substitution of manual tree cutting methods using chainsaws with tree cutting machines had a strong protective effect (OR: 0.27, 95% CI: 0.14–0.52), when compared to the remaining logging industry not using feller-bunchers (usual practice), at long-term follow-up (Comparison A24, chapter 11.1.1.).

Another study also took advantage of a natural experiment where a production line in a paper mill was redesigned. The investigators evaluated the effect on injuries after the introduction of the re-designed machinery, using another department as control (Harms-Ringdahl, 1987). Before the re-designed machinery was introduced, a systems approach was used to analyze safety at the paper mill plant, including job safety analysis, energy analysis, and deviation analysis, as a basis for re-design of the production system. The energy model derives its idea from early engineering prevention, which indicates that the injury of a person is caused from an uncontrolled flow of energy. The deviation analysis built on the assumption that accidents often are preceded by deviations from the normal and planned functions of a system. If deviations are eliminated or controlled, then accidents can be reduced. We considered that the introduction of the redesigned production line was the key component of the safety intervention. Like the feller-buncher study above, a rather strong effect was observed (OR: 0.44, 95% CI: 0.26–0.74) at medium-term (Comparison A23, chapter 11.1.1.). However, the frequency of accidents was about six times higher in the intervention unit compared to the control unit, and there is thus a serious risk of regression to the mean. Even though there are several limitations in these study designs, such studies are important, as it is not easy to acquire access to these types of safety interventions where very expensive re-designed machinery is introduced. The intervention was not designed as a research study, and it is understandable—and not inappropriate—that high-risk groups may be selected for early intervention efforts.

In other types of engineering interventions (2.2.4), Alamgir et al. (2008) reported a gradual decline in musculoskeletal injuries in the 4 years following installation of ceiling lifts in three long-term care facilities (RR: 0.56, 95% CI: 0.47–0.67). In contrast, Schoenfisch et al. (2013) observed an immediate decline in musculoskeletal injuries associated with patient handling in a community hospital following placement of mobile lift equipment and introduction of a “minimal manual lift” policy (RR: 0.56, 95% CI: 0.46–0.87). In another study arm (medical center) no effect was seen (RR: 1.01, 95% CI: 0.79–1.29). Overall, there appeared to be little effect of the intervention. The changes observed were more consistent with suppression of reporting caused by an institutional policy change that called on payment for work-related injuries by each clinical unit; adoption of equipment use was demonstrated to be low.

All four of these studies involved engineering changes in health care settings, yet they have important distinctions. In two of the needlestick interventions (Smollen, 2004; Whitby et al., 2008), once the staff were trained, the older devices were removed essentially assuring use of the new safety devices. Sossai et al. (2010) reported on the long-term effect of safety catheter use on the annual incidence rate of needlestick injuries 2003–2007. The introduction of catheters remained in place throughout 2 years of the follow-up; a campaign was also launched, which might have increased the reporting of cases. It was not possible to estimate an effect size for this study.

In these cases, the use of the engineering solution did not require any active decision making on the part of staff, and no alternative was available, that is, reducing dependence on human behavior. The lift equipment can reduce hazardous exposures, but it is dependent on staff adoption to be effective.

Briggs et al. (2003) reported mixed findings regarding the effectiveness of Supermax prisons' effect on officers' safety. For the purposes of such a drastic move to isolation of prisoners, the authors considered the findings “did not provide the necessary onus of evidence” to support the use of Supermax prisons. This study involved retrospective analyses of data taking advantage of a natural experiment (Comparison B10, chapter 11.1.2.).

Prezant et al. (1999) demonstrated a decline in lower extremity burns among firefighters in NYC, following requirements for a new protective uniform across the entire department that was designed to prevent extremity burns, compared to the traditional uniform. The modified uniform significantly decreased incidence and severity of lower extremity burns (by 85%) in structural fires. Analyses controlled for number of structural fires and serious fires (OR: 0.36, 95% CI: 0.34–0.39). Head burns, which would not be affected by the altered uniform, decreased 40% as well; this may reflect differences in the fires, and full effects on the lower extremity burns may not be attributable solely to the uniform (Comparison B9, chapter 11.1.2.).

Effects of administrative controls (Type 2.2.5, Supporting Information Appendix 12.3)

Administrative controls comprise organizational policies and procedures, such as the introduction or modification of safety policies (e.g., new lifting policies) or Safety Management Systems, including modifications of monitoring, feedback and learning systems (e.g., procedures for incident or injury reporting, internal audit systems and inspections). Administrative controls exert some sort of coercive power on organizational members. However, they are dependent on being monitored, and that organizational procedures are in place to ensure a sustained effect of the controls.

One study examined the effect of administrative controls (2.2.5) and provided evidence of effectiveness. Crime levels and assault rates in smaller liquor stores (Casteel et al., 2004) decreased through business participation in Crime Prevention Through Environmental Design (CPTED) that included safer measures to handle cash, lighting, signs etc.; effectiveness was greater among stores with higher compliance with recommended components.

Effects of soft regulation (Type 2.2.3, Supporting Information Appendix 12.3)

Soft regulation is a group of safety interventions where firms or organizations accept voluntary measures, such as agreements among social partners, corporate social responsibility policies, benchmarking, or voluntary external controls (e.g., ISO/OHSAS or other safety standards or guidelines). We identified one study (RCT) evaluating the effect of soft regulation (one study arm of the Hogg-Johnson et al., 2012 study, the other study arm was enforcement).

Hogg-Johnson et al. (2012) evaluated consultation where interested firms could consult with an sector specific Health & Safety Association (HSA) consultant to review their current practices, conduct gap analyses, or develop OHS plans, compared to no intervention. HSAs were able to provide a range of products and services including training and OHS certification. There was no significant effect (OR: 1.05, 95% CI: 0.92–1.20) (Comparison A20, chapter 11.1.1.).

This one RCT study provided no evidence for the effect of soft regulation. We judge the level of evidence to be limited, as only one high-quality study was included in the analysis.

Effects of social marketing (Type 2.2.8, Supporting Information Appendix 12.3)

One study using serial measures (Chapman et al., 2011; Chapman et al., 2013) investigated the effect of social marketing by use of existing and trusted channels of information to increase awareness and encourage adoption. The aim of this approach was to increase the information flow on measures that were not widely used, in this case effort to improve use of three safer and more profitable production practices, including barn lights, silage bags, and calf feed mixing sites. We used the updated version of the study from 2013 with a 7-year follow-up to measure the long-term effect. The study used adoption of a practice as outcome (behavior), and found an effect of the intervention (SMD: −1.72, 95% CI: −2.22, −1.22) (Comparison B17, chapter 3.1.2.).

However, the study had some limitations. Comparisons between different states were used in the two reports. There was no difference in adoption of any of the three measures between the Wisconsin and the Maryland farmers after 4 years, while there was indeed a difference in adoption between the Wisconsin and NY farmers after 7years. However, we know nothing about the baseline in NY. Both the intervention group and the comparison group reached similar levels of adoption points toward a secular trend. Since the intervention was also partly implemented in the comparison group, and this group had a delayed response, the conclusion that the intervention was effective is not unreasonable. The shape of the intervention was not described, and there was an initial effect but no effect after 1–2 years. In addition, we considered some risk of reporting bias, as data for all years was not included in the final analysis.

Summary of the effect of structural safety interventions

Overall, from this review process we judge that there is some evidence demonstrating that regulation can contribute to the prevention of accidents at work at long-term follow-up, even though effect sizes were little to moderate. We also find evidence that in some cases regulation does not work. Specifically, if there is no scientific evidence for the legislative requirements. It is therefore not surprising that injury reductions were not observed (Marlenga, 2006; Monforton & Windsor, 2010). In contrast to legislative effects, the evidence that enforcement activities can work was more consistent, but with even lower effect sizes, compared to legislative effords.

The engineering approaches in general provided modest to high reductions in injuries. The effects seem to be particularly high in cases where the safety intervention was made independent of human activities (Alamgir, 2008; Jensen et al., 1997; Bell, 2002; Grimmond et al., 2010; Smollen, 2004; Whitby et al., 2008). In other cases, the human factor seemed to reduce effects to a degree (Reddy & Emery, 2001; Schoenfisch et al., 2013).

Relatively strong effects were observed for engineering controls aimed at reducing needlestick injuries in hospitals, in particular when the required human behavior component could be reduced. Engineering controls are classical approaches where equipment with high-risk, such as traditional syringes, are replaced with equipment with less risk, such as needleless intravenous injection systems. Such initiatives often have a major impact, as they will reach all employees concerned because they are independent of human behavior. These studies are not only relevant in the specific work context of health care, as it must be assumed that the substitution principle (eliminating the risk at the source) of injury prevention, applies more broadly to safety prevention at work. Based on this, we judged that engineering controls have medium to strong effects, and that the quality of the evidence is strong. Larger effects were seen with passive engineering controls, that is, safety works independently of “decision-to-use” by workers at the worksite.

The results overall show that effect sizes of the structural measures range from modest to strong effects, with higher effects when safety interventions to a lesser degree are independent of human activities. This is well in accordance with theoretical (conceptual) knowledge related to the Public Health Hierarchy of Hazards Control (PHHHC).

Effect of combinations of components at the individual level (type 3.1)

This group includes interventions directed at the individual level that combine attitude modification and behavioral modifications. We retrieved two RCT studies (Rasmussen et al., 2003; Rautiainen et al., 2004) and four CBA studies (Forst, 2004; Kim et al., 2004; Knapik et al., 2004; Peate et al., 2007). These interventions each combined a variety of intervention elements to achieve an effect, including both attitudinal and behavioral components. We found two safety interventions that integrated physiological changes with attitude modifications (Knapik et al., 2004, Peate et al., 2007). We found no studies in this group using a serial measures design to assess effects (Comparisons A25–A30, chapter 11.1.1.).

Two studies evaluated effects of a combination of approaches directed at the individual level on injuries in farming. Rautiainen et al. (2004) study investigated the Iowa Certified Farm Safety (CFS) program in a sample of 316 farms in the US and evaluated the effect at 3 years. The size of the farms was not clear from the paper. The CFS program comprised onsite safety reviews, health screening for farmers (farm operators), as well as education and monetary incentives for participation. No significant difference was found between intervention and control groups (OR: 0.99, 95% CI: 0.63–1.56) (Comparison A27, chapter 11.1.1.). The multifaceted intervention in Rasmussen et al. (2003) study included 201 small farms in Denmark with 6 month follow-up. The intervention program comprised some of the same elements as in Rautinen's (2004) study, excluding monetary incentives. Additionally, focus group discussions were conducted on accident occurrence and prevention, and individual feedback on problems, risks, and hazards was provided. Rasmussen found a difference between intervention and control that was not significant for the outcome “all injuries” (OR: 0.57, 95% CI: 0.29–1.12); also no significant effects were observed for more serious injuries requiring medical care (Comparison A25, chapter 11.1.1.).

Knapik et al. (2004) evaluated the effect of a multifaceted safety intervention that included modified physical training, injury education, and injury surveillance in two groups of soldiers. There was a significant difference in injury outcome between male soldiers attending United States Army Ordnance School Advanced Individual training and a historical control group (OR: 0.66, 95% CI: 0.47–0.93). For the female soldiers no significant difference was seen in injury outcome (OR: 0.73, 95% CI: 0.30–1.78) (Comparison A26, chapter 11.1.1.).

Peate et al. (2007) evaluated a multifaceted intervention including group seminars and a firefighter competency training program with individual training. The authors mention that the intervention reduced the number of injuries by 42% over a 12 month period as compared to a historical control group (Peate et al., 2007), but this could not be justified by the data, and thus a clear effect estimate was not possible to establish from this study. In addition, the group level analyses were not tied to any measure of fitness intervention that was applied at the individual level. For this reason we do not know if those with better fitness scores are the ones who had fewer injuries.

Kim et al. (2004) evaluated a back education program (the Back Informed Program) to reduce injuries to firefighters in Toronto, Canada. It offered employees job-specific education and ergonomic advice, exercises and hands-on sessions. The study was not strong and the data presented were unclear. The number of lost time injuries was two in the intervention group, and “about the same size” in the control (not reported) (Comparison A27, chapter 11.1.1.).

Forst (2004) evaluated the Community Health Worker (CHW) “promotor de salud” model as a tool for reducing eye injuries in Latino farm workers. The effort included safety training and safety promotion related to use of protective eyewear, as well as information on preventing eye injuries and illnesses in Latino farm workers. An effect of the program was found on the self-reported use of eyewear among the Latino workers (SMD: −0.58, 95% CI: −1.04, −0.13). However, the study had several shortcomings, and there was considerable loss to follow-up, also related to the difficulties of evaluating this among Latino farm workers (Comparison A27, chapter 11.1.1.).

Multifaceted safety interventions combining approaches at the group or organizational level (type 3.2)

Safety interventions directed at the group or organizational level may combine various approaches, such as modifications of climate, norms, engineering, or legal initiatives. We found eight studies exploring effects of approaches at the group or organizational level, two studies with a CBA design (Carrivick et al., 2002; Lopez-Ruiz et al., 2013) and four studies with a serial measures (ITS) design (Chhokar et al., 2005; Fujishiro et al., 2005; Gershon, 1990; Park et al., 2009) (Comparison A30 and Comparisons B13–B14).

Lopez-Ruiz et al. (2013) evaluated the effectiveness of an occupational injury prevention program, known as the PAP that focused on companies with high incidence rates of occupational injuries. The intervention combined official visits by a technician of the Safety and Health authority of the Valencia Region to companies with an action protocol; the technician writing a report describing the preventive situation of the companies, evaluating whether companies followed legal rules concerning preventive measures—mainly related to safety aspects of machinery, equipment, tools, devices, clean spaces, etc.; offering solutions and technical support and establishing deadlines to correct detected faults; carrying out a second official visit to check on the advances and changes produced; and finally, even proposing possible sanctions by the labor authority, after finding that recommendations had not been fulfilled. They reported a reduction in injuries at long-term follow-up (OR: 0.46, 95% CI: 0.33–0.64) (Comparison A29, chapter 11.1.1.).

Carrivick et al. (2002) evaluated the effectiveness of a consultative workplace risk assessment team in reducing the rate and severity of injury among cleaners in a 600-bed hospital. The intervention included an iterative injury risk identification, assessment, and control process, and feedback/recommendations to leaders. The study applied a participatory approach, where a workplace risk assessment team including employees supported the risk identification and control process, and resulted in a reduction in injuries (OR: 0.35, 95% CI: 0.23–0.55). This study had serious RoB, and thus low quality (Comparison A28, chapter 11.1.1.).

Two studies on the same initiative (but different populations) in the US State of Ohio focused on reducing musculoskeletal injuries in long-term health care establishments (Fujishiro et al., 2005; Park et al., 2009). The state policy initiative allowed institutions to seek consultation and training as well as funds for the purchase of lifting devices and assistance with initiation. The institutions could select what aspects of the overall program they wanted for their own needs. Fujishiro et al. (2005) reported that their results suggested an effect of consultation and support to purchase ergonomic interventions (RR: 0.77, 95% CI: 0.69–0.86). The Park study that followed several years later reported that the program expenditures were equivalent to savings from fewer workers' compensation claims (RR: 0.59, 95% CI: 0.41–0.85); they further concluded that injury rates did not decline with consultation activities alone, and findings regarding ergonomics alone were inconclusive (Comparison B13, chapter 3.1.2.).

Chhokar et al. (2005) reported that long-term follow-up of the introduction of overhead ceiling lifts in a long-term care facility was effective in reducing the risk of injury to workers, as claims continued to decline for 3 years post-intervention. This was a small study (involving only 50 injury events) in one institution, and the authors described the work as a longitudinal case series. Injury counts, rather than rates, were reported, and there were not stable numbers of events before the intervention, raising some concern about possible regression to the mean. We recalculated the number of claims by use of regression analysis based on three points before and three points after (the slopes b2-b1), and calculated the SE from the p-value. The effect was of borderline significance (OR: 0.96, 95% CI: 0.93–1.00) (Comparison B13, chapter 3.1.2.). However there was a gradual decline in lift transfer injuries that was not observed for repositioning injuries, providing some support for an intervention effect.

Gershon et al. (1999) reported needlestick injury reductions (OR: 0.29, 95% CI: 0.19–0.46) in one hospital associated with a multifaceted approach to prevention that included engineering modifications (safety devices, new disposal systems) and administrative controls in the workplace. Reductions were sustained over a 6 year follow-up (Comparison B14, chapter 3.1.2.). The last study in this group (Bell & Grushecky, 2006) found no evidence of effect of the West Virginia Loggers Safety Initiative (Comparison B14, chapter 3.1.2.). The initiative included 8 h of training provided to company representatives and site visits to assess safety practices with associated incentives (OR: 1.00, 95% CI: 0.44–2.27). This was the only study in the group (multifaceted at group/organizational level) where engineering controls were not included in the safety intervention. High risk of injury persisted in the industry even in companies participating in the initiative. In this respect, it is interesting to compare with the other logger study (Bell, 2002), where manual tree cutting methods were replaced by tree cutting machines (engineering control), which had a strong effect (OR: 0.27, 95% CI: 0.14–0.52).

Effect of multifaceted safety interventions across levels (type 3.3)

This group includes interventions directed at both the individual level and the group or organizational level, and may include various components across levels, such as attitude modifications or modification of behavior, combined with climate intervention, engineering controls, or changes in legislation. These interventions thus have in common that various approaches at the individual and group or organizational level are combined to achieve an effect. Integration of individual level and group or organizational level studies, was the largest group of multifaceted safety intervention evaluations we identified, with 20 safety interventions. Two interventions with RCT design (Peek et al., 2004; Srikrajang et al., 2005), five with CBA design (Black, 2011; Evanoff et al., 1999; Parker et al., 2009; Rasmussen et al., 2006; Valls et al., 2007), and 13 interventions using serial measures (Bell et al., 2008; Bull, 2007; Garg, 1999; Lipscomb et al., 2008; Lipscomb et al., 2010; Mancini, 2005; Miller et al., 2007; Mode, 2012; Passfield, 2003; Saari & Näsänen, 1989; Spangenberg et al., 2002; Wickizer et al., 2004; Zafar et al., 1997) (Comparison A30 and Comparisons B15–B16, chapter 13.1).

Peek et al. (2004) evaluated a multifaceted safety prevention approach, CPTED, which included environmental, administrative and behavioral approaches. The included intervention businesses made their own decisions about which recommendations to implement. We note that there is an increased level of crime reporting for both intervention and control, and no significant effect of the interventions (OR: 0.88, 0.67, 1.17) at short-term follow-up (Comparison A30, chapter 11.1.1.).

The outcome of the other RCT study (Srikrajang et al., 2005) was risk or behavior. This multifaceted intervention integrated the use of posters, educational activities, problem solving in work groups, and technological modifications to promote safe nursing practices. There was a significant effect at short-term follow-up (12 months) of this multifaceted approach to change nursing practices related to the prevention of needlestick and sharp injuries (SMD: −1.43, 95% CI: −1.67, −1.19). Parker et al. (2009) evaluated the effect of the Minnesota Machine Guarding Program that consisted of inspection of businesses for machinery problems, feedback on problems and remediation. There was no effect reported. It was concluded that simple and easy-to-use assessment tools allowed businesses to significantly improve their safety practices, and safety committees facilitated this process. However, the conclusions are based on before and after and risk factor analysis, and not the data used for the RCT study design. It is therefore reasonable to say that results of the randomized evaluation had no effect. We calculated the SMD of the intervention using the provided machine safety scores before and after the intervention, and by using the management only group as control, and found a non-significant effect (SMD: −0.26, 95% CI: −0.90, 0.39) (Comparison A30, outcome: Risk and behavior, chapter 11.1.1.).

Black et al. (2011) evaluated the effectiveness of a multifaceted Transfer, Lifting and repositioning (TLR) program to reduce musculoskeletal injuries among direct HCW. The study integrated safety training and the implementation of a standardized patient handling needs assessment and management system. They found about a 30% decrease in injuries at short-term follow-up (12 months) (OR: 0.69, 95% CI: 0.67–0.72) (Comparison A30, chapter 13.11.).

Passfield (2003) investigated a multifaceted safety intervention including safety training and safety policy (NO-LIFT Policy) and provision of lifting equipment in a public hospital. It was unclear whether equipment was used, (not reported), but probably the policy was established and the population participated in the training. The effect was significant (OR: 0.38, 95% CI: 0.16–0.93).

Rasmussen et al. (2006) evaluated a multifaceted intervention with safety training, safety coaching of workers and managers, and employee participation. There was a decrease in accidents from falls over time, but the accidents in the control group decreased further, and overall there was no significant effect of the intervention (depicted data from graphs, OR: 1.31, 95% CI: 0.62–2.77).

Valls et al. (2007) evaluated a multifaceted intervention including a slide show on risk factors and consequences of blood borne pathogens, posters and safety training on correct use of phlebotomy systems, and technological modifications in the form of implementation of new phlebotomy systems (engineering control). They observed a decrease in the rate of accidents after 6 month follow-up (short-term) (OR: 0.10, 95% CI: 0.02–0.49) (Comparison A30, chapter 13.11.).

Evanoff et al. (1999) evaluated a multifaceted intervention including safety training and feedback, and implementation of new lifting procedures including a participatory ergonomics program in health care among hospital orderlies. There was a significant effect of this intervention (OR: 0.50, 95% CI: 0.35–0.71) after 2 years follow-up (medium-term). However, employment turnover was high in the intervention group, and no information was provided as to whether the control and intervention group had similar characteristics at the outset (Comparison A30, chapter 13.11.).

Effectiveness of interventions that included more than one component across levels (3.3) was more likely for those with an engineering or administrative control. Spangenberg et al. (2002) studied a natural experiment in the construction industry, specifically a major bridge and tunnel project, which aimed to decrease overall injury rates on this project. The study found that the safety campaigns and economic incentives (award given twice a year to be shared by the employees at the safest site), including feedback to workers and workplace (site results and total results) did not result in significant behavior change, even though most had awareness of the effort; only 10% of the workers indicated that they had changed their working habit during this intervention. It showed a borderline significant drop in injury rates of 25% after control for periods of heavy and light construction work (OR: 0.75, 95% CI: 0.57–0.99).

Bell et al. (2008) demonstrated a 58% reduction (OR: 0.42, 95% CI: 0.33–0.54) of STF accidents in hospitals using a multifaceted intervention that included assessment of causes, hazard assessments, housekeeping changes, awareness campaigns, ice removal, flooring changes, slip resistant footwear for high-risk subgroups of workers, etc. This was a well-informed intervention; the approach was supported by the epidemiology of these injuries, which included a multi-causal problem. An ITS design with an external comparison (to US Bureau of Labor Statistics data reported by hospitals over a 10-year period) was used to measure effect.

Martin et al. (2009) demonstrated a non-significant reduction (OR: 0.79, 95% CI: 0.59–1.06) in patient handling injuries, through a program that included attitude modification, training, engineering controls (lift equipment), administrative controls, and employee participation in the intervention process. Back injury rates declined in the implementation period, but not as much afterward. The authors reported intermediate measures, and reported threats to sustained activity including fewer funds, storage issues for equipment, no designated resource people; they stated that waning levels of interest over time required constant or more sustained activity to maintain effects. There were no changes seen in patterns of injury in control injuries, that would not be affected by the program and lift equipment, which added strength to their findings (Comparison B15, chapter 11.1.2.).

Lipscomb et al. (2008/2010) demonstrated effectiveness of a safer trigger design on pneumatic nailguns and training in prevention of residential construction injuries at long-term follow-up through serial cross-sectional analyses based on a natural experiment. Over 4-years of observation, crude injury rates based on hours of tool use declined 43% as safer trigger use and prevalence of training increased. The test for trend approached statistical significance (p = 0.056); the authors discussed the loss of power over time as injury rates declined resulting in a less robust measure in the final year of follow-up. Data on exposure (type of trigger used and training) and injuries were collected on an individual basis allowing calculation of population attributable risk measures. In each year of follow-up, the engineering intervention was more effective from a population perspective. The training measurement was whether having had any prior training or not, and as such, this intervention could be considered largely one of engineering controlled for effects of training (Comparison B16, chapter 11.1.2.).

Mode (2012) and Porru (2012) reported on public health practice efforts to respectively decrease air crashes in Alaska, and to prevent injuries among foundry workers in Italy. Both safety interventions included structural components, such as competitions, benchmarking, legislative initiative with congressional funding (Mode, 2012), technical and organizational procedures (Porru, 2012). It appears from these reports that public health practice over a period of time can be effective in preventing injuries; the authors made no attempts to disaggregate elements involved, but rather evaluated the process of surveillance, risk factor, identification, intervention, and evaluation as an ongoing process over several years (Comparison B16, chapter 11.1.2.).

Bull (2007) and Mancini (2005) demonstrated effects of prevention of eye injuries at long-term in metal workers in Norway and Italy, respectively, in efforts that included use of appropriate PPE and administrative controls that required and/or fostered use. Bull reported on the experiences in one setting using presentation of simple rates over time. The intervention involved an enforced policy requiring use of eye protection; workers were given two warnings and then fired for lack of compliance. Injury rates immediately declined markedly (6.09 to 0.42 injuries per million working hours) after eye protection became mandatory, and the decline was sustained at 10 year follow-up. The Italian intervention (Mancini et al., 2005) focused on multiple small facilities in one region of Italy. The intervention was based on considerable input from a needs assessment and follow-up of reported injuries. Interviews were conducted with workers about barriers to eye protection, unions were consulted about approaches and worksites were visited. A tailored educational brochure was developed, as were messages for TV and radio, and physicians provided workers with individual consultations. A 42% sustained reduction in eye injuries was reported in analyses of this ITS design with both internal and external controls (Comparison B16, chapter 11.1.2.).

Miller et al. (2007) at long-term follow-up (Comparison B16, chapter 11.1.2.), and Wickizer et al. (2004) at medium-term (Comparison B15, chapter 11.1.2.), both found evidence of effect of peer-based substance abuse prevention and federally mandated drug and alcohol testing in a transportation company, as well as a multifaceted Drug Free Workplace program in various industries, respectively. Results varied by occupational group and for Wickizer et al. (2004), regression to the mean was a bias, as large differences in injury rates at baseline.

Serial measures (ITS) studies

The following intervention did not fall into one of the designated intervention classifications demonstrated some evidence of effectiveness in prevention injuries.

Menendez et al. (2012) found some indication in the Fatality Assessment and Control Evaluation (FACE) program in the US—and associated follow-up activities—may decrease fatal injuries in targeted areas (falls from height and electrocution); these multiple state run programs involved considerable variation in program development and follow-up. Evidence was not conclusive, and not over-stated, with the investigators simply reporting “some indications” that the programs may be effective.

Effectiveness of attitude and belief modifications (Main type 1.1.0) by use of three main types of safety interventions

We included 12 safety interventions (three using RCT, six using CBA, and three safety interventions using serial measures) as shown in Table 3 (meta-analyses) and Table 4 (narrative analyses).

Safety campaigns (type 1.1.1): We assessed two studies evaluating safety campaigns, where both studies were using a CBA design; one high-quality study (Gregersen, 1996) evaluated the effect on injuries, and the other low quality study measured the effect on safety behavior through the use of preventive tools and eyewear (Forst, 2004).

We conclude that the evidence base was not strong enough to conclude on the effect of safety campaigns (Table 3, meta-analyses, and Table 4, narrative analyses).

Counseling approaches (type 1.1.2): We assessed four safety interventions, (two using RCT design and two using CBA design) that evaluated effects of counseling approaches, one of high quality and three with moderate quality. The key methodological limitations in the studies were that allocation concealment was not possible, and that the CBA study was not randomized. In addition, we judged that self-reported outcomes in two of the studies as a RoB. A meta-analysis was carried out (Table 3).

We conclude that the level of evidence is limited that counseling approaches has no effect at short-term follow-up and strong effect at medium-term follow-up.

Teaching and educational approaches (Type 1.1.3): We assessed three CBA studies and two studies using serial measures which evaluated teaching and education as means for modifying belief and attitudes, and in turn accidents.

Two CBA studies evaluating the effect of teaching and educational approaches on needlestick injuries had several methodological limitations, such as incomplete outcome data and unclear implementation fidelity (Mehrdad et al., 2013; Wang et al., 2003). Another study evaluating teaching and educational approaches (Johnson & Owoaje, 2012) used self-reported behavior in face-to-face interviews, that we considered a RoB. We conducted a meta-analysis for these studies (Table 3). Two studies using serial measures found no or modest effect (Table 4).

We conclude that the level of evidence is limited for no effect of teaching and education at short-term follow-up and at medium-term follow-up.

Thus there appears to be limited support for the KAP model in a workplace setting (Lund & Aarø, 2004), which assumes knowledge leads to changes in attitudes or beliefs, which in turn leads to changes in safety practices and injury. Limitations to this model in an occupational safety context have been described previously (McDonald et al., 2009).

There could be a potential for effects of group discussions or counseling approaches, which involve stronger interaction and involvement than one-way educational approaches or safety campaigns through media, pamphlets, or other ways. Attitude shaping may be relevant in the context of other types of safety interventions, for example, multifaceted safety interventions (see below).

Overall, we did not find clear evidence that safety interventions based on attitude and belief modification as one-way communications are effective in changing behaviors or injury risk. Meta-analysis (Table 3) showed limited evidence of a strong effect of more interactive counseling approaches to reduce accidents at work at medium-term follow-up.

Effectiveness of behavior based approaches (Main type 1.2.0)

We included six studies, representing seven behavior based interventions (four interventions using RCT design: Cheng & Chan, 2009; Daltroy et al., 1997; Jinnah et al., 2014 with two study arms; three safety interventions using CBA design: Banco et al., 1997; Gregersen, 1996; Quintana, 1999).

Safety training (Type 1.2.2): We evaluated two high quality CBA studies evaluating safety training. This category includes skill based training, for example, proficiency or dexterity of doing a manual work task, and not just attempts to change attitudes. Gregersen's (1996) study on Swedish telephone repair workers showed an effect at medium-term follow-up (2 years) of training on the reduction of accidents. A study to reduce cut injuries among young and inexperienced workers (Banco et al., 1997) used training in use of case-cutters. The study demonstrated a modest change in reducing injuries, but was not significant.

We conclude that the level of evidence is limited for no effect of safety training at short-term follow-up, and moderate effect at medium-term follow-up (Table 4, narrative analyses).

Individual feedback or coaching (Type 1.2.4): Two studies judged to be of higher quality (Daltroy et al., 1997; Jinnah et al., 2014) pointed in different directions, where one study favored control and the other the intervention. Jinnah et al. (2014: study arm parent) found a significant effect of the behavioral approach on injuries, but did not use intention to treat analysis, and the outcome is measured as changes in behavior (seat belt use in ROPS tractors). The other RCT study of higher quality favored the control (Daltroy et al., 1997), even though it was not at a statistically significant estimate.

The RCT study of individual job coaching (Cheng & Chan, 2009) had some limitations, and we considered it low quality. The low quality was mainly due to the Hawthorne effect and that possible underreporting of injuries in the control group was likely to have biased the results, as management was keen on limiting work related muscular injuries. In addition, there was serious RoB for several other RoB dimensions.

We conclude that the level of evidence is moderate for no effect of individual feedback or coaching at short-term follow-up and limited evidence for no effect at medium-term follow-up.

All in all, the results indicate that behavioral approaches have no or little effect on prevention of injuries at work. Only safety training at medium term follow-up provided limited evidence for a moderate effect (Table 4, narrative analyses).

Effectiveness of physiological modifications (Main type 1.3.0)

Safety interventions based on modification of physiology to increase individuals' resistance to accidental risks are a relatively new category of safety intervention. We included four studies (one RCT, three CBA and one serial measures study) evaluating five safety interventions. The studies used heterogeneous approaches, mainly mixed physical training methods, and one intervention was about weight loss. None of these safety interventions demonstrated significant effects on safety behavior or accidents, apart from one study (Morgan, 2012) that reported a positive effect of personal fitness, in the form of weight loss (type 1.3.9, other types). However, we judged the evidence of the relationship between weight loss and injuries to be unfounded.

Data allowed for a meta-analysis for individual mixed physical training at short-term follow up, and showed limited evidence for no effect on injuries of safety interventions using physiological modification. Analysis of medium-term follow-up gave insufficient evidence (Table 3, meta-analyses, and Table 4, narrative analyses).

We conclude that there was limited evidence for no effect of mixed physical training on reducing injuries at work at short-term follow-up, and insufficient evidence at medium-term follow-up.

Summary

Overall, individual level approaches appear to have little or no impact on reducing accidents or improve safety and behavior at the workplace, and the evidence range from insufficient to limited evidence for the interventions, apart from individual feedback or coaching providing moderate evidence for no effect.

Effectiveness of climate, norms or culture modifications (main type 2.1.0)

We included no studies that investigated safety culture as a means to improve safety and reduce accidents. We included 11 studies under this main heading (two RCT, three CBA, and six studies with serial measures) assessing effectiveness of safety interventions to improve climate and safety norms, that is, improving the (shared) priority of safety in an organization/group, compared to other competing goals, as a means to improve safety and reduce accidents.

Seven studies used goal setting and feedback at group or organizational level (Cooper et al., 1994; Cunningham & Austin, 2007; Fellner & Sulzer-Azaroff, 1984; Mattila & Hyoedynamaa, 1988; Ray et al., 1997; Sulzer-Azaroff & de Santamaria, 1980; Sulzer-Azaroff et al., 1990). These studies had several methodological limitations, such as no blinding of outcome assessors, regression to the mean, serious risk of spillover, and attrition.

We conclude that there was limited evidence for no effect of goal setting and feedback at group or organizational level at short-term follow-up (Table 5, narrative analyses).

Four studies using leadership based approaches to improve safety climate (Kines et al., 2010, 2013; Zohar, 2002; Zohar & Luria, 2003). Of those studies we considered higher quality, only one (Zohar, 2002) used occupational injuries as the outcome. This study showed a significant effect on injuries.

However, the quality of the evidence overall is limited that leadership based safety interventions can improve safety and reduce accidents. The effect pattern in the included studies was inconsistent and, in particular, there was no clear relationship between intermediary measures (climate scores, safety index scores and behavior) and reduction in injuries. There is a need for a better empirical basis in support of the well-developed theoretical and conceptual approach used within safety climate research (Zohar, 1980; Zohar & Luria, 2003).

We conclude that there is limited evidence for a little or moderate effect of leader based safety interventions on safe behavior and decrease in injuries at short term follow-up (Table 5, narrative analyses).

Effectiveness of legislation (Type 2.2.1)

We included eight studies evaluating nine long-term effects of legislation, which were all serial measure studies. These legislative efforts, or rule making, all used serial measures and all were long-term evaluations. Methodological quality was limited by informal comparisons to other data, by a short “pre-intervention” period, and inconsistency in effects measured by various types of outcome. However, several investigators justified their approaches well, used techniques to improve the internal validity of their work including internal and external comparisons, and stratified results documenting changes in groups more consistent with an intervention effect. Of note, we found evidence from two studies that legislation or rule-making that lacks an evidence-base for the intervention did not appear to affect injury rates (Marlenga, 2006; Monforton & Windsor, 2010). Attribution of declining fatalities to legislation was not consistently demonstrated in some of the studies (Bulzacchelli et al., 2007; Derr et al., 2001; Suruda et al., 2002), as they all demonstrated declining fatalities, but the findings did not clearly allow attribution to the standard in question. This weakens the overall evidence base for legislative safety interventions.

We conclude that the level of evidence is limited that legislative efforts have little to moderate effects on prevention of injuries at work at long-term follow-up (Table 5, narrative analyses).

Even though effect sizes were modest, population-based impact can be considerable, given the number of work sites and workers that potentially can be affected through legislative interventions.

Effectiveness of enforcement and compliance efforts (Type 2.2.7)

The policy initiatives implementing the enforcement of legislation were quite diverse. Still, the three studies looking at effects of enforcement in three different US states, respectively, demonstrated some consistent findings (Foley et al., 2012; Haviland et al., 2012; Levine et al., 2012). Of particular note were the findings of larger effects when penalties or citations were imposed (Foley et al., 2012; Haviland et al., 2012), rather than consultation visits or inspections without penalty (Haviland et al., 2012; Hogg-Johnson et al., 2012; Levine et al., 2012). However, information on the level of penalties, and whether penalties were consistently imposed if violations were found, were lacking in the included studies.

Three of the six studies (16 study arms) exploring effects of activities designed to increase compliance with existing regulations such as consultation, inspections or enforcement included the more experimental designs. In contrast to legislative effects, the evidence that enforcement activities can work was more consistent, however, with lower effect sizes, overall. While legislation theoretically could have effects without enforcement, it seems reasonable that longer-term effects would be enhanced by enforcement. The only RCT study (Hogg-Johnson et al., 2012) did not show any effect. Two other high-quality studies did show effect (Benavides et al., 2009; Levine et al., 2012), but no significant effect at medium-term follow-up. Some studies had problems with selective reporting (Haviland et al., 2012), as subgroup analysis was not specified (size of enterprises), and there was overlap of intervention periods in two study arms (Lopez-Ruiz et al., 2014).

Meta-analysis of the RCT and CBA studies showed that there was limited evidence for a little effect of enforcement and compliance at short-term follow-up, moderate evidence of from none to a little effect at medium-term follow-up, and limited evidence of a little effect at long-term follow-up. For the studies using serial measures to evaluate enforcement and compliance we assessed that there was moderate evidence for no or little effects of enforcements and compliance at long-term follow-up. Studies with control showed lower effect sizes, and RCT studies the lowest effect size (Hogg-Johnson et al., 2012).

We conclude limited evidence for a little effect of enforcement and compliance efforts at short-term follow-up and strong evidence for a little effect at long-term follow-up and moderate evidence of no or little effect at medium-term follow-up (Table 6).

A cautious interpretation indicate a differential effect of enforcement over time, and that this effect is lower at medium-term follow-up. We cannot explain the mechanism of this pattern of differential effect over time on the basis of these studies.

Effectiveness of economic incentives (Type 2.2.2)

Two studies reporting on economic incentives, one study using serial measures to evaluate economic incentives in agriculture (Rautiainen et al., 2005), and one study using CBA design evaluating economic incentives in the transport industry (Gregersen, 1996). Evidence of effectiveness is mixed with a weak link between economic incentives and work injuries.

The use of insurance premiums seems to influence the propensity of reporting accidents at work, rather than clearly improving safety on farms (Rautiainen et al., 2005). The study reported decreased rates of workers' compensation injury claims, but not for injuries with 30 or more days of work disability. This pattern was consistent with some underreporting.

The CBA study by Gregersen (1996) investigated economic incentives for workers in a transport company and found an effect of this.

We conclude that the level of evidence is limited that economic incentives have from little to moderate effect on the reduction of accidents at medium-term follow-up (Table 5).

We could not estimate the effect of economic incentives at long-term follow-up (Rautiainen et al., 2005).

Effectiveness of engineering controls (Type 2.2.4)

We included 17 studies evaluating 19 safety interventions using engineering controls, four evaluations of engineering controls using RCT study designs, four using CBA designs and eleven using serial measures (ITS) designs. The engineering approaches in general provided modest to strong effects on improving safety and reducing accidents (Table 5, narrative analyses, and 5.4 meta-analyses). The effects were particularly high in cases where the safety intervention was made independent of human behavior (Alamgir, 2008; Bell, 2002; Grimmond et al., 2010; Jensen et al., 1997; Smollen, 2004; Whitby et al., 2008). In other cases the human factor was reduced to a lesser degree, requiring decisions or allowing options on the part of workers potentially exposed to hazards (Reddy & Emery, 2001; Schoenfisch et al., 2013), which was reflected in lower effect sizes.

The reduction of needlestick injuries in hospitals provides a particularly strong example of the effectiveness of engineering changes that design out risk and thus reduce the opportunity for human decision making to influence effects. This type of engineering control works by eliminating the risk, and refers to a higher level of the hierarchy of controls.

Meta-analyses were performed on evaluations of engineering controls using RCT and CBA designs, and showed limited evidence for a strong to very strong effect of engineering controls at post-test, strong evidence for a strong effect at short-term follow-up, and insufficient evidence for medium and long-term follow-up, respectively (Table 6). On the basis of the serial measures we judged that there was insufficient evidence at short-term follow-up; strong evidence for a moderate effect at medium-term follow-up; and limited evidence for a little effect at long-term follow-up (Table 5).

We conclude that there is limited evidence for a strong to very strong effect of engineering controls at post-test, strong evidence for a moderate effect at short-term follow-up, strong evidence for a moderate effect at medium-term follow-up, and limited evidence for a little effect at long-term follow-up (Tables 5 and 6).

Overall engineering controls provide moderate to strong effects on reducing accidents, however, with varying levels of evidence depending on the follow-up. Strong effects were in particular seen in cases where the safety intervention works independently of individual human decision making or work practices, or where the risks were “designed out.”

Administrative controls (Type 2.2.5)

One study examined crime levels and assault rates in smaller liquor stores (Casteel et al., 2004) participating in a CPTED that included safer measures to handle cash, lighting, signs, etc. Injury patterns were compared to non-participating businesses, controlling for crime trends in neighborhoods. Robbery and shoplifting were lower among participating stores, and even greater among stores with higher compliance with recommended components.

Another study evaluated whether the adoption of Universal Precautions (UP) or Body Substance Isolation (BSI) resulted in decreased needle recapping (risk factor) or injury rates among critical care nurses. Findings suggest that UP and BSI did not have significant impact on healthcare workers' greatest source of exposure to blood borne pathogens. We were not able to calculate effect measures for this study.

We found insufficient evidence for the effect of administrative controls at short and medium-term follow-up (Table 5).

Summary of structural measures

The results overall show that effect sizes of the structural measures range from modest to strong effects, apart from legislation and enforcement that had somewhat lower effect sizes overall (Table 5, narrative analyses, and Table 6, meta-analyses). This contrasts individual level safety interventions that mainly provided no to little effects overall. Structural safety interventions yielded higher effects when safety interventions were independent of human decision making. This is well in accordance with theoretical knowledge related to the PHHHC.

Effectiveness of multifaceted safety interventions at the individual level (Type 3.1)

This type of safety intervention combines components directed at the individual level, such as, attitude modification and behavioral modifications. We identified eight studies using combinations of components at the individual level (Table 7).

Safety interventions combining attitudinal and behavioral components seem to point in the direction of a small non-significant effect. It seems that this type of safety intervention follows the results we found for safety interventions directed at the individual level. There is moderate evidence that combining various approaches at the individual level has little effect on improving safety or reducing accidents at work.

We included one RCT study and six CBA studies, representing eight safety interventions. Two safety interventions used RCT study design, but did not provide significant results on safety interventions combining both attitudinal and behavioral components. The CBA studies indicated some effect, but only one study demonstrated significant results.

We conclude that multifaceted interventions at the individual level have moderate evidence of no effect at short-term follow-up, and limited evidence of little to moderate effect at medium-term follow-up on reduction of accidents and improvement of safety (Table 7).

Effectiveness of multifaceted safety interventions at the group/organizational level (Type 3.2)

We identified nine studies (two CBA and seven studies using serial measures) that evaluated multifaceted safety interventions directed at the group or organizational level. The two CBA studies combined interventions of consultation, advice and feedback, and sanctions in a group of mixed industries and in a hospital. Both of these studies had serious RoB.

The multifaceted interventions using serial measures combined various combinations of intervention components. One study combined skills training, workers' compensation premium rate reductions and administrative procedures at company level to encourage companies to maintain safety in the logging industry (Bell & Grushecky, 2006). Four studies combined engineering and administrative controls and sometime including training in use of new devices (Chhokar et al., 2005; Fujishiro et al., 2005; Gershon, 1999; Passfield 2003). The last study combined safety training, administrative controls, employee participation and feedback in an industrial setting (Park et al., 2009).

Effect sizes in the individual studies range from little to strong effect of the intervention, and thus the heterogeneity in outcomes is substantial, indicating that differences between studies are not due to pure sampling error. These differences can be related to the variation in combinations of components in a safety intervention when multifaceted approaches are evaluated, but also differences in study populations, as nurses, cleaners and industry workers are represented in various degrees in the interventions. Multifaceted components using engineering controls provide higher effects.

We conclude that safety interventions combining group or organizational level components provide moderate evidence of a strong effect at medium-term follow-up, and limited evidence of moderate effect at long-term follow-up (Table 7).

Effectiveness of multifaceted safety interventions across levels (type 3.3)

Multi-level safety interventions integrating individual level and group or organizational level components are the largest group of multifaceted safety interventions, including two RCT studies, five CBA studies, whereof two studies used risk and behavior as outcomes. Furthermore 13 studies using serial measures were included in this group of safety interventions, all using injury as outcomes.

Two CBA studies evaluating training and administrative controls or engineering controls provided significant and non-significant effects on injuries in health care and grocery store workers, respectively. The three other CBA studies found significant effects of combining individual and organizational level components, mainly safety training, worker involvement, engineering controls and administrative controls. The two RCT studies (Peek et al., 2004); Srikrajang et al., 2005) provided borderline and significant results, respectively, of the safety interventions. In the studies using serial measures, the effectiveness of interventions that included more than one component across levels was more likely for those with an engineering or administrative control. The heterogeneity was high for safety interventions at medium-term follow-up, but not for interventions at long-term follow-up. Overall, the multifaceted safety interventions using serial measures were favored over the control, apart from one study.

Meta-analyses were performed on evaluations of multifaceted safety interventions across levels using RCT and CBA study designs with injury outcomes. We found moderate evidence of none to moderate effect of the multifaceted safety intervention, as only the CBA studies provided significant effect (Table 8). The heterogeneity for the CBA studies was moderate (I ² = 74% for short-term follow-up studies). The single RCT study did not provide a significant effect (Table 8). For the serial measures we judged a strong evidence of a moderate effect of multifaceted safety interventions across levels at medium term follow-up, and moderate evidence for a strong effect at long-term follow-up. One serial measure provided a lower estimate, and the components of the interventions differed somehow, as it mainly consisted of a multifaceted Drug Free Workplace program that varied from the other components combined in this group of multifaceted safety intervention studies. If we excluded this latter study in the evaluation, then the heterogeneity became lower and the effect sizes higher and statistically significant.

We conclude that there is moderate evidence that safety interventions combining individual level and group or organizational level components have none to moderate effects at short-term follow-up (Table 8); strong evidence for a moderate effect at medium-term follow-up; moderate evidence for a strong effect at long-term follow-up (Table 7).

Summary

Based on the studies reviewed, the greatest effects were achieved with safety interventions directed toward the group or organizational level, whereas safety interventions directed at the individual level provided smaller effect sizes overall. For the latter group of safety interventions there was in general limited evidence for an effect of the safety interventions. In particular, evidence is strong that safety interventions based on engineering controls, alone or in combination with other types of interventions, were more effective than the other approaches.

These results of the relative effectiveness of the main types of intervention are well in accordance with the Public Health Hierarchy of Hazard Control (Castillo et al., 2006; Herrick & Dement, 1994).

Effectiveness of attitude and belief modification

Five previous reviews looked at the effects of attitude modification on accidents at work. In general, they found no evidence or weak evidence for the effectiveness of interventions aimed at attitude modifications in preventing occupational injuries. Attitude modification by means of educational efforts has been found to be ineffective in general (Robson et al., 2012). Likewise, two reviews of the effectiveness of back school programs in preventing back injuries found limited evidence that education is not effective in preventing back injuries (Hogan et al., 2014; Van Poppel et al., 1997). Although slightly more positive findings have been seen for farm safety educational programs (DeRoo & Rautiainen, 2000), and educational efforts aimed at preventing falls in the construction industry (Rivara, 2000), methodological limitations and inadequate study designs restrict the conclusions that can be drawn from these latter two reviews.

The results from these reviews thus support the findings of this review, in which attitude modification has a weak link to the improvement of safety and behavior and to reduction of accidents at work. While these previous reviews found limited evidence for no or little effect of attitude and belief modification, the present review found insufficient evidence.

Effectiveness of behavior modification

Reviews of behavioral interventions to prevent injuries have generally found weak, insufficient or no evidence of effect. Reviews on training interventions have shown either no effect in preventing nonfatal injuries in the construction industry (Molen et al., 2012), or insufficient evidence regarding training in general (Robson et al., 2012), or in reducing injuries among HCW due to violence (Runyan et al., 2000).

Insufficient evidence was also found for the use of drug and alcohol testing of professional occupational drivers for preventing injuries (Cashman et al., 2009), although a later review, not limited to occupational drivers, concluded that the evidence of the effectiveness of drug testing is weak in general, due to poor quality research. However, some evidence for the efficacy of random alcohol testing in the road transport industry was identified (Pidd & Roche, 2014).

The more classical BBS interventions, based on monitoring of behavior and feedback, have also shown limited effect and methodological problems. Although a meta-analysis of BBS interventions in the workplace revealed a statistically significant reduction in injuries after conducting a BBS intervention, the authors recommended that the results should be interpreted with caution, due to the poor marginal methodological quality of studies included in the meta-analysis (Tuncel, 2006). Similarly, a review of interventions for the prevention of eye injuries found that a low quality study using behavior modification techniques to improve the use of safety equipment showed a non-significant decrease in injury rates (Shah et al., 2009), while another review based on two studies concluded that behavioral interventions had an effect on eye injuries in manufacturing settings (Lipscomb, 2000).

Overall, the results of these reviews are consistent with the present review. BBS interventions, such as monitoring of behavior and feedback, have also shown limited effect in these studies, and methodological problems in the studies is also a limitation. Random alcohol testing among drivers, as a BBS intervention approach, indicated some effect in road transport industry, and thus provided some evidence of this type of behavior based approach.

Effectiveness of physiological modification

Physiological modification is a safety intervention that works either by exercising and thus increasing the flexibility and strength of the worker's body, or by using different forms of personal supportive equipment. The evidence for the effectiveness of personal protective or supportive equipment is mixed, and dependent on the type of equipment and injury.

A review of interventions toward physical training related injuries primarily within a military setting found strong evidence for mouth guards, semi rigid ankle braces and synthetic socks as effective interventions (Bullock et al., 2010). Two other reviews looked at the effectiveness of lumbar supports in the prevention of back injuries among hospital workers, and among managers and shop floor workers in industry. These two studies identified the same original studies, and reached the same conclusions, that there is no conclusive evidence for or against lumbar support (Hogan et al., 2014; Van Poppel et al., 1997).

The idea of physical training or exercise as a way to prevent accidents is relatively new in an occupational context, thus not many studies have been carried out. Therefore, the reviews in this area often end up being inconclusive. Mixed findings and low quality studies led to inconclusive evidence in a review of stretching to reduce work-related musculoskeletal injuries (Costa et al., 2008). Likewise, no firm conclusion could be drawn on the possible effectiveness of multifaceted physical training programs in preventing anterior cruciate ligament injury, although moderate evidence was found among female athletes engaged in team sports (Stojanovic & Ostojic, 2012). However, one review found strong evidence for agility-like training and prevention of overtraining as prevention approaches toward physical training related injuries (Bullock et al., 2010).

The reviews thus support our findings to some extent, but with a more mixed level of evidence. There seems to be some effect on athletes in relation to overtraining that are included in the present review. However, the latter seems to cover injuries in professional sports, and thus is not similar to the occupational groups that we have included in the present review. In this sense it does not directly contradict our findings concerning the effects of modification of employee physiology in a workplace setting.

Effectiveness of structural modifications

Structural interventions can either be initiated at the company level or at the societal level, and can be either “hard,” as is the case with legislation and engineering controls, or “soft” as is the case with economic incentives, corporate social responsibility, industry agreements and other more voluntary approaches. Generally the reviews evaluating the effectiveness of structural changes have found mixed results at both the societal and company level. However, there is a tendency that the results are more positive the “harder” the intervention.

Effectiveness of multifaceted safety interventions

Multifaceted interventions are often more complex and less comparable across studies than other types of interventions. Thus, reviews of these types of interventions seldom aggregate across studies due to the great heterogeneity between studies. Overall, mixed results have been found in previous reviews, although most reviews demonstrate a positive effect of the multifaceted safety interventions (DeRoo & Rautiainen, 2000; Lipscomb, 2000; van der Molen et al., 2012).

Other reviews have found no evidence that multifaceted educational interventions are effective in decreasing injury rates among agricultural workers (Rautiainen et al., 2008), or that regional safety campaigns consisting of a contest and inspections have any effect on nonfatal injuries within construction (Molen et al., 2012). These multifaceted safety interventions are mainly combining components at the individual level, and their conclusions are in line with the present review. Safety interventions combining components at the individual level have no or limited effect on the prevention of accidents at work.

Reviews have identified evidence of efficacy of multifaceted interventions across levels, including safety campaigns within farm safety (DeRoo & Rautiainen, 2000), eye injuries in manufacturing (Lipscomb, 2000), and on non-fatal injuries within the construction industry (van der Molen et al., 2013).