Live-Action Role Playing for Safety Training: Effectiveness Evaluation in two Italian Companies

Abstract

Researchers, corporate decision-makers, government agencies, and international bodies want to know how to improve safety and limit the serious social and economic consequences of occupational injuries. Understanding how to deliver effective training is important to achieving this aim. Gamification is considered a new frontier in effective safety training. This study tested the use of gamification in safety training, considering two Italian companies that used a live-action role-play methodology called “PratiCARE la Sicurezza.” Workers who were required by Italian law to receive safety training or retraining participated in a pre–post study. Collection of measures relating to knowledge, attitudes, behaviors, and perception of the health and safety climate took place at baseline and immediately after the intervention. Participants’ average safety knowledge at the end of the training was significantly better than the initial level, indicating that the training improved workers’ knowledge level.

Keywords

safety training knowledge gamification live-action role play simulated learning environment safety climate

Introduction

The unacceptably high number of occupational injuries in the EU-27 (3332 fatal injuries and more than 3.1 million non-fatal injuries in 2018)¹ costs 476 billion euros per year.² It is necessary to improve occupational health and safety interventions in order to reduce injuries.

There is very little research investigating the effectiveness of safety training. Hence, many questions are still unanswered.^3–5 While training is only one of many critical aspects of safety science, it is important for companies and policy makers to know both the best training content and the best training methods to deliver that content to workers.

More engaging training may be better because it encourages the application of safety knowledge and skills,⁶ as does using examples of both safe and unsafe responses to potentially dangerous situations.^7–9 Furthermore, active participation during training increases learners’ interest¹⁰ According to the principles of andragogy (adult education),¹¹ the use of examples, a participatory approach, and linking the content covered to every-day practice all contribute to facilitating learning in adults and experienced workers.

Training interventions using a constructivist approach¹² were delivered at a knitted fabric manufacturing company (KFPC) and a heavy lifting and transportation company (H&HL). Therefore, reference was made to the experiential learning model¹², which categorizes the acquisition of new knowledge, skills, and attitudes into four modes: concrete experience, reflective observation, abstract conceptualization, and active experimentation.

This study primarily addresses whether occupational health and safety training with live-action role-playing (LARP) has a beneficial effect on workers. Following Robson et al.⁴ and Ricci et al.,⁵ the effect of LARP training was assessed for knowledge, attitudes, behaviors, and health. In addition, following Ricci and Nucci,¹³ this study collected data about a safety climate and satisfaction as training outcomes.

Citing Griffin and Neal,¹⁴ Kapp¹⁵ has conceptualized work performance by two dimensions: safety compliance (with rules and procedures, related to the task) and safety participation (aimed at proactively promoting safety as a value and not as a mere fulfillment of obligations, related to the context). This frame of reference avoids the victim-blaming bias inherent in behavior-based safety, which tends to result in the failure to report injuries and hazards, leading to a lack of awareness of the related risks and the absence of mitigation. For these reasons, behavior-based safety prevents injured workers from getting the care they need and shifts medical costs from employers to workers.¹⁶ When injuries are not reported, the apparent improvement in the incidence of adverse events does not necessarily correspond to better safety performance. Even the possible absence of injuries would not be synonymous with safety. From a Safety-II perspective, while accidents clearly indicate the absence of safety, a lack of accidents cannot necessarily be used to infer a low probability of harm.¹⁷ Safety performance is therefore not dependent on ensuring that as few things as possible go wrong but that as many things as possible go right. To this end, workers are a resource who help make the system safer by developing effective ways to manage the conditions under which people work.¹⁸

Is it possible to learn safety through play? The use of features from the world of games in different contexts is called “gamification,” which allows the application of game mechanics to promote learning and problem-solving.¹⁹ Coonradt²⁰ was among the first to analyze the differences between the world of play and that of work, noting that the main difference between the two lies in the clarity and centrality of the objectives. The world of work suffers from the lack of clear objectives. At work, feedback is often inconsistent and vague. By contrast, in a game, winning is defined as achieving a precise goal or objective. Taking inspiration from games has proved to be an effective method of involving individuals in improving their results and in achieving pre-established objectives, such as safety.^21–24 Much depends on how the teacher introduces gamification. An active learning process through direct experience and problem-solving may be most effective.²⁵

Gamification's goals are to increase people's interest in the content presented^26–28 and to improve learning levels.²⁹ The concept of gamification implies that the user has fun. This differs from training without entertaining.^30,31

The game has been defined as a system in which players collide in an artificial, rules-defined conflict that allows for the achievement of a quantifiable result.³² It has a systemic component, defined based on its construction, and an experiential one that derives from the player's involvement.³³ It is the set of mechanisms and methods of a game that supports the player in creating a gaming experience.

Gamification has been successfully applied in health promotion³⁴ through role-play aimed at influencing users’ choices about their own dietary lifestyles. In another application, game elements were included in a training program on technological safety. The course initially presented the content as a bulleted list in a series of slides. To prevent trainees from finding the course boring, the trainers converted each point on the list into a small part of a story, forming a larger fictional tale.³⁵ The data suggest that learners appreciate narrative material, perceiving it as more fun, engaging, and easily recognizable than purely descriptive or explanatory text.³⁶ Hence gamification can positively help to improve learning results, but only with carefully designed training. Poorly designed training will not lead to improvement and may in fact result in worse performance.³⁵

Why gamification? Training, even mandatory safety training, does not always lead to learning, due to boredom, distractions, lack of clarity, and the reduced involvement of workers in learning activities. Through gamification, on the other hand, it is possible to involve participants who enjoy the experience and open up to new learning opportunities because they are driven by the curiosity to learn or win. In addition, gamification offers the possibility of repeating a scenario with an adverse outcome to analyze and make improvements. This can lead to a positive attitude towards learning and the achievement of ultimate success.³⁷

A simulated learning environment (SLE), which represents figures and features of everyday reality, can be an effective example of gamification for the acquisition and retention of new knowledge.^38–40 In an SLE, it is necessary to address fidelity, verification, and validation.⁴¹ The fidelity, or level of realism of the simulation, defines the degree of similarity of the training situation to the operational situation.⁴² Verification, on the other hand, assesses whether an SLE works as expected, while validation checks whether the conclusions reached by a simulation are similar to those of the real world.⁴³ Simulations do not have to reproduce the real world exactly. In fact, simpler simulations make it possible, especially for the less experienced, to focus on priority variables and details.⁴¹

A relevant feature, immersion, is the degree to which one perceives the game as realistic and affectively engaging.⁴⁴ In the case of LARP described here, immersion depends on (1) the role and characteristics of the characters; (2) the perception of self within the scenes represented; (3) the reaction produced by the representation; and (4) a guarantee of being able to express oneself without suffering negative consequences, fully allowing the playful learning experience. Immersion allows each learner to have a distinct role within the ludic activity, making the content immediate and relevant to the players. Immersion is determined by the participants’ perception of the game narration and by participants’ consequent responses. It refers to the degree to which the player feels like a real agent within the game world. Immersion can be achieved only to the extent that the game interaction is carefully designed.

Another aspect to consider is the presence of surprising events during the gaming experience: such events improve the understanding of a story, creating uncertainty, increasing the user's cognitive interest,⁴⁵ and involvement.⁴⁶ In addition, evidence indicates that surprising events do not change the amount of effort, subjective difficulty, or fun compared to games with no surprising events.⁴⁷

The training course “PratiCARE la Sicurezza” (in English the title can be considered as “Practice, care, safety”) was founded in 2014⁴⁸ through the establishment of a community of practice,⁴⁹ made up of experts in safety sciences and communication sciences. It draws upon the Theatre of the Oppressed⁵⁰ pedagogy of Paulo Freire,⁵¹ as well as upon andragogy,¹¹ and it is grounded in typical principles of gamification. Through LARP,⁵² the trainers (who are prevention experts) represent to workers, entertainingly, adverse events that actually happened on the job (typical SLE). After the first enactment of an accident, workers have to replace the trainer-actors, repeat the scenes, modify them, and avoid adverse events. This gives everyone a chance to learn from any unsuccessful attempts. It leads to learning through the analysis and correction of errors. All of this results in continuous unforeseen and surprising events, changing the scenario arising from the improvisation of those who play the different roles from time to time (typical in gamification). This serious but entertaining didactic experience produces a piece of shared knowledge, through interaction between participants/players and trainers/actors. The method allows participants to experience and not simply to observe, which helps them to learn through active involvement.^53,54 This training course seeks to change the conditions in which workers operate by addressing the systemic factors that, according to Reason,⁵⁵ result in errors.

The aim of this study was to test the effectiveness of “PratiCARE la Sicurezza.” This gaming approach is a LARP, representing with theatrical modalities what really happened in the workplace or what may very well happen. The study collected measures of knowledge, attitudes, behavior, and health perceptions.^3–5 In addition, we aimed to test the effect of training on a safety climate, because it represents a contextual factor that contributes to antecedents of safety performance, at both an organizational and an individual level.⁵⁶ Further, we sought to understand the effect of training in enabling workers to improve the safety conditions in which they work, since error is a consequence of systemic factors.⁵⁵ We therefore conducted a qualitative follow-up aimed at better interpreting the quantitative results.⁵⁷

The following hypotheses were evaluated:

(H1): The delivered safety training interventions produce improvement in knowledge, attitudes, behavior, health perceptions, and safety climate, when compared to baseline.

(H2): Participants are highly satisfied with the training.

This study is a safety training effectiveness evaluation carried out in two different Italian companies that participated voluntarily. Each company was guaranteed personalized training based on adverse events that had occurred in its own workplace. Workers’ safety representatives in each company were actively involved in the research project. Italian law provides this role in every company with at least one worker. Workers have the right/duty to elect a person who acts as a spokesperson for their demands on occupational health and safety. She/he is elected by the workers and not nominated by the employer. In many cases, the choice coincides with a trade union representative. He/she remains in office for three years and is eligible for re-election. The employer must ensure that she/he will receive adequate training, which consists of a 32-hour course with an annual update of four hours for companies with up to 50 workers and eight hours for companies with more than 50 workers. The minimum number of elected representatives is one for companies or production units up to 200 workers, three for companies or production units from 201 to 1,000 workers, and six for all companies or production units over 1,000 workers.

After describing the materials and methods for the verification of H1 and H2, the results and discussion follow.

Materials and Methods

Participants

A convenience sample of companies was recruited. Included in this study were workers required by Italian law to receive safety training or retraining. In addition, they had to have sufficient comprehension of the Italian language.

Intervention in a Knitted Fabrics Production Company (KFPC)

The intervention involved all the workers of this company (n = 102, n = five non-Italians, corresponding to 4.90%), with the exception of top management personnel, for reasons related to the aim of the study. The mean age of the participants was 43.55 years (SD = 11.90), and 57.84 percent of them were male (n = 59). Regarding the professional qualifications, 28 (27.45%) were white-collar workers, while 74 (87.25%) were blue-collar workers. In the textile industry as a whole, demographic features are the following: mean age of 43 years, 52 percent male, 22 percent white-collar, and 78 percent blue-collar worker.⁵⁸

Furthermore, 13 out of the 102 participants were precarious workers (12.75%). In accordance with legal regulations, fixed-term employment contracts may be freely stipulated to meet technical, organizational, productive, or replacement requirements, even if they relate to the employer's ordinary business activities. Finally, 28.43 percent of the sample (n = 29) had been given a specific role in the company's adverse event prevention system (e.g., member of the first aid team).

The training consisted of three four-hour sessions: one on risk perception and non-technical skills, by an occupational psychologist, an expert in adult learning; one about prevention of musculoskeletal disorders, conducted by a specialist in exercise and sport sciences; and one with the LARP “PratiCARE la Sicurezza.” In the LARP session, work-based scenarios included collisions between pedestrians and forklifts, risks of interference with contractors working on-site, electrocution risk, risk of cutting, risk of burn, work-related stress risk, use of hearing protection, repetitive movements, manual handling of loads, and emergency management. At the beginning of this training module, a woman who was injured at work told her story.

Six sessions of the course were held, with about 15–20 participants in each. The groups were set up in such a way as to ensure the greatest possible homogeneity regarding potential occupational risks. The research measured knowledge, attitude towards safe practices, and safety climate.

Intervention in a Hauling and Heavy Lifting (H&HL) Company

The intervention involved 143 workers of this company (31 women, 21.68%, and 112 men, 78.32%). The mean age was 45.44 years (SD = 9.57), company seniority 16.05 years (SD = 9.40), and 12 workers (8.39%) were not Italian. In terms of education, 39 (27.27%) had completed compulsory education, 71 (49.65%) had a diploma, 30 (20.98%) had graduated, and in three cases (2.10%) the data were missing. Finally, 84 (58.74%) participants were professionally qualified (e.g., white-collar workers) while 59 (41.26%) were blue-collar workers. In this industry as a whole, demographic features are the following: mean age of 42 years, 22 percent female, 48 percent white-collar workers, 11 percent not Italian workers, and a diploma is required.^59–63

The training was provided in nine different sessions with about 15–20 participants in each; the sessions each lasted four hours and were conducted through the LARP “PratiCARE la Sicurezza.” The groups were set up to guarantee the maximum possible homogeneity concerning the potential occupational risks present in different job tasks. The work-based scenarios were these: suspended load safety; forklift safety; near miss, accident, and injury analysis; maintenance procedures; risks of interference with external companies; work-related stress risk; risk of crushing; safety in heavy skidding and lifting; and signaling and delimitation of construction sites.

The measures of effectiveness, described below, detected knowledge, self-reported behaviors, perceived health, and the safety climate.

Intervention in a KFPC, Scales Description

Knowledge: adapted version of the scale proposed by Ricci et al.,⁶⁴ characterized by different formats: two items, recognition of the safe behaviors presented as a photographic stimulus (e.g., safety signage), with three alternatives and only one correct answer; 15 items, production—paper and pencil—of the correct answers to questions on the role of prevention and the safety procedures (e.g., “What is the safest action in the event of an earthquake tremor?”); three items, recognition of the correct answer, among the four alternatives, on obligations and sanctions that the law imposes on workers.

Attitudes towards the safe practices: self-evaluation questionnaire, made up of four items (e.g., “Applying the safety procedures at work prevents one from working well”), adapted from Ricci et al.,⁶⁵ with responses given on a five-point Likert-type scale (from one “completely disagree” to five “completely agree”).

Safety climate: Italian version in a reduced form to 11 verbal items, rated on a four-point Likert scale (from one, “completely disagree” to four “completely agree”), of the NOSACQ-50.⁶⁶ The items measure perceptions with respect to the actions of management (e.g., “the management involves the workers in decisions concerning safety”) and colleagues (e.g., “the workers of this company help one another to work safely”).

Satisfaction with regard to the training: questionnaire made up of five items,⁶⁵ rated on a five-point scale (from one, “wholly unsatisfactory” to five, “wholly satisfactory”).

Intervention in an H&HL, Scales Description

Knowledge: adapted version of the scale proposed by Ricci et al.,⁶⁴ composed of six items, production—paper and pencil—of the correct answers to questions concerning personal protective equipment, emergency procedures, and occupational risks and dangers (e.g., “What is the safest action in the event of an earthquake tremor?”).

Behaviors: self-evaluation questionnaire to measure the frequency of certain behaviors, made up of four items (e.g., “I use the safety walkways correctly when passing through the work areas”), adapted from Ricci et al.,⁶⁵ with responses given on a seven-point scale (from one, “never” to seven, “always”).

Health perception: VAS scale of EuroQol-5D (EQ-5D), an instrument for the measurement of quality of life.⁶⁷

Safety climate: Italian version in a reduced form to ten verbal items, rated on a seven-point Likert scale (from one, “completely disagree” to seven, “completely agree”), of the NOSACQ-50.⁶⁶ The items measure perceptions with respect to the actions of management (e.g., “the management involves the workers in decisions concerning safety”) and colleagues (e.g., “the workers of this company help one another to work safely”).

Satisfaction with regard to the training: questionnaire made up of five items,⁶⁵ rated on a five-point scale (from one, “wholly unsatisfactory” to five, “wholly satisfactory”).

Procedure

The data collection took place immediately before the opening session and immediately after the last one. The last questions asked were about worker satisfaction with the training.

The questionnaires were filled out in a rigorously anonymous form. In order to match the before and after responses of the participants, each participant was asked to calculate a univocal code according to the criteria established by the researcher. All participants freely agreed to take part in the study, signed an informed consent form, and provided their sociodemographic data. All the research activities were performed in total compliance with the ethical and deontological code of psychologists, and with the Declaration of Helsinki. This study was performed according to the American Psychological Association (APA) and National Association of Psychology ethical standards for the treatment of human subjects. Since data collection was anonymous and involved no identifying information and no medical treatment, no human research subjects approval for the study was requested. Participants were informed that their participation was voluntary, that they could leave the study at any time, and that their data would be treated anonymously. In addition, they were informed that by starting the survey they would be considered to have read and accepted the informed consent.

Data Analysis

In terms of the descriptive analyses, the means, standard deviations, counts, and percentages were reported for continuous and categorical variables. A paired samples two-tailed t-test was used to compare the means pre- and post-training, after evaluation of normal distribution with the Kolmogorov-Smirnov test. The significance level was set at alpha = 0.05. The Cohen's d was used to measure the effect size. Statistical analyses were conducted with SPSS 21.0.

Six-Month Follow-Up Qualitative Investigation

Eight workers from different departments were randomly selected from each of the two companies involved (KFPC, six women and two men; H&HL, two women and six men) to participate in this qualitative survey. A worker health and safety representative; the health, safety, and environment manager; and three safety supervisors (in Italian preposti) were always present in the two different groups as safety specialists. In this way, it was possible to collect heterogeneous contributions concerning the different work activities, with a fairly wide range of potential responses and at the same time ensuring the involvement of each participant.⁵⁷

Instrument

In order to understand, on a qualitative level, the degree to which training enabled workers to improve their safety conditions, a single focus group technique⁵⁷ was used for each company. This measure allows evidence to be gathered through group interaction, on the topic proposed by the researcher.⁵⁷ See Table 2 for results.

Table 1.

Mean (Standard Deviation) of all the Measures Surveyed T₁, T₂ per Intervention (Minimum and Maximum Score), and Corresponding p-Value Within Subjects t-Test.

Measures	Intervention	T₁	T₂	p
Knowledge	KFPC (0–35)	11.24 (5.10)	30.69 (4.16)	<.001
Knowledge	H&HL (0–12)	7.16 (2.46)	10.46 (1.91)	<.001
Attitudes	KFPC (1–5)	4.15 (0.85)	4.16 (0.84)	n.s.
Behaviors	H&HL (1–7)	5.03 (1.05)	5.27 (1.01)	n.s.
Perceived health	H&HL (0–100)	72.60 (15.69)	73.80 (16.36)	n.s.
Safety climate	KFPC (1–4)	3.03 (0.67)	3.05 (0.62)	n.s.
Safety climate	H&HL (1–7)	5.97 (0.67)	6.09 (0.86)	n.s.
Satisfaction with training	KFPC (1–5)	NA (NA)	4.52 (0.59)	NA (NA)
Satisfaction with training	H&HL (1–5)	NA (NA)	4.73 (0.56)	NA (NA)

Procedure

Data collection took place in a company meeting room, during usual working hours, more than six months after the end of the training. In order to ensure an adequate level of attention during the discussion and the emergence of all content relevant to the survey, each meeting lasted about an hour.⁶⁸

Each focus group was structured with a “funnel” pattern, so as to maintain the same initial standard questions for both situations considered, but with the opportunity of delving into specific content for each of the two different occupational contexts.⁵⁷

In order to ensure data validity, the moderator took a structured approach to managing interactions among participants. He encouraged those who tended to speak less and limited those who tended to impose themselves in the discussion.⁵⁷

During each focus group, data were collected by taking notes on what participants were saying and by observing their behaviors.

Data Analysis

Using qualitative content analysis,⁶⁹ information was systematically coded into categories, thereby uncovering patterns not detectable by simply reading the notes taken by the researcher.

The data were then coded in two stages.⁶⁸ In the first step, category codes were created, associated with the different reflections that emerged and recurring keywords for each specific theme. In the next step, the principal investigator eliminated, combined, or subdivided the initial categories, paying attention to the recurring ideas and themes related to each other. This allowed for a comparison of the content that emerged in the two different companies involved in the survey.

A synthesis of data from the six-month follow-up qualitative investigation is presented in Table 2.

Table 2.

Focus Groups in the two Companies Involved: Questions and Answers.

Question 1: What do you remember, and what impressed you the most about the training you participated in?
KFPC	H&HL
There are aspects of the work that we can improve by keeping them in mind and taking care of them every day (e.g., sweeping the floor to avoid slipping). In the same way, we can find solutions to unforeseen problems (e.g., handling loads that are too heavy). The experience of seeing each other again, even when you make mistakes or take things for granted because the scenes referred to events that actually happened in the company. A great help in rethinking comes from the entertainment with which the situations are represented. In this way we all become our own controllers, to recognize mistakes and avoid them (e.g., safe use of tools, maintaining correct postures). The training was more immediate and engaging than with slides. In particular, the testimony of the worker who had suffered a serious injury was very impactful. It made it possible to understand what to do and what not to do to protect health (e.g., use of paths, space management, and visibility in warehouse areas). It focused attention on keeping the rush under control to avoid making mistakes.	You are more attentive, more involved, and you remember more, especially when you live the moment in which you go from being a spectator to being an actor, intervening in the scenes. This allows you to experience the situations directly. An emotion, a memory that helps to create culture. It is effective to review specific events of our company. Relive what happened, to learn. Workers keep talking about it; it doesn't end with the end of the course. A nice stimulating moment of complicity and sharing that doesn't happen in standard courses. It is more enjoyable than training with slides, it is fun even if the topics are serious, and it gives a much better idea than many numbers of what must or must not be done to avoid an injury. It produces a greater sense of reality. It is a way to interpret in a more complete and participatory sense what safety is about because it is not possible to proceduralize every activity. Not being a passive subject allows people to make a difference in improving safety.
Question 2: How has this training allowed you to improve the safety conditions of your work?
KFPC	H&HL
It has allowed us to identify hazards, even for those who work as white collars, so as to act safely (e.g., use appropriate shoes in hazardous areas and move carefully when going to the warehouse). Now we stop and think before we do something (e.g., don't wear clothes that can get caught in equipment), and we are more inclined to use PPE because we understand better than before their usefulness and the meaning of the rules. Workers are more interested in talking to each other about how to improve safety. So everyone is involved to avoid doing the wrong thing (e.g., be careful not to put obstacles in front of escape routes) and to report risky conditions and obtain improvement actions by the employer (e.g., the height of raw materials put on pallets has been lowered to ensure visibility).	It has increased participation and interest in safety, prompting people to stop to reflect and collaborate before starting the planned activities, even for those who work in offices. We have requested and obtained permission to arrange the workstations and to keep the work areas clear of obstacles. You notice the wrong situations more than before, “Like when you use the highlighter.” The company has implemented an app to collect reports from workers, which arrive in a simple, clear, spontaneous way. Thanks also to the support of safety supervisors (in Italian preposti).

Results

The data collected partially confirm H1 because the mean value of knowledge significantly increased after the course, showing a large effect size (Cohen's d): KFPC, 4.18; H&HL, 1.50. The other outcomes (safety climate, attitudes, perception of health, and behaviors), instead, remained substantially unchanged. Furthermore, the very high level of satisfaction with training, in both subsamples (KFPC, 4.52/5; H&HL, 4.73/5), confirmed H2. Table 1 reports the mean and standard deviation surveyed before (T₁) and after (T₂) the training and corresponding p-values related to changes pre- and post-training within subjects. Satisfaction with training, for evident reasons, was measured only after the intervention.

Safety training using gamification significantly increased knowledge relating to personal protective equipment, emergency procedures, and occupational risks and dangers. There was a non-statistically significant increase in other measures after the training. This could encourage companies to use gamification as an alternative to less effective traditional methodologies (e.g., classroom lectures).

The findings of the six-month follow-up qualitative investigation are summarized in Table 2.

Discussion

The gamification of “PratiCARE la Sicurezza” is effective because it addresses the needs of the participants and corresponds to the principles of adult learning. This method facilitated the development of knowledge and produced a very high level of satisfaction with training. The method allows workers to learn by experiencing an SLE rather than by observing from the outside. As action theory states, the active involvement of the participants facilitates learning.^53,54

Content analysis (see Table 2) of the qualitative data collected through focus groups corroborates the evidence of effectiveness in both workplaces studied. Workers learned “to recognize mistakes and avoid them” (KFPC), “to understand what to do and what not to do to protect health” (KFPC), “keeping the rush under control to avoid making mistakes” (KFPC), “what must or must not be done to avoid an injury” (H&HL), and to “notice the wrong situations more than before” (H&HL). They also expressed a great satisfaction with the training, saying that “the training was more immediate and engaging than with slides” (KFPC), and that it showed “a nice stimulating moment of camaraderie and sharing that doesn't happen in standard courses” (H&HL), and that “it is more enjoyable than training with slides; it is fun even if the topics are serious” (H&HL). Participants clearly confirmed that this method allows workers to learn by experiencing an SLE and not simply by observing. KFPC workers appreciated “the experience of seeing each other again, even when you make mistakes or take things for granted because the scenes referred to events that actually happened in the company. A great help in rethinking comes from the entertainment with which the situations are represented.” H&HL workers said that “you are more attentive, more involved, you remember more, especially when you live the moment in which you go from being a spectator to being an actor, intervening in the scenes. This allows you to experience the situations directly. An emotion, a memory that helps to create culture.” Likewise, they said that “it is effective to review specific events of our company. Relive what happened, to learn. Workers keep talking about it, it doesn't end with the end of the course.” Moreover, participants noted that “not being a passive subject allows people to make a difference in improving safety.” Thus, workers specifically stated that they had experienced immersion⁷⁰ through the LARP in which they participated. This is because the content of the role-play was immediate and relevant to the actors, so that they felt like protagonists within the play world.

It may seem obvious and trivial to be able to say that a training intervention has produced an improvement in knowledge, compared to the baseline. This is what we all hope for after any training intervention. However, it is a common experience that this does not always happen and that, when it does occur, the degree of effectiveness can be highly differentiated. One meta-analysis³ looked at 49 studies that had evaluated safety training effectiveness for knowledge improvement, according to commonly accepted principles.⁷¹ The analysis found a lack of effect for two studies, and in the remaining studies, it found an effect size that was small (10), medium (14), large (8), and very large (15). Eleven of these showed levels higher than the result of the H&HL sample (1.50), but all were under the KFPC sample score (4.18). Another systematic review⁴ considered five studies that had assessed the effect of training on knowledge, none involving similar workplaces with similar learners to those covered by the present study. One study showed no evidence of an effect, and the others ranged from medium (1.50) to very large (3.58) effect, but always under the result of the KFPC sample. Similarly, another meta-analysis⁵ identified seven studies in the knowledge domain, all but one evidencing a positive training effect. The overall combined effect size was 1.03 (confidence interval: 0.49–1.58) and far exceeded the evidence synthesis algorithm's criterion of large effect (0.80). Again, no intervention showed evidence of approaching the level of performance recorded here for the KFPC, and none of the samples involved workplaces and learners similar to the contexts we analyzed. In light of these data, it is clear that our findings add knowledge not yet published, considering the work contexts and employee characteristics analyzed. There was an Italian study¹³ that evaluated the effect of a training partially provided through LARP, but it was done in a sample of metal-mechanical companies, which are very different workplaces from those in this study.

The lack of improvement in attitudes is consistent with data classifying this kind of evidence as insufficient⁴ or unsatisfactory.⁶⁵ The unchanged perception of health is consistent with recent meta-analyses, which have found modest effects in some cases and no effect in others.⁵ Significant effects were found only in long-term interventions. These findings indicate that it is not possible to demonstrate the short-term impact of training on occupational health.⁴ In this sense, training alone may not be sufficient to achieve significant effects.

Even the behaviors do not vary from baseline. This may be due to the absence of data collected over time. Robson et al.⁴ examined ten studies that found behavioral effects, most of which were positive. Follow-up typically took place at six months. Similarly, the absence in this study of an effect of training on safety climate may be due to the time needed for training to have an effect on climate. It is possible that time may have a greater effect on the impact of training on behaviors and damage to health than it does on knowledge.³ We therefore intend to collect follow-up data to be presented in a forthcoming publication.

Given the nature of an ecological study, the training and data collection, in this case, involved a convenience sample, rather than a fully representative sample of companies. Bearing in mind the necessary caution in generalizing the results, all the workers who were required by Italian law to receive safety training or retraining participated. It would certainly have been useful to provide for a control group, but this limitation also follows from the ecological nature of the study. In fact, the participating companies decided to adopt this method because they considered it more effective than others, and it was not possible to recruit companies that would make a different choice. Data collection for a meta-analytic evaluation of safety training effectiveness included 95 quasi-experimental studies.³ A systematic review of these studies⁴ evidenced a general lack of high-quality randomized controlled trials. Another meta-analysis⁵ identified only seven randomized controlled trials out of 28 studies. Five studies made use of random sampling. There were three non-random investigations comparing training and control groups. Why these limitations? It is likely that this is because of the difficulty of carrying out complex investigations in the field. Investigations have to be done at places and times and by methods that are possible. The body of evidence available from these less-than-perfect studies offers us a great opportunity to know more about the criteria for quality training and how to evaluate its effectiveness.

A final methodological limitation is a fact that this study did not compare different training methods. This was due to the impossibility of imposing on the companies that voluntarily participated in this research training other interventions that they considered less suitable and in which they were not interested. However, the previous evidence³ has demonstrated greater knowledge acquisition for training methods requiring trainees’ active participation, similar to the intervention presented in this manuscript.

The results of this study will provide scholars and practitioners with knowledge about gamification and useful elements that allow for improving the quality and avoiding errors in the delivery of safety training. The courses presented here, combining examples and a participative approach, overcome the limits of traditional classroom lectures. This is especially useful to improve learning about occupational health and safety and assumes a great value for high-risk sectors such as manufacturing and hauling and heavy lifting.

Experimentation with this methodology and the measures used in this study offer an intervention that allows companies to deliver quality training and to test its efficacy. In addition, we emphasize the need for future research examining the influence of situational variables (e.g., safety climate) on safety training.

Finally, the results indicate that gamification may be effective in additional occupational contexts (e.g., the construction industry). Applying LARP to safety training in other areas and verifying its effectiveness appears to be the most promising avenue for further research.

Conclusions

Unfortunately, we still know too little about factors that make safety training effective. The scarcity of data regarding the effectiveness of safety training seriously hinders the evidence-based identification of the strengths and weaknesses of training. More training effectiveness data could make an important contribution to limiting the serious social and economic consequences of injuries at work.

Having demonstrated the quality of the LARP adopted the current study was limited in not having quantified the impact of other training methods. An opportunity was missed to integrate the LARP into training sessions on risk perception and on prevention of musculoskeletal disorders in one of the companies involved (KFPC). However, our research corroborates the findings of Burke et al.,³ who analyzed 95 quasi-experimental studies and concluded that training involving behavioral modeling, a substantial amount of practice, and dialogue is generally more effective than other methods of safety and health training. Given the ecological nature of this study, it would not have been possible to exclude some of the companies that voluntarily joined the experimentation, imposing the adoption of different training interventions.

Based on the qualitative data collected, we can document that this training allowed workers to improve the safety conditions of their work. In the KFPC focus group, it emerged that following the training, “workers are more interested in talking to each other about how to improve safety. So everyone is involved to avoid doing the wrong thing (e.g., be careful not to put obstacles in front of escape routes) and to report risky conditions and obtain improvement actions by the employer (e.g., to ensure visibility, the height of raw materials put on pallets has been reduced).” Similarly, in the H&HL example, “The company has implemented an app to collect reports from workers, which arrive in a simple, clear, spontaneous way. Thanks also to the support of safety supervisors.” Additional comments were that “We have requested and obtained permission to arrange the work stations and to keep the work areas clear of obstacles,” and that the training “has increased participation and interest in safety, prompting people to stop to reflect and collaborate before starting the planned activities, even for those who work in offices.”

With proper organizational and economic resources, it is appropriate to collect data over time by repeated follow-up. In this way, it is possible to test the persistence of effects and possible delayed outcomes (e.g., behaviors).

Thanks to the voluntary participation of two Italian companies that used a LARP called “PratiCARE la Sicurezza,” this study made it possible to evaluate the effects of gamification in safety training.

The encouraging results contribute to filling a scientific gap by showing a marked increase in safety knowledge on the part of the participants at the end of the training. The excellent satisfaction scores testify to the quality of the intervention. This suggests that providing this kind of training in other contexts would improve the knowledge level of workers in those contexts as well. The secret of success lies in delivering interesting interventions with content that is closely related to everyday practice, instead of being limited to illustrating normative obligations. These results provide researchers, corporate decision-makers, government agencies, and international bodies with additional knowledge as to how to deliver training to workers.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Federico Ricci, PhD, is an occupational psychologist since 1993 and professor of occupational psychology at University of Modena and Reggio Emilia. He is an expert in andragogy and a trainer on non-technical skills in occupational safety. Scientific leader of several projects, in occupational safety training area. He conducted his master's degree thesis in the field of training effectiveness evaluation and did his Ph.D. on safety training effectiveness evaluation. Author of publications in international journals, coordinates various research activities in the safety training effectiveness evaluation. Holds hundreds of hours of occupational safety training annually, including training of trainers, for multiple higher educational institutions and companies.

Giulia Bravo, Public Health PhD, has many years of experience performing data management and data analyst. She is involved in various public health research projects in the areas of occupational, clinical and public health.

References

Eurostat. Accidents at work statistics, https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Accidents_at_work_statistics (2019, accessed 28 June 2021).

EU-OSHA, Work-related accidents and injuries cost EU €476 billion a year according to new global estimates, https://www.eurofound.europa.eu/news/news-articles/eu-osha-work-related-accidents-and-injuries-cost-eu-eu476-billion-a-year-according-to-new-global (2017, accessed 28 June 2021).

Burke

Sarpy

Smith-Crowe

, et al. Relative effectiveness of worker safety and health training methods. Am J Public Health 2006; 96: 315–324.

Robson

Stephenson

Schulte

, et al. A systematic review of the effectiveness of occupational health and safety training. Scand J Public Health 2012; 28: 193–208.

Ricci

Chiesi

Bisio

, et al. Effectiveness of occupational health and safety training: a systematic review with meta-analysis. J Workplace Learn 2016; 28: 355–377.

Kao

Spitzmueller

Cigularov

, et al. Linking safety knowledge to safety behaviours: a moderated mediation of supervisor and worker safety attitudes. Eur J Work Organ Psychol 2019; 28: 206–220.

Merrill

Tennyson

Posey

. Teaching concepts: an instructional design guide. 2nd ed. Englewood Cliffs, New Jersey: Educational Technology Publications, 1992.

Foshay

. Some principles underlying the cognitive approach to instructional design. In: Silber

Foshay

Watkins

, et al. (eds) Handbook of improving performance in the workplace, vol. 1–3. Hoboken, New Jersey: John Wiley & Sons, 2009, pp. 3–22.

Taylor

Wirth

Olvina

, et al. Experimental analysis of using examples and non-examples in safety training. J Saf Res 2016; 59: 97–104.

10.

Feinstein

Mann

Corsun

. Charting the experiential territory. J Manag Dev 2002; 21: 732–744.

11.

Knowles

. Andragogy in action. San Francisco, CA: Jossey-Bass, 1984.

12.

Kolb

. Experiential learning: experience as a source of learning and development. New Jersey: Prentice-Hall Inc, 1984.

13.

Ricci

Nucci

. Safety integrated model-training (SIM-t) and its evaluation: a safety training proposal for mechanical companies. Saf Sci 2022; 146: 105538.

14.

Griffin

Neal

. Perceptions of safety at work: a framework for linking safety climate to safety performance, knowledge, and motivation. J Occup Health Psychol 2000; 5: 347–358.

15.

Kapp

. The influence of supervisor leadership practices and perceived group safety climate on employee safety performance. Saf Sci 2012; 50: 1119–1124.

16.

Dekker

. The dereliction of behavior-based safety. https://safetyrisk.net/the-dereliction-of-behavior-based-safety/ (2017, accessed).

17.

Hollnagel

Wears

Braithwaite

. From safety-I to safety-II: a white paper. The resilient health care net. University of Southern Denmark; University of Florida; USA; and Macquarie University, Australia, 2015.

18.

Hollnagel

. Safer systems: people training or system tuning? EJIHPE 2021;11: 990–998.

19.

Khaleel

Ashaari

Wook

TSMT

, et al. Gamification-based learning framework for a programming course. In: Proceedings of 6th International Conference on Electrical Engineering and Informatics (ICEEI), Langkawi, Malaysia, 25–27 November 2017.

20.

Coonradt

. The game of work. Salt Lake City, Utah: Shadow Mountain, 1984.

21.

Mora

Planas

Arnedo-Moreno

. Designing game-like activities to engage adult learners in higher education. In: Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality, November 2016. New York, NY, USA: Association for Computing Machinery.

22.

van Roy

Deterding

Zaman

. Uses and Gratifications of Initiating Use of Gamified Learning Platforms. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, 21–26 April 2018. New York, NY, USA: Association for Computing Machinery.

23.

Deterding

Dixon

Khaled

, et al. From game design elements to gamefulness: defining “gamification”. In: Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments, September 2011. New York, NY, USA: Association for Computing Machinery.

24.

Malone

. Toward a theory of intrinsically motivating instruction. Cogn Sci 1981; 5: 333–369.

25.

Martí-Parreño

Méndez-Ibáñez

Alonso-Arroyo

. The use of gamification in education: a bibliometric and text mining analysis. J Comput Assist Learn 2016; 32: 663–676.

26.

Domínguez

Saenz-de-Navarrete

de-Marcos

, et al. Gamifying learning experiences: practical implications and outcomes. Comput Educ 2013; 63: 380–392.

27.

Lee

Hammer

. Gamification in education: what, how, why bother? Acad Exch Q 2011; 15: 1–5.

28.

Slater

Rouner

. Entertainment—education and elaboration likelihood: understanding the processing of narrative persuasion. Commun Theory 2002; 12: 173–191.

29.

Kapp

. The gamification of learning and instruction: game-based methods and strategies for training and education. San Francisco, CA: Pfeiffer, 2012.

30.

Abt

. Serious games. Lanham, MD: University Press of America, 1987.

31.

Karagiorgas

Niemann

. Gamification and game-based learning. J Educ Technol Syst 2017; 45: 499–519.

32.

Salen

Tekinbaş

Zimmerman

. Rules of play: game design fundamentals. Cambridge, MA: MIT Press, 2004.

33.

Huotari

Hamari

. Definition for gamification: anchoring gamification in the service marketing literature. Electron Mark 2017; 27: 21–31.

34.

Peng

. Design and evaluation of a computer game to promote a healthy diet for young adults. Health Commun 2009; 24: 115–127.

35.

Armstrong

Landers

. Gamification of employee training and development. Int J Train Dev 2018; 22: 162–169.

36.

Narvaez

van den Broek

Ruiz

. The influence of reading purpose on inference generation and comprehension in reading. J Educ Psychol 1999; 91: 488–496.

37.

Nah

FF-H

Zeng

Telaprolu

, et al. Gamification of Education: A Review of Literature. In: Proceedings of International Conference on HCI in Business, Government, and Organizations, Heraklion, Crete, Greece, 22–27 June 2014.

38.

Ricci

Salas

Cannon-Bowers

. Do computer-based games facilitate knowledge acquisition and retention? Mil Psychol 1996; 8: 295–307.

39.

Wainess

Oneil

Baker

. Classification of learning outcomes: evidence from the computer games literature. Curric J 2005; 16: 455–474.

40.

Sitzmann

. A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Pers Psychol 2011; 64: 489–528.

41.

Feinstein

Cannon

. Constructs of simulation evaluation. Simul Gaming 2002; 33: 425–440.

42.

Hays

Singer

. Simulation fidelity in training system design: bridging the gap between reality and training. New York, NY, US: Springer-Verlag Publishing, 1989.

43.

Pegden

Sadowski

Shannon

. Introduction to simulation using SIMAN. New York, NY, US: McGraw-Hill, Inc., 1995.

44.

Bedwell

Pavlas

Heyne

, et al. Toward a taxonomy linking game attributes to learning: an empirical study. Simul Gaming 2012; 43:729–760.

45.

Campion

Martins

Wilhelm

. Contradictions and predictions: two sources of uncertainty that raise the cognitive interest of readers. Discourse Process 2009; 46: 341–368.

46.

Howard-Jones

Demetriou

. Uncertainty and engagement with learning games. Instr Sci 2008; 37: 519–536.

47.

Wouters

van Nimwegen

van Oostendorp

, et al. A meta-analysis of the cognitive and motivational effects of serious games. J Educ Psychol 2013; 105: 249–265.

48.

PratiCARE la sicurezza, www.praticarelasicurezza.it (2021, accessed 28 June 2021).

49.

Wenger

. How we learn. Communities of practice. The social fabric of a learning organization. Health Forum 1996; 39: 20–26.

50.

Boal

. Theater of the oppressed. London, UK: Pluto Press, 2000.

51.

Freire

. Pedagogy of the oppressed. London, UK: Bloomsbury Publishing, 2018.

52.

Vanek

Peterson

. Live action role-playing (LARP): insight into an underutilized educational tool. In: Schrier

(ed.) Learning, education and games, vol. 2. Pittsburgh, PA: ETC Press, 2016, pp. 219–240.

53.

Hacker

. Activity: a fruitful concept in industrial psychology. In: Frese

Sabini

(eds) Goal directed behavior: the concept of action in psychology. Mahwah, NJ, USA: L. Erlbaum Associates, 1985, pp. 262–284.

54.

Frese

Zapf

. Action as the core of work psychology: a German approach. Handb Ind Organ 1994; 4: 271–340.

55.

Reason

. Human error: models and management. Br Med J 2000; 320: 768–770.

56.

Beus

McCord

Zohar

. Workplace safety: a review and research synthesis. Organ Psychol Rev 2016; 6: 352–381.

57.

Morgan

. Focus groups. Ann Rev Sociol, 1996; 22:129–152.

58.

Soges. I principali numeri sull’occupazione del tessile-abbigliamento italiano prima e durante la crisi, https://informatex.it/wp-content/uploads/2017/06/Occupazione-nel-Tessile-Abbigliameto-in-Italia.pdf (2014, accessed 19 March 2022).

59.

Paradowska

Platje

. Key challenges facing the European transport labour market (No. 2016–03). Ordnungspolitische Diskurse, 2016.

60.

CESE. Declaration on equal opportunities for women and men in the transport sector, https://www.eesc.europa.eu/it/agenda/our-events/events/women-transport-eu-platform-change/declaration-equal-opportunities-women-and-men-transport-sector (2017, accessed 19 March 2022).

61.

iStat. Imprese lavoratori dipendenti, http://dati.istat.it/Index.aspx?DataSetCode=DICA_ADIPWP (2022, accessed 19 March 2022).

62.

Ministero del lavoro. X rapporto annual, https://www.lavoro.gov.it/documenti-e-norme/studi-e-statistiche/Documents/Decimo%20Rapporto%20Annuale%20-%20Gli%20stranieri%20nel%20mercato%20del%20lavoro%20in%20Italia%202020/Sintesi-X-Rapporto-Annuale-stranieri-nel-mercato-del-lavoro.pdf (2020, accessed 19 March 2022).

63.

EUR-Lex. Road Haulage and passenger transport companies: operating rules, https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=LEGISSUM:mi0054 (2020, accessed 19 March 2022).

64.

Ricci

Pelosi

Panari

, et al. Safety Value in Practice for an Effective Occupational Health and Safety Training. In: Proceedings of Organization 4.1: The role of values in the organizations of the 21st century ISSWOV (International Society for the Study of Work and Organizational Values), Trieste, Italy, July 1–4 2018.

65.

Ricci

Pelosi

Panari

, et al. Safety Training 4.0: Active, Collaborative, Human-Focused Practices, to Improve Health at Work. In: Proceedings of Transdisciplinary Engineering Methods for Social Innovation of Industry 4.0: 25th ISPE Inc. International Conference on Transdisciplinary Engineering, Modena, Italy, July 3–6, 2018.

66.

The Nordic Council of Ministers. NOSACQ 50item, Italian version, https://nfa.dk/da/Vaerktoejer/Sporgeskemaer/Safety-Climate-Questionnaire-NOSACQ50/NOSACQ50-translations (2021, accessed 28 June 2021).

67.

Balestroni

Bertolotti

. EuroQol-5D (EQ-5D): an instrument for measuring quality of life. Monaldi Arch Chest Dis 2012; 78: 155–159.

68.

Nyumba

Wilson

Derrick

, et al. The use of focus group discussion methodology: insights from two decades of application in conservation. Methods Ecol Evol 2018; 9: 20–32.

69.

Moretti

Van Vliet

Bensing

, et al. A standardized approach to qualitative content analysis of focus group discussions from different countries. Patient Educ Couns 2011; 82: 420–428.

70.

Bedwell

Wildman

Diaz Granados

, et al. Collaboration at work: an integrative multilevel conceptualization. Hum Resour Manag Rev 2012; 22: 128–145. doi:10.1016/j.hrmr.2011.11.007

71.

Cohen

. Statistical power analysis for the behavioral science. Hillsdale, NJ: L.E.A, 1988.