Modelling and Simulating of Risk Behaviours in Virtual Environments Based on Multi-Agent and Fuzzy Logic

Abstract

Due to safety and ethical issues, traditional experimental approaches to modelling underground risk behaviours can be costly, dangerous and even impossible to realize. Based on multi-agent technology, a virtual coalmine platform for risk behaviour simulation is presented to model and simulate the human-machine-environment related risk factors in underground coalmines. To reveal mine workers' risk behaviours, a fuzzy emotional behaviour model is proposed to simulate underground miners' responding behaviours to potential hazardous events based on cognitive appraisal theories and fuzzy logic techniques. The proposed emotion model can generate more believable behaviours for virtual miners according to personalized emotion states, internal motivation needs and behaviour selection thresholds. Finally, typical accident cases of underground hazard spotting and locomotive transport were implemented. The behaviour believability of virtual miners was evaluated with a user assessment method. Experimental results show that the proposed models can create more realistic and reasonable behaviours in virtual coalmine environments, which can improve miners' risk awareness and further train miners' emergent decision-making ability when facing unexpected underground situations.

Keywords

Virtual Human Virtual Reality Multi-Agent Emotion Fuzzy Logic

1. Introduction

Underground mining is a hazardous industrial task. The harsh working environment results in great hindrances or constraints on identifying underground risk behaviours, which involve various factors related to humans, machines and the environment. Traditional experimental approaches to modelling and simulating underground risk behaviours normally are costly, dangerous or even impossible to realize because of the safety and ethical issues involved. Virtual Reality (VR) technology brings forward an effective approach to diagnosing and preventing underground risk behaviours. In a virtual coalmine environment, a virtual miner agent can direct trainees to practice and experience underground situations, activities and processes without putting them in any actual danger and further improve trainees' risk awareness. Moreover, some emergent situations, which have not previously occurred but could be encountered at a mining site, can also be reproduced. However, most current coalmine virtual environments hardly take into account the complex behaviour mechanism of human-machine-environment related risk factors in underground coalmine accidents [1–2].

Multi-agent technology has been widely applied in problem solving, virtual environments, collective robotics and many other areas [3]. In a multi-agent virtual environment, agents can autonomously sense the environment and reason about their goals. By defining the interactive rules among individuals, agents can fully cooperate with each other to create the complex mechanism of a dynamics system. Therefore, multi-agent technologies are particularly suitable for exploring the risk behaviours in an underground coalmine. Based on multi-agent technologies, in this paper we present a framework of virtual coalmine environments to model and simulate underground risk behaviours.

According to the recent investigation and statistics of underground accident cases in China and abroad [4–5], unsafe human behaviours account for more than 80% of underground accidents. Therefore, a significant challenge in designing a coalmine virtual environment is to effectively model and simulate underground human risk behaviours. Moreover, typical coalmine accident investigations conducted in recent 20 years in China also show major accidents mainly result from “violations of safety rules and regulations” [4]. This involves a broad range of physical, psychological and social factors, such as work experience, training level, fatigue, stress, panic, emotional states, conscientiousness and personality characteristics. Therefore, in order to approach underground miners' behaviour characteristics, we must take into account underground mine workers' physiological and psychological attributes such as emotions, personality and motivation. In this paper, we focus on modelling virtual miners' emotional behaviours based on the typical risk factors resulting in coalmine accidents.

Currently, many virtual human models are available for real-time applications. However, because of the scarcity of human and social behavioural data for underground coalmines, most current virtual human models can't be directly used in a virtual coalmine environment. There are a lack of techniques to model and simulate underground human behaviours that allow human risk factors, such as fatigue, stress and panic, to be taken into account. In this paper, we attempt to incorporate current virtual human technologies and underground miner's characteristics into modelling underground human risk behaviours. Based on OCC (Ortony, Clore and Collins) appraisal theories [6] and fuzzy logic techniques, we present a fuzzy emotional behaviour model for a virtual miner to simulate underground human responses to potential hazardous events. The proposed emotional behaviour model can create more believable virtual coalmine environments. It can be used as a training tool for potential miners to improve their risk awareness and emergency decision-making ability. In addition, it can also be used to analyse the risk factors of underground accidents, especially human risk factors related to underground mine workers' emotions, personality and motivation needs.

The main contributions of this paper are two-fold. First, it presents a modelling methodology of underground risk behaviours in a virtual coalmine environment based on multi-agent technologies. Second, it implements the emotional behaviour model for underground virtual miner agents based on OCC appraisal theories and fuzzy logic techniques.

This paper is organized as follows. In Section 2 related work is reviewed. In Section 3 we propose a multi-agent based virtual coalmine environment framework to model and simulate underground risk behaviours. Section 4 introduces the modelling of human risk behaviour. A fuzzy emotional behaviour mode will be presented for a virtual miner agent. Section 5 presents the simulating cases. Section 6 describes the evaluation of believability. Finally, Section 7 presents some conclusions and possible future work.

2. Related work

In the last few years, VR technology has increasingly been used in mining industry. In [1], a PC-based VR training simulator is developed to improve risk awareness for underground miners. After identifying one of the hazards, trainees are required to select appropriate corrective actions from on-screen button icons. But the training simulator pays little attention to the modelling of human risk behaviours. In [7], a generic safety training tool, SAFE-VR, is introduced. One of the key components within the SAFE-VR world is the Hazards constructed by oriented-object technology. In SAFE-VR, the Hazards are inactive objects and are not suitable for modelling human behaviours. In [8], a VR based safety training program is presented for instructing the safety procedures and guidelines of underground conveyor belts. However, the proposed training program doesn't involve modelling human risk behaviours. Other existing research [2] also focuses on geometric presentation or real-time dynamics to create immersive experiences. They hardly take into account human-machine-environment related risk factors resulting in underground safety accidents, especially human risk behavioural factors.

Human behaviour modelling and simulation in virtual environments is still one of the most challenging research topics. Many virtual human models have been presented for real-time applications from training/education systems to human computer interfaces and entertainment films/computer games [9]. Each of these application areas requires different properties at different levels such as autonomous behaviour, natural language communication, personality modelling, emotional simulation, adaptation to environmental constraints and user needs.

In order to improve the believability of virtual agents in virtual environments, a number of emotional agent models have also been proposed. Most of these computational models are based on the well-known OCC model. According to OCC theory, emotions are triggered by the subjective appraisal of the consequences of an event on the agent's goals, the actions performed by the agent and the objects in the environment. The perception and the cognitive appraisal of the event determine the type and the intensity of the felt emotions. In the following subsections, we focus on the prominent models that use OCC theory.

EM Architecture used in the OZ project is the first emotion model [10, 11]. The aim of the OZ project is to provide the users with the experience of living in dramatically interesting micro-worlds that include moderately competent emotional agents. EM generates emotions based on the importance level of the goal and the success/failure of goals. A Fuzzy Logic Adaptive Model of Emotions (FLAME) [12] is used to produce emotions and simulate emotional processes. FLAME uses fuzzy logic to represent emotions for mapping events and observations of emotional states. However, the agent in FLAME does not have clearly defined goals. Moreover, personality is also not mentioned. In [13], Greta's emotion is triggered by belief that the probability of completing one of its goals has been modified. Emotion and Adaptation (EMA) [14] models consider both the triggering of virtual humans' emotions and their coping behaviour. For the intensity of emotions, two parameters are used: probability of goal attainment and goal importance. The primary contribution of EMA is to show how decision-theoretic planning techniques can calculate the impact of events on goals and appraisals. EMA is updated with recent developments [15]. The new work also models a naturalistic emotional situation that involves both rapid and slower emotional responses. In [16], a generic emotional agent model, GEmA, is proposed based on an OCC model. GEmA simulates 16 emotions by appraising events and actions with respect to both goals and standards of agents. However, GemA doesn't include personality as well as motivation.

Research in social science and human psychology has shown that personality influences the emotion and behaviour of an individual. For example, people who have optimistic traits are inclined to have positive emotions longer than pessimistic people. In [17], a model of the dynamics of emotions is proposed for non-player characters (NPCs) in a game. The NPC emotion model focuses on the influence of personality on triggering emotions. However, this model doesn't take motivations into account. In [18], an action selection mechanism is presented to simulate human behaviours in realistic and believable virtual humans according to their personality and affective states. The proposed action selection model takes into account a virtual human's beliefs, desires, intentions and the level of intensity of the events. Based on personality and perceived intensity, a fuzzy model is proposed to update the affective states of virtual humans. In [19] a computational model of motivation is presented. This model integrates motivation, emotion, personality, behaviour and stimuli together. However, it only gives a primary outline for the motivation model and is restricted to being tested by a 3D facial animation system [18]. By adopting the OCEAN (Openness, Conscientiousness, Extroversion, Agreeableness, and Neuroticism) Model of personality [20], a hierarchical fuzzy rule-based system is constructed to facilitate the personality and emotion control of the body language of a dynamic story character [21]. There are other typical emotion models and systems reviewed in [22, 23] such as Cathexis, ParleE, FearNot! and Affective Reasoner. These models generally put emphasis on explaining human emotion processes. They have different advantages and disadvantages. However, at this moment, it is not possible to find a universal model that describes how emotions affect behaviours based on personality and the events perceived from the environment [17, 18].

Uncertainty is an important aspect of human behaviour under stress and panic. Traditionally, fuzzy systems are usually used to model uncertain domains. Our research involves developing a fuzzy logic based emotional behaviour model to simulate the internal mental states of a virtual miner. The virtual miner's emotions are based on the OCC model. We implement a subset of the OCC model including Joy, Hope, Relief, Distress, Fear and Disappointment, which focuses on the appraisal of the perceived events and suits the specific application environment in an underground coalmine. In order to approach the personality traits of underground mine workers, we adopt the OCEAN [20] model of personality to enhance emotional simulation for the virtual miner. We stress Conscientiousness, Extroversion and Neuroticism factors' impacts on the motivation update process, the emotion intensity threshold and the emotion decay. Finally, virtual miner's behaviours are constructed based on their motivation needs, emotional states and behaviour selection thresholds to simulate the emotional responses to potential hazards and events in an underground coalmine.

3. Risk behaviour simulation environment based on multi-agent

A multi-agent simulation framework can present a computational methodology to build an artificial environment populated with various agents. Based on agent-oriented methodology [3], agent based virtual environment technology [24–25] and behaviour analysis of an open complex system [26], the virtual coalmine environment for underground risk behaviour simulation is modelled as a three-tier architecture shown as in Figure 1. The proposed framework consists of the Simulation Platform layer, the Multi-agent Environment layer and the Intelligent Man-Machine Interface layer.

Figure 1.

Multi-agent based virtual environment for underground risk behaviour simulation

The Simulation Platform layer provides system running support. This layer is composed of a hardware platform, software development library, a distributed network communication environment, an operation system and a coalmining database. A typical software library includes OpenGL, DirectX and Vega Prime. The coalmining database mainly stores the required geological data, measurement data, hydrological data and accident cases. The geometry and physical rending is also implemented in this layer. In addition, this layer is also responsible for the organization, optimization and management of geometric scenes and the calculation of physical properties such as collision detection and scenario-based event management.

The Multi-agent Environment layer includes entity agents (EntityAgents) and service agents (ServiceAgents). In this layer, entity agents interact with each other and endow the geometric entities with specific behavioural rules. Correspondingly, EntityAgents can be classified as a Device Agent, Miner Agent and Environment Agent. Each of the Device Agents has a geometric shape of a specific device, such as a tramcar, transducer or rescuer. A Device Agent can be constructed according to the dynamics of a specific device. A Miner agent is critical for building the virtual coalmine environment for underground risk behaviour simulation. It should take into account underground miners' physiological and psychological attributes. The Environment Agent mainly describes the risk behaviour of all kinds of underground hazards. It can be implemented using the risk factors and environmental parameters stored in the coalmining database.

In the Multi-agent Environment layer, ServiceAgents offer basic system services and can generally be categorized into three classes: communication service agent (CommAgent), information service agent (InfoAgent) and specific application service agent (AppAgent). CommAgent is responsible for network communication provided by the Simulation Platform layer such as High Level Architecture (HLA) services. InfoAgent provides the necessary semantic information and knowledge for intelligent behaviour in the virtual coalmine environment. AppAgent acts as a service provider in specific applications such as tutor agent and expert agent in safety training.

ServiceAgents may interact not only with EntityAgents, but also with the geometric entities in the virtual environment. InfoAgent is responsible for translating commands from EntityAgents into the state changes of specific entities in geometric and physical scenes. In addition, InfoAgent also translates the state changes of geometric entities into meaningful knowledge and transmits them to EntityAgents for decision-making. Each agent has its own control mechanisms, knowledge base and communication interfaces. Communication protocol defines the basic messaging mechanisms and interaction methods among agents.

The Intelligent Man-Machine Interface layer is the interface with which the user interacts directly with virtual environment via VR multi-modal presentation devices. A typical immersive VR system consists of a data glove with a hand tracker, a Head Mount Display (HMD) with a head tracker, a 3D sound server and a host computer. The hand tracker, the head tracker and the data glove control the head, hand and finger orientations respectively while an HMD presents computer-generated images. Inside a virtual environment, the user can manipulate and interact with graphical objects with a computer-generated virtual hand. The orientation of this virtual hand is controlled by the measured hand and fingers' positions.

In the proposed framework, a multi-agent based simulation environment can fully create the overall behaviours of underground risk accidents. Moreover, this framework is helpful to the system development, maintenance and upgrade. To make things more precise and ambiguity-free, a multi-agent based virtual coalmine environment is formally defined as follows.

Definition 1 The multi-agent based virtual coalmine environment (MBVC) can be described as a three-tuple: MBVC::= <CEnvironment, Agents, EAInteractions>, where CEnvironment is an underground natural physical environment model that represents the global knowledge of coalmine properties, Agents is a multi-agent system and EAInteractions: CEnvironment × Agents is the interaction between agents and the virtual environment.

Definition 2 Agents can be defined as a two-tuple: Agents::= <AGENT,COMMUNICATIONS>, where AGENT is the set of agents, including ServiceAgents and EntityAgents, i.e., AGENT= {Agent = 1, 2… n; (Agenti ∈ ServiceAgents) or (Agenti ∈ EntityAgents)} and COMMUNICATIONS is the set of communications.

Definition 3 COMMUNICATIONS can be represented as: COMMUNICATIONS = {<Sender, Receiver, MsgContent> | (Sender∈ AGENT) and (Receiver ∈ AGENT) and (MsgContent ∈ Communication (L))}, where Sender is the communication sender, Receiver is the communication receiver, MsgContent is the communication content related to the communication protocol and L represents the communication language.

Definition 4 Agent-environment interaction, EAInteractions, can be described with a partially observable Markov Decision Process [25] and be represented as a six-tuple: EAInteractions::= <State, Action, T, R, O, P>, where State is the finite set of environmental states, State = {statei | i=1,2, …,n}, Action is the finite set of agent actions, Action={action_j |j= 1,2, …,m}, T is the state transition function from state_t into state_t+1 when an action, action_t, is taken, T(state_t,action_t,state_t+1), R is the immediate reward function for taking action_t in state_t+1, R(state_t+1,action_t) : State×Action → R, O is the finite set of observations, O(state_t,r_t) and P is the probability of making observation p_t+1 from state_t+1 after having taken action action_t, P(action_t, state_t+1, p_t+1).

Definition 5 Event in the virtual coalmine environment can be described as an eight-tuple: Event ::=<EventID, EventCause, EventObject, EventRegion, EventLevel, StartTime, EndTime, EventEvolution>, where EventID is the event ID number, EventCause is the event cause, EventObject is the lists of agents that are affected by the event, EventRegion is the region affected by the event, EventLevel is the intensity of the event, StartTime is when the event begins, EndTime is when the event ends and may be NULL and EventEvolution is the event dynamics, which can be a process description for simple events or be defined with a finite state machine for complex events.

4. Human risk behaviour modelling

According to the behaviour characteristics of mine workers in underground coalmines, we focus on modelling underground human risk behaviours. Based OCC appraisal theories and fuzzy logic, we propose a fuzzy emotional behaviour model for a virtual miner agent to simulate human responding behaviours to underground potential risk events.

4.1. Virtual miner model

According to the actual coalmine environment, underground mine worker's behavioural characteristics[4–5] and related virtual human models [27–28], a virtual miner agent is organized as a hierarchical model, which incorporates perception, motion control, internal states, behaviour and cognition sub-modules, as shown in Figure 2. The model is reactive and able to rapidly adapt to external unexpected situations and can also plan consistent goal-oriented behaviours.

Figure 2.

Virtual miner agent model

In the Motion Control level, we employ a human animation software package called DI-Guy to parameterize the basic motions. Due to the actual environment in an underground coalmine, the virtual miner's behaviours are mainly reflected with its gestures and postures, including changes in direction, changes in speed, gaze direction, body orientation, etc. When accepting motion commands from the Behaviour module, the virtual miner selects the basic parameterized motion sequences from its motion repertoire to update its gestures and postures.

The Behaviour module is constructed by a set of parameterized basic actions and is controlled by a behaviour selection mechanism. Basic actions are described by finite state machines and/or hierarchical finite state machines. These parameterized basic actions are semantically independent of each other and serve as the building blocks to build complex behaviours.

Formally, the state space of virtual environment was S, S={state_i|i=1,2,…}, the effective action space of the virtual miner was A, A={action_j|j= 1,2,…}. The virtual miner can obtain a set of precepts P, P={perception_k|k=1,2,…} and has a set of Internal states I, I={innerAttr_m|m=1,2,…}. The Perception module can perceive the environment states S and Perception: P → S. The Cognition module generates the current task T for the virtual miner to implement its goal G with task planner Plan:G → T . The Behaviour module can create motion control commands B according to external environment perception P, internal attributes I and domain knowledge K to implement its current task T, Behavior(T): P×I×K → B. Finally, the motion control commands are implemented by a movement mechanism in the Motion Control module, MotionControl: B → A. In each step t, the virtual miner takes actiont and the environment state indefinitely changes from state_t to state_t+1, i.e., state_t+1=Environment (state_t, action_t).

In this paper, we focus on the virtual miner's emotion model, taking into account personality and motivation and further simulate the emotional responses to external events based on OCC theories and fuzzy logic techniques. The concrete implementation of other modules such as Perception, Motion Control and Cognition can be referenced in [29].

4.2. Fuzzy emotional behaviour modelling

Underground human behaviour is uncertain and fuzzy. With the aim of implementing fuzzy emotional behaviour, we apply fuzzy sets to represent emotional states and fuzzy rules to represent the mapping from events to emotions. Moreover, to approach underground mine workers' characteristics, we take into account underground miners' personality traits and the impacts of perceived events on goal attachment.

Firstly, according to the actual underground work environment, personality p of the virtual miner is constant. p is initialized with the level of intensity of each personality scale defined by the OCEAN at time t=.

Definition 6 Personality can be described as a vector p: p=[O, C, E, A, N], where O, C, E, A and N represent Openness, Conscientiousness, Extroversion, Agreeableness and Neuroticism, respectively. The value of O, C, E, A and N is in the interval of [0, 1].

In this paper, we stress the impacts of Conscientiousness, Extroversion and Neuroticism on the intensity threshold of emotions, the emotion's decay and the motivation update process.

The virtual miner's emotional state vector e_t is dynamic over time and initialized to 0 at time t=0. At each time t, e_t represents the intensity of six basic emotions defined by the OCC model.

Definition 7 At time t, the virtual miner's emotional state can be described as vector e_t: e_t=[Jo, Ho, Re, Di, Fe, Da], where Jo, Ho, Re, Di, Fe and Da represent Joy, Hope, Relief, Distress, Fear and Disappointment, respectively. The value of Jo, Ho, Re, Di, Fe and Da is in the interval of [0, 1].

In the OCC model, the intensity of the emotions of Joy, Distress, Relief, Disappointment, Hope and Fear is positively correlated with the degree of desirability/undesirability of the event and to its probability of occurrence. Once a virtual miner has perceived an event, an update process is carried out in order to change its affective states. The new emotional state vector e_t+1 over time t+1 is calculated by taking into account the current perceived event and the goal of the virtual agent as follows:

e_{t + 1} = e_{t} + I_{e} (g o a l, e v e n t),

(1)

where e_t is the intensity of emotion at time t, I_e(goal, event) represents an emotional influence vector that contains a desired change of intensity for each of the six emotions.

To compute the intensities of emotions, the OCC model postulates four global intensity variables (arousal, unexpectedness, proximity and sense of reality) and up to four local variables for each emotion type [6]. While the virtual miner's emotion generation is based on the OCC model, we do not represent the complex intensity system of the OCC model but instead focus on a simple implementation of OCC cognitive processing [30]. Moreover, according to OCC emotional variables, we define fuzzy rules to update emotion states for the virtual miner agent. Following [13] and [17], we take the impact of perceived events on goal attainment and the goal importance into account. These variables are denoted as ProbabilityImpact(goal, event) and Importance(goal), respectively. To obtain this I_e vector, we define a set of fuzzy rules of the form given as follows:

IF ProbabilityImpact(goal,event) IS A_j AND Importance(goal) IS B_k

THEN Intensity(Joy) IS C₁, Intensity(Hope) IS C₂, Intensity(Relief) IS C₃, Intensity(Distress) IS C₄, Intensity(Fear) IS C₅, and Intensity(Disappointment) IS C₆.

where A_j is the impact intensity level (Negative, NoImpact and Positive) of the current perceived event on the probability of goal attainment, B_k is the importance level (NoImportant, SlightlyImportant and ExtremelyImportant) of the goal and C₁,C₂,… and C₆ are the emotional intensity (LowIntensity, MediumIntensity and HighIntensity) to each of the six basic emotions.

Consider a miner aiming to inspect underground hazards to prevent underground safety accidents. An event, such as loose rock falling from the damaged roof may affect the miner's current goal. If the rock falling is not very serious, it can help prevent roof safety accidents. Thus, a slight rock falling event is desirable and has a positive impact on the goal attainment of inspecting potential hazards. Therefore, when some small rocks fall, miners always positively investigate the causes and identify their potential hazards. The fuzzy rule relevant to the situation is as follows:

Rule 1: IF ProbabilityImpact(Inspecting Hazards, Rock Falling) IS Positive AND Importance(Inspecting hazards) IS ExtremelyImportant

THEN Intensity(Joy) IS MediumIntensity, Intensity(Hope) IS HighIntensity, Intensity(Relief) IS LowIntensity, Intensity(Distress) IS LowIntensity, Intensity(Fear) IS MediumIntensity and Intensity(Disappointment) IS LowIntensity.

In this paper, the perceived event's impact on goal attainment and the goal importance are fuzzified using a Gaussian-shaped membership function. Based on related fuzzy rules, we can obtain the fuzzy emotional vector Intensity (e) with the fuzzy inference process. We choose the Mamdani model and Sup-Min composition to compute the matching degree for each rule. The inference formula is given in Equation (2).

μ_{C^{'}} (z) = \underset{x, y}{\lor} [μ_{A^{'}} (x) \land μ_{B^{'}} (y) \land μ_{A} (x) \land μ_{B} (y)] \land μ_{C} (z)

(2)

where x is the impact of the perceived events on goal attainment, y is the goal importance and z is the intensity of emotion. The ∧ operator takes the minimum of the membership functions and the ∨ operator takes the maximum of the membership function. A, A', B, B', C and C' are fuzzy sets. Correspondingly, A and A' are the impact intensity set (Negative, NoImpact, Positive) of the current perceived event on the probability of goal attainment, B and B' are the importance level set (NoImportant, SlightlyImportant, ExtremelyImportant) of the goal and C and C' are the emotional intensity set (LowIntensity, MediumIntensity, HighIntensity).

To calculate the crisp intensity of emotional vector I_e, fuzzy emotional vector Intensity (e) is defuzzified by the Mamdani centroid defuzzification model as:

Z_{M O N} = \frac{\int_{z} μ_{C} (z) z d z}{\int_{z} μ_{C} (z) d z} .

(3)

According to Equation (1), adding I_e to the virtual miner's current emotional state e_t, we can obtain the new emotional states e_t+1.

Furthermore, we define an intensity threshold for each emotion according to the character's personality [13, 17]. The intensity threshold, denoted as α, controls the activation of the emotion type. Emotion types with an intensity level lower than α are considered as moods. The impact of mood on the virtual miner's emotions and behaviours is out of the scope of this paper and will not be presented here. On the other hand, the emotion with the highest intensity inhibits other emotions. For example, an emotion like sadness will tend to inhibit joy if sadness is more intense than joy. Moreover, we give a slight preference to negative emotions since they often dominate in situations where opposite emotions are triggered with nearly equal intensities [12]. Only the most dominant emotion whose computed intensity exceeds the pre-defined threshold α does have an impact on the agent behaviours. Therefore, when perceiving an event, the virtual miner can obtain its current most predominant emotion state based on the fuzzy rules and the emotion filter mechanism.

The intensity of emotions will naturally decay through time once the triggering event has disappeared. Using Picard's decay function [31], at any time t the value for the intensity of an emotion is given as:

e_{t} = e_{t - 1} e^{- d (t - t_{0})},

(4)

where e_t and e_t-1 is the emotion intensity at time t and t-1, respectively, t₀ is the emotion triggering time and d is the decay rate, which determines how fast the intensity of the particular emotion will decrease over time. Parameter d is related to the emotion type and the agent's personality straits. Generally, negative emotions tend to decay slower than positive emotions [12]. In our current implementation, each emotion will be attached to a specific d value according to the agent's personality straits. An extrovert individual feels positive emotions with a lower decay rate than neutral or introvert personalities. However, extroversion has no impact on negative emotions. A neurotic individual feels negative emotions with a higher decay rate, without any impact on positive emotions.

On the other hand, the virtual agent's motivation tends to interrupt the cognitive process to satisfy a higher internal need. In order to create more believable behaviours in virtual entertainment, a virtual character should also select its actions taking its motivational states into consideration [19]. In each simulation step, the intensity of motivational needs will be impacted by internal physiological parameters and external environmental factors [32]. Internal physiological parameters include behaviour type, behaviour duration time, motivation duration time and so on. For instance, the intensity of run is higher than that of walk. External environment factors include hazards, safety warning signs, water, other agents, etc. In this paper, the virtual miner's motivational states vector m_t is dynamic over time t and is initialized to 0 at time t=0. At each time t, m_t represents the intensity of fatigue, hunger and thirst.

Definition 8 At time t, motivation can be defined as vector m_t: m_t=[Fa, Hu, Th], where Fa, Hu and Th represent Fatigue, Hunger and Thirst, respectively. The Value of Fa, Hu and Th is in the interval of [0, 1].

The virtual miner's internal needs are updated according to Equation (5).

m_{i} (t) = \min {m_{i} (t - 1) δ_{i} + β_{i} + \sum_{k} γ_{k i} (t), 1},

(5)

where m_i(t) is the need intensity value of the ith motivation variable at time t, δ_i is the decay rate of the ith motivation, β _i is the growth rate of the ith motivation and γ_ki (t) is the impact of the kth external environment factor on the ith motivation need at time t, γ_ki ∈ [-1, 1]. The value of δ _i and β_i is set according to the internal physiological parameters and the personality traits. Generally, the Conscientiousness parameter in OCEAN influences how soon goals will be abandoned and new goals are adopted [33]. A conscientious personality will try and achieve all goals. However, neurotic people will in general experience negative thoughts, which can lead to less ambition. We also define activation threshold m for each motivation variable. Once these states reach an activation threshold, they send a signal to the cognitive process indicating a specific desire.

Finally, combined with internal attributes such as emotion, motivation and perceived external situations, an action selection mechanism initiates appropriate behaviours to satisfy these desires. In case more than one desire awaits fulfilment, the most important desire ranked by the action selection mechanism receives the highest priority. In this paper, the internal variable with the highest intensity will be dominant, motivation inhibits emotion with equal intensity and fear inhibits motivation variables when the fear level is higher than the motivation level. Once a desire is fulfilled, the value of the internal variable begins to change back asymptotically to its nominal value according to Equation (4).

In the virtual coalmine environment we classify miners by their job categories. Each of the job categories of the miner has an associated action selection mechanism with behaviour-triggering combinations of internal state thresholds and situation patterns set accordingly. For a safety inspector, when the current site must be inspected (such as spotting a roof support, inspecting the equipment status and hitting sidewall conditions), the virtual miner will firstly complete these tasks but when the fear level is dominant, the virtual miner will escape from the dangerous site, when the miner is tired, he will find a place to rest until the value of his Fatigue value returns to normal levels and when he is happy because of identifying unexpected hazards, the virtual miner will powerfully walk to the next goal place.

5. Case study and implementation

To test the proposed simulation environment and related behaviour models, we have reconstructed some accident cases in an underground coalmine [4], which can train miners' decision-making ability when facing occasional or emergency situations in an underground mine environment. We have collected the typical coalmine accident cases from the last 20 years in China. We employed Multigen Creator to construct the geometric modes of the virtual mine, the virtual miner, a safety lamp, a self-rescuer, etc. The other simulating tools and developing environments include VC++, DI-Guy SDK, Vega Prime and Open GL.

In the first case, the virtual miner is a safety inspector who aims to spot potential hazards in the underground coalmine to prevent safety accidents. The virtual miner will first enter into the underground coalmine from the coalmine entrance. Then, he will walk through the shaft station, the main roadway, the transformer substation, the working face, etc. In this process, some typical hazards, such as loose rocks, an unsupported roof in a work area, excessive water, a damaged support and shorts on electrical cables may appear. The virtual miner may also possibly engage in some risk behaviours, which violate underground safety rules and regulations, because of his motivation needs. When identifying a hazard, the virtual miner will inspect it with nonverbal postures/gestures and some basic actions such as gaze, aim, hit and hammer according to his current emotional states and internal motivation needs.

In this case, the virtual miner is a very neurotic person with medium conscientiousness, who is anxious, nervous and prone to depression. Therefore, the virtual miner's personality p is initialized as p=[0, 0.45, 0.3, 0, 0.7] according to Definition 6. Currently, we can't take into account the Openness and Agreeableness of the OCEAN model.

At each simulating step, the virtual miner's motivational needs are updated based on Equation (5) (introduced in Section 4.2). According to our tests, the value of the decay rate δ_i is in the interval of [0, 0.05] and the value of the growth rate of β_i is in the interval of [0, 0.1]. As the safety-inspection tasks are fulfilled, the virtual miner's fatigue value becomes very high. Fatigue becomes the most dominant motivation state.

On the other hand, the virtual miner's emotion vector e_t is updated based on Equation (1). In the OCC model, Relief is generated when an undesirable event failed to occur. In this case, the intensity of Relief increases more quickly when no risk events occur during safety inspection. However, according to the action selection mechanism, the virtual miner's motivation state, Fatigue, is the most predominant. As a result, the virtual miner's risk awareness to potential dangers decreases because of his need to take a rest, which is motivated by the Fatigue state.

Figure 3 shows some screenshots of the simulation case. The right frame shows the changing intensity of emotions and motivations in the simulation process. The checked states of emotions and motivations indicate the most dominate internal attributes, which influence the virtual miner's behaviours. In Figure 3 (a), the motivation state, Fatigue, is the most predominate. The virtual miner tiredly walks through a tunnel without spotting the loose rocks on the damaged roof shown in the red frame.

Figure 3.

Underground hazard spotting case. (a) Tiredly walking through a tunnel, (b) avoiding a rock falling event

The roof state is controlled by a roof agent, which belongs to an Environment Agent. The interaction between the roof agent and the virtual miner is implemented by the communication mechanism introduced in Definition 3. In this case, when the roof strength value is less than the pre-set value, a rock falling event on the roof will occur. Then, the roof agent instantly messages this event to virtual miners in its influence region.

When perceiving the rock fall event, the virtual miner carries out an appraisal using fuzzy rules. In this case, a severe rock fall generally indicates that a more severe roof falling accident is likely to occur. Therefore, a severe rock falling event is generally undesirable. It can stop the virtual miner from inspecting potential hazards and even threaten the miner's life. The fuzzy rule relevant to the event is as follows:

Rule 2: IF ProbabilityImpact(Inspecting Hazards, Rock Falling) IS Negative AND Importance (Inspecting hazards) IS ExtremelyImportant

THEN Intensity(Joy) IS LowIntensity, Intensity(Hope) IS LowIntensity, Intensity(Relief) IS MediumIntensity, Intensity(Distress) IS MediumIntensity, Intensity(Fear) IS HighIntensity and Intensity(Disappointment) IS MediumIntensity.

According to the fuzzy inference process introduced in Section 4.2 and Rule 2, the virtual miner believes a more severe roof falling accident will occur and his safety-inspection goal likely fails. According to Equation (1), emotional influence vector I_e can be obtained based on Equation (2) and Equation (3). In this case, the emotion state, Fear, becomes very high and is the most dominant. The motivation state, Fatigue, is temporarily inhibited. Thus, the virtual miner escapes in a panic from the site of the falling rocks to pursue his safety needs. In Figure 3(b), facing an unexpected event of a severe rocks fall, the virtual miner runs away from the falling rocks.

The context of the second simulation case is a typical coalmine locomotive transport accident. In underground coalmines, all mine workers should be aware of the job-specific hazards in their workplace. Moreover, all of them should be able to identify all the generic hazards or risk behaviours violating safety rules and regulations outside of their workplace.

In this case, the virtual miner's motivational needs are updated with Equation (5) (Section 4.2). When finishing his tasks, the virtual miner's Fatigue value increases. Thus, the virtual miner's awareness to hazards will greatly decrease. Driven by his desire to go out of the underground mine, the virtual miner hurries to tiredly pass through an intersection of roadway. As a result, the virtual miner is seriously hit by an approaching locomotive. Figure 4 shows some screenshots of the simulated accident. Figure 4 (a) is the scene of the coalmine entrance. In Figure 4 (b), the virtual miner is walking in a main roadway. In Figure 4 (c), the virtual miner is tiredly going through the intersection of the transportation roadways. He violates the related rules of “when a mine car is going through, the human must stop.” Figure 4 (d) is the scene where the virtual miner was seriously injured by a locomotive. The case explores the causal relationships among psychological motivation needs, violating behaviours and risk accidents. The reconstructed case can be used for miner training to improve the ability of hazard awareness in underground working environments.

Figure 4.

Typical locomotive transport accident simulation. (a) the scene of the coalmine entrance, (b) walking in the main roadway, (c) hurrying to pass through an intersection of the transportation roadway, (d) injured by a locomotive

6. Evaluation of believability

One challenge in designing a virtual environment is to validate it. Due to safety and ethical issues, it is difficult to fully validate a coalmine virtual environment, which really requires multiple real-life studies with people in underground risk situations. To evaluate the believability of the proposed system and behaviour models, we chose a user assessment method. Users evaluated the system and then feedback was gathered via a questionnaire. Previous studies have fully proved this method is advantageous and effective [12, 16, 30, 32, 34].

Participants in the evaluation were recruited by email from our lab. We selected 20 students. The ages of the subjects were in the range of 22-25 years old. The male-to-female ratio was 3:1. All the participants were first year graduate students. Most of them received their bachelor degree in computer science, automation science, or system engineering. While the results depend on the ability of our subjects to accurately judge how realistic the behaviours of the virtual miners are, we feel that these students have the necessary common knowledge to perform this task according to our experiment introduction and instruction.

Before starting the evaluation, participants were given a twenty-minute introduction to the system and the evaluation method. In this introduction, they were first notified that their responses would be used to further enhance the system. This way, users were encouraged to give constructive criticism of the system, while avoiding assessing the system in an unduly positive fashion because they are either trying to be nice or are impressed by the interface. In addition, the participants were given an introduction to the underground mining environment. Emphasis was put on the typical hazards in underground coalmine and the typical risk behaviours resulting from the violation of coalmine safety rules and regulations.

Then participants were handed instruction sheets, which walked them through different scenarios and showed them the different aspects of the virtual miner's behaviours. Eventually, they were asked to answer the following question on five levels: very low, low, medium, high and very high.

How well does the virtual miner show its behaviours?

The question was also quantified from one (very low) to five (very high) in order to further verify the evaluation results. In order to ensure the anonymity of the response, the participants were also asked not to write their names or mark the papers in any way that might be used to identify them at a later time period.

In the evaluation, participants worked with the system in two modes: first without the fuzzy emotional behaviour model (i.e., Random-Mode) and second with the fuzzy emotional behaviour model (i.e., Emotion-Mode). In Random-Mode, the virtual miner's behaviour is chosen at random when facing emerging events.

The questionnaires were collected and analysed for 20 subjects. In addition, participants' interpretation of the evaluation scores enable us to further calibrate the behaviour model. Figure 5 shows the average satisfaction of users in two modes. As Figure 5 shows, in Random-Mode, the satisfaction of users is 55% medium and 5% high. In Emotion-Mode, the satisfaction of users is 25% medium, 45% high and 15% very high. The introduction of the fuzzy emotional behaviour model conveys a more realistic or convincing behaviour to the users since the rating is 45% high, which improves this measure by 40% on average. On the other hand, there is 5% of users whose satisfaction is high in Random-Mode. This shows the selected random behaviour at that time was consistent with the virtual scene.

Figure 5.

The comparison of the risk behaviour of virtual miner without the emotion model (Random-Model) and with the emotion model (Emotion-Model)

To further compare these results, we applied the difference of the scores derived by subtracting the Random-Model's score from the Emotion-Model's. In the evaluation, the mean score for the Emotion-Model is 3.55. The mean score for the Random-Model is 2.5. The data for the differences in believability scores is presented in histogram form in Figure 6. This histogram shows the differences in how the Emotion-Model and the Random-Model are scored on a scale from one (very low) to five (very high). As Figure 6 shows, 15 of the 20 participants felt the Emotion-Model was more believable, two felt they were evenly believable and three felt the Random-Model was more believable. Also, nine of the 20 participants score the Emotion-Model one point higher than the Random-Model on the one-five scale.

Figure 6.

The comparison of the difference of the believability scores derived by subtracting the Random-Model's believability scores from the Emotion-Model's

In order to test the statistical significance of these results more rigorously, we applied a t-test [30] to the difference of the scores that each user gave the Emotion-Model and the Random-Model. A t-test is used to make statistical claims about the actual mean of some population given the results of some sample of the population. The t-test is used in this case because it does not require the standard deviation of the larger population to be known, also it is reasonably robust in cases where the distribution is not normal as long as the sample size is at least 15, the distribution is not significantly skewed and there are no strong outliers.

In this test, the sample size is 20. For each user, we determine the difference in believability scores given to the Emotion-Model and the Random-Model. If each participant gave them the same scores, then the mean of the distribution would be zero. The parameters and calculated values for the t-test are listed in Table 1.

Table 1.

Results of the mean t-test of the difference in believability scores given to Emotion-Model and Random-Model

n	Mean	SD	SE	T value	t _0.01(19)
20	1.05	1.43	0.32	3.28	2.86

As shown in Table 1, this actual sample has a mean of 1.05, standard deviation (SD) of 1.43 and mean standard error (SE) of 0.32. The t value of 3.28 is greater than t_{0.01 (19)} of 2.86. This data confirms the hypothesis that the mean of the population is greater than zero with 99% confidence. Therefore, we can show with high probability that, given this sample, if we were to sample the entire population, the Emotion-Model's mean score for believability would be greater than the Random-Model's. In other words, the behaviours of the Emotion-Model are more believable than that of the Random-Model.

7. Conclusion and future work

In this paper, a multi-agent based virtual coalmine environment framework is developed to simulate underground coalmine risk behaviours. The proposed approach can not only avoid the disadvantages of traditional experiences based analysis of coalmine risk accidents, but also build a systematic methodology for risk behaviour simulation of the virtual coalmine. In addition, a fuzzy emotional behaviour model is proposed to simulate the underground human risk behaviours, which maps the impact of perceived events on goal attainment and the goal importance to create more believable emotion responses to underground unexpected events. To capture the underground mine workers' behaviour characteristics, we take into account the impacts of underground mine workers' personality traits on the intensity threshold of emotions, the emotions' decay and the motivation update process.

Finally, we introduce the implementation of the hazard spotting case and the typical locomotive transport accident. To further evaluate the believability of the proposed system and behaviour models, we chose a user assessment method. In addition, we applied a t-test to the difference in the scores each user gave the Emotion-Model and the Random-Model. Experimental results show the proposed model can create a more believable virtual coalmine environment to simulate emotional behaviours, which promises to assist in improving underground miners' risk awareness and further training miners' decision-making abilities when facing occasional or emergency situations in underground coalmines.

There exist many factors that influence the behaviours of an individual, which make it impossible to predict with accuracy the actions that a person performs in the face of certain events [16]. In the OCC model, emotions are triggered by appraisal of the events, objects and other agents. This paper has implemented event appraisal based on OCC theories and fuzzy logic. In our next work, we will enhance the emotion model, taking into account other factors' influences on the affective state of the virtual miner.

Validation by capturing data from real-life underground emergency situations is very difficult because of safety and ethical issues. In this paper, we evaluate the believability of risk behaviours of a virtual miner to a limited extent by a user assessment method. Behavioural data used to construct a human risk behaviour model are based on the underground risk factors that resulted in coalmine accidents. In order to calibrate the behaviour model, we can further investigate all kinds of risk factors that have resulted in accidents, especially mine workers' physical and psychological parameters, to obtain more accurate behavioural rules. In addition, some environmental parameters that may affect mine workers' physiology and psychology, such as gas density, environment temperature and air pressure, can be collected to improve the behaviour model. We will take these factors into account in our next work. Moreover, we also plan to perform experiments in more complex situations with field users in coalmines and to further validate the system and modes.

Footnotes

8. Acknowledgements

This work is supported by the National Natural Science Foundation of China (50804061, 61102145), the Research Project of Chongqing Municipal Education Commission (KJ130522) and the Natural Science Foundation Project of CQ CSTC (CSTC, 2009BB2281, cstc2013jcyjA40042, cstc2013jcyjA40014)

References

van Wyk

de Villiers

(2009) Virtual Reality Training Applications for the Mining Industry. Proc. of AFRIGRAPH Conference. pp. 53–63.

Mallett

Unger

(2007) Virtual Reality in Mine Training. Society for Mining, Metallurgy, and Exploration, Inc. 2: 1–4.

Russell

Norvig

(2009) Artificial Intelligence: A Modern Approach, 3rd Edition, USA: Prentice Hall.

Liu

J.M

B.T

(2011) Case Analysis on Coalmine Typical Accidents. China, Coal Industry Press.

Brnich

M.J

Kowalski-Trakofler

K.M

(2010) Underground Coal Mine Disasters 1900- 2010: Events, Responses, and a Look to the Future. Extracting the Science: A Century of Mining Research. CO: Society of Mining, Metallurgy, and Exploration. pp. 363–372.

Ortony

Clore

G.L

Collins

(1998) The Cognitive Structure of Emotions. Cambridge: Cambridge University Press.

Damian

Noond

Burton

(2002) Reconstructing Events: Simulating Accidents using Virtual Reality. Proc. of the 30th Applications of Computers and Operations Research in Mineral Industry. pp. 25–27.

Lucas

Thabet

(2007) Benchmarking User Performance by Using Virtual Reality for Task-Based Training. Proc. of 7th International Conference on Construction Applications of Virtual Reality. pp. 70–79.

Kasap

Magnenat-Thalmann

(2008) Intelligent Virtual Humans with Autonomy and Personality: State-of-the-Art. Studies in Computational Intelligence. 140: 43–84.

10.

Bates

Loyall

B.A

Reilly

(1994) An Architecture for Action, Emotion and Social Behavior. Proc. of MAAMAW Conference. pp. 55–68.

11.

Reilly

S.N

(1997) A Methodology for Building Believable Social Agents. Proc. of IAA conference. pp. 114–121.

12.

EI-Nasr

M.S

Yen

Ioerger

T.R

(2000) Flame: Fuzzy Logic Adaptive Model of Emotions. Autonomous Agents and Multi-Agent Systems. 3 (3): 219–257.

13.

de Rosis

Pelachaud

Poggi

Carofiglio

De Carolis

(2003) From Greta's Mind to her Face: Modeling the Dynamics of Affective States in a Conversational Embodied Agent. International Journal of Human-Computer Studies. 59: 81–118.

14.

Gratch

Marsella

S.C

(2004) A Domain-Independent Framework for Modeling Emotion. Cognitive Systems Research. 5 (4): 269–306.

15.

Marsella

S.C

Gratch

(2009) EMA: A Process Model of Appraisal Dynamics. Cognitive Systems Research. 10 (1): 70–90.

16.

Kazemifard

Ghasem-Aghaeea

Ören

T.I

(2011) Design and Implementation of GEmA: A Generic Emotional Agent. Expert Systems with Applications. 38 (3): 2640–2652.

17.

Magalie

Nicolas

Vincent

(2009) Simulation of the Dynamics of Nonplayer Characters Emotions and Social Relations in Games. IEEE Transactions on Computational Intelligence and AI in Games. 1 (4): 281–297.

18.

Orozco

Ramos

Thalmann

(2011) An Action Selection Process to Simulate the Human Behavior in Virtual Humans with Real Personality. Vis Comput. 27: 275–285.

19.

Liu

Y.S

(2008) A Motivation Model for Virtual Characters. Proc. of MLC Conference. pp. 2712–2717.

20.

McCrae

R.R

John

O.P

(1992) An Introduction to the Five Factor Model and Its Applications. Personality. 60: 175–215.

21.

W.P

Pham

Wardhani

(2009) Personality and Emotion-Based High-Level Control of Affective Story Characters. IEEE Transactions on Visualization and Computer Graphics. 13 (2): 281–293.

22.

Vinayagamoorthy

Gillies

Steed

Tanguy

Pan

Loscos

Slater

(2006) Building Expression into Virtual Characters. Eurographics Conference State of the Art Reports.

23.

Marsella

S.C

Pynadath

D.V

(2010) Modeling Appraisal in Theory of Mind Reasoning. Autonomous Agents and Multi-Agent System. 20 (1): 14–31.

24.

H.C

Wang

Y.L

Cai

L.Q

Wang

Z.H

(2012) Orange Tree Simulation under Heterogeneous Environment Using Agent-based Model ORASIM. Simulation Modelling Practice and Theory. 23: 19–35.

25.

Lugrin

J.L

Cavazza

Palmer

Crooks

(2006) Artificial Intelligence Mediated Interaction in Virtual Reality Art. IEEE Intelligent Systems. 21 (5): 54–62.

26.

Cao

L.B

Dai

R.W

(2008) Open Complex Intelligent System: Fundamentals, Concepts, Analysis, Design and Implementation. China, Posts and Telecoms Press.

27.

Shao

Terzopoulos

(2007) Autonomous Pedestrians. Graphical Models. 69 (5–6): 246–274.

28.

Silverman

B.G

Bharathy

O'Brien

Cornwell

(2006) Human Behavior Models for Agents in Simulators and Games: Part II: Gamebot engineering with PMFServ. Presence. 15 (2): 163–185.

29.

Cai

L.Q

Mei

Luo

Z.Y

(2008) Behavior Modeling and Motion Control of Intelligent Virtual Human Agent. Journal of Huazhong University of Science and Technology (Natural Science Edition). 36: 4–7+22.

30.

Reilly

S.N

(1996) Believable Social and Emotional Agents. Ph.D Thesis. Carnegie Melon University.

31.

Picard

R.W

(1998) Affective Computing, Cambridge, MIT Press.

32.

Yang

H.W

Pan

Z.G

Liu

G.D

(2008) A Comprehensive Computational Model of Emotions. Journal of Computer Research and Development. 45 (4): 579–587.

33.

Egges

Kshirsagar

Magnenat-Thalmann

(2004) Generic Personality and Emotion Simulation for Conversational Agents. Comput. Animat. Virtual Worlds. 15 (1): 1–13.

34.

David

Zoraida

(2013) An Architecture to Develop Multimodal Educative Applications with Chatbots, International Journal of Advanced Robotic Systems. 10: 1–15.