Fostering human–robot cooperative architectures for search and rescue missions in urban fires

Abstract

Urban fires are probably the most frequent catastrophic incidents in urban areas, requiring a prompt response due to life endangerment in highly populated zones and the high risk of fire propagation to buildings and parked cars in the vicinity. Robot assistance has been identified as a valuable resource for such urban search and rescue (USAR) scenarios by taking advantage of robots’ expendability. However, it is still unclear or not quantified how advantageous such human–robot cooperation can be to the final outcome of firefighting operations and other USAR missions. This article reports research in this context by modeling key features of a firefighting mission in response to an urban fire in a large basement garage. Besides building a behavioral model of human firefighting teams based on interviews with a Portuguese Fire Department, and assessing their performance under different circumstances, this work studies the addition of robotic teams in cooperation with human firefighters to overcome communication issues and improve situation awareness. The results obtained highlight the importance of such human–robot partnership for a more effective response to an urban fire and mitigation of life endangerment of first responders and victims.

Keywords

Search and rescue simulation modeling multi-robot systems human–robot cooperation

1. Introduction

Human rescuers constantly risk their lives when responding to urban disasters, such as collapses, fires, floods, and earthquakes. In addition, they usually face a shortage of agents because these events are intrinsically of unpredictable nature.

While research in multi-robot systems (MRSs) has been highly in focus for the past two decades, it is also true that there is a gap in studying the potential of cooperative interaction between teams of mobile robots and teams of humans.¹ A search and rescue (SAR) scenario is precisely one of those situations where one would expect robots to assist a team of human agents, by alleviating human resources and improving effectiveness, necessary due to human fatigue during crisis operations, taking advantage of robots’ expendability. Having this in mind, the CHOPIN (Cooperation between Human and rObotic teams in catastroPhic INcidents) project¹ aims at exploiting this human–robot symbiosis in the development of human rescuers’ support systems for SAR missions developed in urban catastrophic incidents, an application domain with an unquestionable beneficial impact for society. In this kind of critical scenario, mobile robots can provide human responders with improved situation awareness and assist them in the course of the mission so that it can be accomplished more effectively while reducing the risk exposure of humans.

The problem in hand involves numerous challenging scientific issues. Examples of such issues are as follows: the ability of the robot to explore unknown environments and build a map in rough conditions; detect victims using multi sensorial systems (e.g., image, audio, environmental features); receive input or being fully controlled by human operators; provide important environmental information to the rescuing teams (e.g., temperature status and fire outbursts localization) and maintain a mobile ad hoc network (MANET) to communicate or assist the communication of human first responders, among several other challenges. Therefore, augmenting the team of rescuers with robotic agents has the potential to increase the safety of human rescuers and victims, as well as to attain a faster, more precise, and cost-effective response in SAR applications.²

The top priority of any urban rescue team is to quickly locate and save human lives in imminent danger. It is essential to provide first responders with adequate tools and detailed information of the whereabouts of the rescue mission. Moved by such motivation, many research teams have proposed advanced field technology to assist human agents in the “theater of operations.” It is clear that assistive technology, such as human–machine interface (HMI) ground exploration robots and advances in communication systems, can make a difference in field operations. However, organizations, such as those of firefighters and paramedics, follow strict institutional protocols, which means that they reluctantly accept the use of extra devices or equipment, even when they comply with regulations; in addition, they waive any overhead, due to severe timing restrictions when saving victims and extinguishing fires, while carrying heavy firefighting gear.

1.1. Statement of contributions

In this work, we study the behavior of teams of firefighters undertaking an urban fire in a large basement garage by realistically simulating all relevant physics and behaviors. Also, the benefit of using mobile robot teams to assist and cooperate with teams of firefighters is assessed. To that end, we propose a simulation framework that integrates several mathematical models, namely: (a) fire spreading; (b) victims’ behavior; (c) firefighter behavior; (d) radio frequency (RF) communication; and (e) voice propagation. It is noteworthy that the proposed approach does not encompass all possible constraints encountered by field agents in an urban fire situation, such as dynamic physical constraints that can impair mobility, such as burning floors and objects falling in the way or collapsing underneath. This assumption holds by considering that both humans and robots possess similar terrain traversability. Hence, while integrating dynamic obstacles, adopting the same approach carried out for fire and other agents would increase the realism of the simulator, which would equally affect the performance of both humans and robots, maintaining, to some extent, a similar gap between both scenarios.

The models are implemented in the MRS toolbox Multi-Robot Simulator (MRSim) based on evidence extracted from the literature. MRSim follows a distributed agent-based principle where, in each simulation step, the controlling algorithms of all agents as well as all herein proposed models are updated. Intensive simulations are conducted under a set of different configurations to draw initial conclusions based on quantitative data. We then model the cooperation between human and robotic teams, by adding a team of four mobile robots to the firefighting team and assess the global performance under different circumstances and the added value of expanding the team with robotic agents.

This article is organized as follows. A brief revision of the literature concerning the use of robots to assist first responders on urban search and rescue (USAR) scenarios is presented in Section 2. This is followed by the description and mathematical modeling of the CHOPIN use case, which consists of an urban fire in a large basement garage (Section 3). Afterwards, some robotic strategies to support firefighting operations are proposed in Section 4, and several numerical simulation experiments are presented to evaluate the success of firefighting operations using human teams and mixed human–robot teams in Section 5. The main conclusions are outlined in Section 6.

2. Related work

In this section, we conduct a literature review on systems developed to assist first responders in SAR scenarios, with particular interest in the use of MRSs and human–robot interaction in fire-related indoor operations.

Aiming to support firefighting navigation in indoor scenarios, Gandi et al.³ proposed a location tracking system using Bluetooth and radio frequency identification (RFID) technology, which assists firefighters in finding the nearest exit location and presenting the responsible commander with their current locations superimposed on a map of the building floor. The system makes use of augmented reality running in android-based smart phones to display the direction of near exits and relative distances to other teammates. Strong assumptions are involved in this work, such as the a priori existence of a blueprint and passive RFID tags on the walls, as well as Bluetooth tracks located in smoke detectors on the ceiling. Similarly, in the REFIRE project,⁴ a reliable localization system is discussed and RFID tags are deployed in the environment to assist the localization of firefighters in Global Positioning System (GPS)-denied areas. According to the authors, “even though predeployed systems work well under favorable conditions, they are unreliable in a disaster and might not even exist in many locations due to the changes occurred during emergencies.” In fact, feedback reported to us by Portuguese firefighters indicates that in most real-world situations, when called to action, they do not possess a representation of the environment before the mission starts.

In the GUARDIANS EU FP6, a swarm of robots is deployed in a large warehouse filled with smoke, toxic gases, and inflammable materials to support a team of human firefighters.⁵ Aiming at a minimal addition of mental and/or communication load to the agents involved, a minimalist light array visor solution embedded in the firefighter’s helmet has been proposed for human–swarm interactions. Based on sensed data, the swarm recommends a direction to proceed and calculates the firefighters’ pose and direction, presenting this information through a light array. Furthermore, a formation algorithm for a swarm of robots that follow human agents has also been developed,⁶ which does not require explicit communication. Swarm agents react to human actions, while providing guidance information to firefighters.

The NIFTi⁷ and TRADR EU projects⁸ focus mainly on developing systems for human–robot teaming in USAR. Having a real-life tunnel accident as a use case and communicating through a multi-modal team interface and spoken dialogue, the authors describe a complex socio-technical system to promote joint exploration of human–robot teams in accident sites. Working closely with rescue services to define requirements and evaluate the proposed system, these projects follow a user-centric design approach, mimicking human understanding and operational procedure. Situational awareness plays a key role in the system to join robot and human sensing and, similar to other projects, investment in human–robot interaction is conducted to facilitate the process. Real-world results show that there is still a long way to go. The authors highlight that the robot’s autonomous behavior must be transparent to promote collaboration.

While several research works ultimately seek a high degree of robot autonomy in real-world rescue missions, according to Murphy et al.,⁹“only a few [proposed robots] have actually participated in a rescue or been allowed to operate on site after a disaster for testing purposes; all of these have been teleoperated.” This is still the case nowadays due to the exceeding demands of SAR missions, restrictive timing constraints, and lack of trust associated with effective robotic performance and full robot autonomy, which may raise legal issues and suspicions about the risk involved for human lives. Recent research work has been conducted on remote interfacing and easy control of robots.¹⁰ Taking one step further, in Doroodgar et al.,¹¹ a semi-autonomous robot architecture in rescue tasks is proposed. The USAR robot is able to learn and make decisions whether autonomous or human control should be used to perform the tasks in a quicker and more efficient manner, considering the minimization of stress and burden placed on the operator.

The literature review conducted above shows that the focus of recent work is not especially put on understanding the behavior of human agents in rescue missions, but more on means to assist them. Yet, it is our belief that there is a gap in the literature, since modeling rescuers’ behavior is vital prior to extending the team with robotic agents, to enable the analysis of the potential benefits of doing so and measure the added value of incorporating the latter. Hence, in this work, we propose a framework for simulating an urban fire scenario in realistic conditions, including the interaction of a large number of agents, such as victims, firefighters, and robots. This falls under the hypothesis that robots can provide invaluable support in scenarios where human responders are highly hampered due to lack of visibility. Robots are capable of performing cooperative simultaneous location and mapping (SLAM) using non-optical ranging sensors to assist the team of firefighters. Robots can create and maintain a communication network for first responders, overcoming the limitations of RF communication in the presence of obstacles. Furthermore, robots can additionally contribute to the firefighting operations by acquiring contextual information, informing the first responders on environmental context (e.g., fire outbreaks), mission-related context (e.g., victims’ positions), and agent-based context (e.g., level of the self-contained breathing apparatus). In the scope of the CHOPIN project, the authors previously presented a distributed collective robotic architecture in SAR scenarios.¹ The problem was divided into two phases matching the most critical phases of firefighting operations: reconnaissance and rescuing. Both robotic strategies inherent to each phase will be briefly described in the following sections.

Having this in mind, in the next section we describe the CHOPIN use case and propose a mathematical formulation of the main phenomena involved, gaining inspiration from existing models in the literature and several interviews with members of a Portuguese Fire Department. This enables us to gracefully integrate all models in MRSim, a simulation framework that we openly provide to the community, and draw important conclusions derived from the results extracted in this work by assessing the impact of adding robotic teams in cooperation with human firefighters to overcome communication issues and improve situation awareness.

3. Modeling the CHOPIN use case

This section describes the CHOPIN use case, highlighting the most relevant phenomena that one should consider in the design of human–robot firefighting operations. As previously mentioned, the CHOPIN project (http://chopin.isr.uc.pt) proposes to study such cooperative architectures under a fire outbreak taking place in a large basement garage. We consider an urban fire in a large basement garage attended by people and containing inflammable materials, like a garage of a shopping mall with several cars, drivers, and people passing by. This is a particularly challenging urban fire due to the confined nature of the environment. As the fire evolves, the space becomes rapidly filled with smoke, with very low visibility and with an unbreathable and toxic atmosphere, which is dangerous for both victims and first responders. It is noteworthy that in this article we will only focus on the most critical phases of the firefighting operation: reconnaissance and rescue.

The following sections present the modeling of the most relevant features and behaviors that may be expected under such conditions. The base models were retrieved from a careful analysis of the literature and adapted accordingly to the information acquired through several regular meetings at the Coimbra Fire Department (BSC; Portugal). The models were implemented using MRSim, a MATLAB toolbox previously developed by the authors, which may be found online (www.mathworks.com/matlabcentral/fileexchange/38409-mrsim-multi-robot-simulator-v1-0). MRSim is an evolution of the autonomous mobile robotics toolbox SIMROBOT (SIMulated ROBOTs), completely remodeled for the newer MATLAB versions and including new features, such as mapping and inter-robot communication. MRSim allows adding a monochromatic bitmap as a planar scenario, changing its properties (e.g., obstacles, size, among others), adding features of each robotic algorithm (e.g., robotic population, maximum communication range, among others), and editing the robot model (e.g., maximum velocity, type of sensors, among others). This simulator was first evaluated in the context of the CHOPIN project.¹

3.1. Fire spreading

Ohgai et al.¹² presented a simple, yet proficient, method to model fire spreading and proved that their simulation results may accurately reproduce real fire spreading records.

The approach uses cellular automata (CA) in which the fire spreading is discretized into neighbor cells based on the idea of the limit of the distance that the fire can spread. In other words, the fire spreading probability of cell ij in the neighborhood of cell kl is given by the fire spreading judgment index F_ij, as follows:

F_{ij} = α (S_{ij} \cdot P_{ij}) \cdot W_{ij}^{β} \cdot p (t_{ckl})

(1)

wherein 0 < α≤ 1, and α is a coefficient to tune the degree of spreading slowdown. Although this coefficient has not been deeply discussed previously, the similarity to real fire spreading records was observed for values between 0.7 and 1.¹² We considered 0.8 as a good starting point to represent some uncertainty regarding cells’ combustibility. Here, S_ij is a building structure parameter representing the combustibility of cell ij itself, in which a value of 1 represents a highly combustible cell, and 0 a fireproof cell. Due to the regularity of large basement garages, regular burnable cells (i.e., without physical obstacles) were defined with a 0.6 combustibility. Here, P_ij plays the role of controlling combustibility by adjusting the value of S_ij according to the combustibility or fire preventive cell ij. A value of P_ij = 0.5 was used to represent a higher uncertainty of combustibility in the scenario. The parameter W_ij is the wind spreading speed and direction, subject to the distance and direction from cell kl to cell ij. In such indoor urban fires, one of the main practices described in the firefighting action plan consists of shutting all entrances so as to minimize, or even suppress, the presence of wind (cf., Section 3.3). Under those conditions, W_ij was defined with a constant small value of 0.05 with random direction. In addition, β is a coefficient that is used to adjust the range and direction of spreading, 0 < β≤ 1. β was set to 1 due to its irrelevance. This value is also in agreement with the data presented by Ohgai et al.¹²

Finally, $p (t_{ckl})$ , $0 < p (t_{ckl}) \leq 1$ , is the ability of cell kl causing fire spreading at time t_ckl, which may be defined by the standard curve as follows¹³

p (t_{ckl}) = {\frac{4}{t_{2} - t_{1}} t_{ckl} + \frac{0.2 t_{2} - 4.2 t_{1}}{t_{2} - t_{1}}, t_{1} \leq t_{ckl} < \frac{t_{2} - t_{1}}{5} + t_{1} \frac{5}{4 (t_{2} - t_{1})} (t_{2} - t_{ckl}), \frac{t_{2} - t_{1}}{5} + t_{1} \leq t_{ckl} \leq t_{2}

(2)

in which $t_{1}$ and $t_{2}$ respectively represent the time from which cell $kl$ starts ( $f_{status} = 3$ from Figure 1) and stops ( $f_{status} = 4$ from Figure 1) having the ability to cause fire spreading after it starts burning. According to the literature, values of $t_{1} = 1$ minute and $t_{2} = 10$ minutes are common values that one may expect in indoor scenarios.¹²

Figure 1.

Work flow model of the fire spreading for cell $kl$ .

The fire spreading model was adapted in such a way that each active fire cell (i.e., $f_{status} \neq 0$ and $f_{status} \neq 1$ ) has the capability of spawning new active cells in its vicinities. Considering this principle, and based on Equations (1) and (2), a cell $kl$ on fire (i.e., active) will affect the status of its neighbor cells $ij$ according to the flowchart depicted in Figure 1. The fire will not only constrain the mobility of agents, that is, firefighters and victims, but may also lead to casualties. Although victims and firefighters will naturally avoid fire, some casualties might still occur due to the dynamical and unpredictable properties of fire spreading.

3.2. Victims’ behavior

This work addresses an ongoing rescue operation in an urban fire with tactical firefighters already deployed on site (see the following section). This suggests that most victims were able to evacuate at an early stage of the fire and, considering that the outbreak is taking place in a large basement garage, the exit flow was smoother due to the building geometry where groups can move as a whole toward the exit.¹⁴ This paper addresses the few victims left behind that, considering the progress made by the fire and the smoke felt in these scenarios, are under stress and panic due to the surrounding circumstances, leading to behavior based on incomplete and filtered information.

Victims are repelled by high temperatures, thus changing their motion according to fire spreading. Therefore, the adapted Helbing model for human behavior in fire evacuations presented by Heliövaara¹⁴ can be employed based on three assumptions: (a) the smoke no longer affects the victims’ motion as it is uniformly spread in the proposed environment; (b) the desired trajectory is no longer straightforward and ideal as victims may no longer use their visual perception; and (c) there is no interaction between victims due to the lack of visibility and harsh atmosphere.

In other words, victims tend to be still while waiting for rescue and call for help (cf., Section 3.5), unless they sense high temperatures in the vicinity. The equation of motion for victim $n$ may then be written as follows:

m_{n} \frac{d v_{n} [t + 1]}{dt} = m_{n} \frac{d v_{n} [t]}{dt} τ - α_{W} \nabla W - α_{T} \nabla T

(3)

wherein $m_{n}$ is the mass of victim $n$ , which was defined as 70 kilograms—the adult average weight. The variables $v_{n} [t]$ and $v_{n} [t + 1]$ are the velocity in the previous iteration and the velocity at the current iteration $t + 1$ , respectively. Here, $τ$ is the stochastic inertial coefficient that may be uniformly distributed between 0 and 0.9. The upper limit is defined in such a way that the previous velocity has a significant influence on the current velocity due to human dynamics.

Here, $\nabla W$ and $\nabla T$ are the gradient vectors of sensed obstacles and temperature, which are respectively affected by parameters $α_{W}$ and $α_{T}$ that represent the sensibility of victims to both constraints. Based on firefighters’ experience, victims display a superior sensibility to high temperatures, and thus parameters $α_{W}$ and $α_{T}$ were defined as 0.8 and 1.2, respectively.

Figure 2 presents a sequence that illustrates the behavior of a victim under the influence of both obstacles and fire. As it is possible to observe, the victim only reacts to the fire in a given proximity, thus getting repelled by it while avoiding obstacles.

Figure 2.

Sequence illustrating the behavior of a victim under the CHOPIN use case. The victim avoids both fire and obstacles and remains still when it does not sense any of those. Fire cells in yellow are catching fire, but have no ability to cause fire spreading, that is, $f_{status} = 2$ . Fire cells in red are on fire with the ability to cause fire spreading, that is, $f_{status} = 3$ . (Color online only.)

3.3. Firefighter behavior

Starting from the different individuals, all the way to the emergent behavior under stressful and dangerous situations, the predictability of firefighters’ actions is far from trivial as each rescuer adapts to its own perception of the context.¹ As such, this paper does not focus explicitly on the aforementioned human behavior, but rather on the action plan that describes the procedures of firefighters under those operations. Such guidelines were obtained by interviews with the BSC (https://www.bombeiros.pt/cbs/cb.php?cb=601) and the Portuguese Civil Defense Authority (ANPC; http://www.prociv.pt).

In short, by the time the occurrence is acknowledged, firefighters divide into two teams: (a) the tactical team and (b) the operations team. The tactical team usually consists of two higher-ranked agents that are the first to arrive on the scene to evaluate the damage and put out an action plan according to the contextual information retrieved so far (i.e., reconnaissance phase). The operations team is usually formed of six agents, that is, three rescue teams of two agents each. After reaching the scene, they are under the command of the tactical team, which decides how many rescue teams will enter the environment. On average, only one or two teams enter the scenario to ensure the permanence of backup teams outside the imminent danger zone. While searching for victims, each rescue team advances in the scenario and maintains a life safety rope linked to the exit. This is a strategy that rescue teams commonly use to avoid getting lost in the site.

During the rescue phase, the tactical team stays in the command center (CC) and exchanges data with the operations team while monitoring the rescue operation. The rescue phase typically lasts between 10 and 15 minutes. After that time, teams will try to control the fire and save any remaining victims. Nonetheless, due to the emphasis of the CHOPIN project, we will not consider anything further than the SAR phases.

Similar to that of victims, firefighter’s n motion may be adapted from Equation (3) as follows:

m_{n} \frac{d v_{n} [t + 1]}{dt} = m_{n} (v_{n}^{0} [t] - v_{n} [t]) τ - α_{W} \nabla W - α_{T} \nabla T

(4)

Although Equation (4) may be closely related to Equation (3), firefighter agents will have a sensing range of only 2 meters, which allows them to get closer to active fire cells so as to overcome those. This lower sensibility to temperatures is explained by their firefighting suits that handle temperatures of approximately $260 ° C$ (i.e., typical proximity suits; http://www.firerescue1.com/fire-products/Personal-protective-equipment-ppe/articles/1636063-What-you-need-to-know-about-proximity-suits/).

Firefighting in an indoor scenario requires a systematic and thorough analysis of an environment where sight is extremely limited due to smoke. This in combination with the heat compels firefighters to slowly follow the walls so as to avoid getting lost. In some situations, they may even have to crawl. At best, they are able to achieve a velocity of $1 m / s$ . As previously stated, each rescue team consists of two firefighters. To allow a straightforward implementation of the wall-following behavior, while keeping a short distance between both agents, only one of them is programmed with this behavior. Let us call it the master agent. The other one, which we will identify as the slave agent, is programmed with a simple attractive mechanism toward the master agent, while keeping both repulsive forces from Equation (4). In other words, the variable $v_{n}^{0} [t]$ from the slave agent is set as the current velocity from the master agent subtracted by a small safe distance, that is, $v_{i} [t] - Δ$ . Note that this safe distance $Δ$ is a vector in which the magnitude is uniformly distributed between $0.5$ and $2$ meters (common inter-agent distance) and the direction is given by the vector between the agents’ positions.

While exploring the scenario, firefighters’ main concern is to find victims. As they cannot rely solely on their vision, the most obvious sensing alternative is their audition. This is described further in Section 3.5. When the rescue team hears a call for help, the master agent moves toward the direction that maximizes the sound amplitude. This is once again accomplished by means of a simple attractive force. While the master is trying to find the victim, the slave stays near the wall, maintaining the life safety rope stretched between them. This ensures a safe return from the master carrying the victim to the slave. At that point, the firefighters must drag the victim to a safer location (i.e., nearest exit) before proceeding with the search. Considering a single entrance scenario, victims are usually dragged to the entrance near the CC so they may get assistance. This is accomplished by the reversed wall-following behavior, thus following the exact path previously taken. Once the CC is reached, the rescue mission is resumed.

Each rescue team is represented by the finite state machine (FSM) depicted in Figure 3.

Figure 3.

Finite state machine (FSM) of a firefighters’ rescue team. The top part of the FSM represents the master agent, while the bottom part represents the slave agent. CC: command center.

Besides the local communication between teammates, the mission requires sporadic communication with the CC so as to keep all the information centralized and up-to-date. This is accomplished by means of RF communication.

3.4. Radio frequency communication

Teams of firefighters do not communicate between themselves during the operation but they continuously communicate with the CC. In fact, the RF communication link between the CC and the firefighters is mostly used to exchange data for monitoring the operation. The decisions taken by the tactical team in the CC are more related to the addition of new resources (either human or tools) than to control the agents in the field, which, in turn, have the autonomy to make their own situated decisions. However, the global knowledge of the CC from all agents in the field may have weight in the agents’ decisions. Therefore, the firefighter action plan considers to be crucial a regular communication with the CC and waits for a decision in specific situations. On the other hand, the communication link between agents and the CC occasionally fails, thus resulting in some delays in the decision-making of agents. Agents will not wait endlessly for the CC’s call, and therefore such delays cannot be neglected. Accordingly, it is necessary to model the RF communication between the CC and all agents in the field.

The work of De Luca et al.¹⁵ evaluated the performance of networks for indoor localization based on RF power measurements. For that purpose, they used an extended multiwall propagation model that was developed on the basis of experimental measurements. In those experiments, it was possible to verify an attenuation of almost $l_{W} = 5.7 dB$ on heavy walls, that is, with wall thickness between 40 and 60 cm, for propagations at $2.4 GHz$ . This will be the reference attenuation value used in this work for RF communication, as we focus on garages wherein both external walls and concrete columns present a thickness within that range. The attenuation over the transmitter–receiver distance $d [m]$ can then be calculated as follows:

L = l_{c} + 10 γ \log \log d + \sum_{W}^{l_{W}}

(5)

where $W$ represents the number of walls between the transmitter and the receiver. The constant factor $l_{c}$ corresponds to the reference loss value at $1 m$ . This was defined as $l_{c} = 47.4 dB$ and experimentally validated in indoor scenarios by Luca et al.¹⁵

The path loss exponent $γ$ is typically defined between $2$ and $4$ , wherein values near $2$ correspond to propagation in free space and values near $4$ to lossy environments. Since our scenario is not propitious to RF propagation due to smoke, fire, and other nonlinear noisy phenomena (e.g., car and fire alarms, ceiling fire extinguishers, among others), a $γ$ uniformly distributed over intervals $3$ and $4$ was assumed. This provides a stochastic effect on the communication propagation, thus emulating the uncertainty of RF energy waves. For instance, as the environment characteristics rapidly change (e.g., fire spreading), this leads to significant deviations of the instantaneous power measured in the same location at different instants of time.

To provide an easier understanding of how RF may propagate in the scenarios considered in this work, an illustration of the phenomenon is presented in Figure 4. Note that the colors and corresponding color bar values are in dB. When the firefighters encounter a victim or fire outbreak they keep trying to communicate with the CC either until it replies back or until 10 seconds have passed. This time threshold of less than 2% of the mission time (cf., Section 3.3) seems to be an adequate value that does not significantly compromise a victim’s rescue but that may still create some disturbance overall.

Figure 4.

Radio frequency communication propagation under a large basement garage environment of approximately $2000 m^{2}$ —the Institute of Systems and Robotics at University of Coimbra garage. CC: command center.

3.5. Voice propagation

As previously mentioned, firefighters cannot rely on their vision due to dense smoke. Thus, firefighters detect victims by hearing their calls for help. Similar to RF propagation, voice is also affected by both distance and obstacles. Moreover, in such noisy scenarios (e.g., car and fire alarms), the success of such help requests may be greatly threatened.

The voice propagation model presents some properties (e.g., received power, interference, etc.) similar to any other kind of communication propagation. The major differences between waves reside in its intrinsic properties (e.g., frequency, wavelength, amplitude, among others). Taking this into consideration, such differences may be brought closer by manipulating the parameters from Equation (5).

A call for help may achieve a level between $72$ and $78 dB$ at approximately $1 m$ away from the source, that is, from a very loud voice to a shouting voice (http://www.engineeringtoolbox.com/voice-level-d_938.html). Moreover, as a rule of thumb, for every doubling of the distance from the source, the sound pressure level is reduced by $6 dB$ . Nevertheless, as for any other wave, voice is highly affected by its surroundings, which leads to losses in transmission.

According to the Fire Alarm Training Manual,¹⁶ one may expect average ambient sound levels between $40$ and $55 dB$ in underground structures and medium-density urban environments. This will significantly reduce the ability to identify a call for help to distances between approximately 7 and 58 m, depending on the source and the ambient noise level. Moreover, the addition of obstacles absorbs part of the sound passing through, usually through conversion to heat energy.¹⁷ Such attenuation may be described as follows:

A = A_{0} e^{- μ d}

(6)

wherein $A_{0}$ is the unattenuated amplitude of the wave before striking the obstacle and $μ$ is the attenuation coefficient of the traveling wave that may vary between 0.01 and 0.035 for regular concrete walls.

By considering the previously presented concepts, the voice propagation was modeled by changing (5) with $l_{W} = 2 dB$ and a path loss exponent $γ$ distributed over the interval. The reference value of attenuation $l_{c}$ was randomly defined between $- 78$ and $- 72 dB$ to represent the stochasticity of a victim’s voice level. Moreover, the ambient noise was also uniformly distributed between $40$ and $55 dB$ , thus resulting in a stochastic effect in the identification of victims for a received power above those values.

Putting this all together, the modeling of a firefighting mission in response to an urban fire in a large basement garage becomes complete. The next section explores how teams of robots can be leveraged in USAR scenarios as well as the approach followed by these teams in the reconnaissance and rescue phases of the mission under study.

4. Firefighting cooperative robotics

In the scenario previously described, robots can be an invaluable help if they are equipped with sensors that enable their navigation in a smoky atmosphere where human responders are highly hampered due to lack of visibility (e.g., ultrasound, thermal cameras, ultra-wide band radars, among others). This lack of visibility not only makes firefighters’ progression very slow but also constrains their movements closer to the walls so as to maintain their reference (cf., Section 3.5). It is therefore clear that robots may provide a systematically updated map of the environment by means of multi-robot (SLAM) strategies, for example, Howard.¹⁸ In this work, robots perform cooperative two-dimensional (2D) SLAM using non-optical ranging sensors, which are not so affected by smoky conditions. Our research group has experience on coupling ultrasound sensors with optical-based LIDARs for mapping smoky environments (see Machado Santos et al.¹⁹). The output of the approach is a joint 2D occupancy grid map of the environment to assist the team of firefighters.

Besides, robots can be used to create and maintain a communication network between the first responders and the fixed base station, known as the CC. This is highly valuable, as RF communication greatly suffers from fading in the presence of obstacles, and firefighters frequently lose communication with the CC (cf., Section 3.4).

Last but not least, robots can assist in the acquisition of contextual information that may be useful for firefighters to successfully accomplish their mission in a shorter period of time with a decreased risk. As time is at the essence of response to such catastrophic scenarios, informing the first responders on environmental context (e.g., fire outbreaks), mission-related context (e.g., victims’ position), and agent-based context (e.g., level of the self-contained breathing apparatus) may greatly boost the firefighting operation. Robots and first responders use the same MANET for communication, which is maintained by constrained spatial distribution of the mobile robots in the environment. This not only allows one to guarantee the efficiency of communications between the team of firefighters, but also the interaction between the teams of humans and robots through the CC. Communication from robots to humans encompasses several major points: sharing the cooperatively built map with humans, location of victims, and monitor the evolution of nearby fire outbreaks. This is all transmitted back to the CC, allowing firefighters to have increased situational awareness through contextual information that will be useful to successfully accomplish their mission in a shorter period of time with a decreased risk.

In the scope of the CHOPIN project, the authors previously presented a distributed collective robotic architecture in SAR scenarios.¹ The problem was divided into two phases matching the most critical phases of a firefighting operation: reconnaissance and rescue. Both robotic strategies inherent to each phase will be briefly described in the following sections.

4.1. Reconnaissance

In the reconnaissance phase, a fleet of cooperative mobile robots aims to explore thoroughly the catastrophic scenario, being in contact with human agents and victims. Hence, the robotic team needs to be able to participate in the search task, signaling the presence of victims and possible evolution of the disaster (e.g., mapping of temperature concentration in a firefighting operation). Similar operations have been recently addressed using optimization and foraging. In fact, the problem of searching is analogous to foraging in the animal kingdom and thus understanding foraging can provide insights for robotics.

More recently, a complete swarm robotic search solution²⁰ based on Darwinian Particle Swarm Optimization (DPSO)²¹ for real mobile robots, denoted as Robotic DPSO (RDPSO), was proposed. In brief, RDPSO allows one to have multiple dynamic swarms, thus enabling a distributed approach divided into multiple networks (one for each swarm). This allows for decreasing the number of nodes (i.e., robots) and the amount of information exchanged between robots of the same network. In other words, robots interacting with other robots through communication is confined to local interactions inside the same group (swarm), thus making RDPSO scalable to large populations of robots. For that purpose, a minimum $N_{min}$ and maximum $N_{max}$ number of robots is required to form an active swarm in which the CC will be considered as a common member of all swarms. This allows the centralization of the information just to keep the CC informed of all robots’ states.

Each swarm individually follows Equations (7) and (8) to search for a solution and has some “punish”–“reward” rules to govern the whole population of robots based on the concept of social exclusion. For more details, refer to Couceiro et al.²⁰:

v_{n} [t + 1] = w_{n} [t] + \sum_{i = 1}^{4} ρ_{i} r_{i} (χ_{i} [t] - x_{n} [t])

(7)

x_{n} [t + 1] = x_{n} [t] + v_{n} [t + 1]

(8)

This set of equations represents the behavior of robot $n$ at each iteration with $t \in N_{0}$ . Coefficients $ρ_{i}$ , $i = 1, 2, 3, 4$ , assign weights to the local best (i.e., cognitive component), the global best (i.e., social component), the obstacle avoidance component, and the network connectedness enforcement component when determining the new velocity, with $ρ_{i} > 0$ . Parameters $r_{i}$ are random vectors wherein each component is generally a uniform random number between 0 and 1. Here, $v_{n} [t]$ and $x_{n} [t]$ represent the velocity and position vector of robot $n$ , respectively; $χ_{i} [t]$ represents the best position of the cognitive, social, obstacle, and MANET components. In this work, each robot explores and maps the scenario “independently”; therefore, the social component is not considered, $ρ_{2} = 0$ , allowing robots to spread out in the environment.

The number of times a swarm $s$ is evolved without finding an improved objective is tracked with a search counter, $S C_{s}$ . If the swarm’s search counter exceeds a maximum critical threshold, ${SC}^{max}$ , the swarm is punished by excluding the worst performing robot, which is added to a socially excluded group. In this situation, the swarm’s search counter is then reset to a value near ${SC}^{max}$ , which can be calculated as follows:

S C_{s} = {SC}^{max} [1 - \frac{1}{N_{s}^{kill} + 1}]

(9)

wherein $N_{s}^{kill}$ represents the number of robots excluded from the swarm $s$ over a period of time in which there was no improvement in the swarm’s objective function. The worst performing robot is evaluated by the value of its objective function compared to other members in the same swarm, i.e., if the objective is to maximize the fitness function, the robot to be excluded will be the one with the lower fitness value. In what concerns the socially excluded robots, instead of searching for the objective function’s global optimum like the other robots in the active swarms, they basically randomly wander in the scenario. This approach improves the algorithm, making it less susceptible to becoming trapped in a local optimum. However, they are always aware of their individual solution and the global solution of the socially excluded group.

Therefore, it may be possible that if any swarm is unable to improve, that is, robots cannot find better solutions for a specific time—the maximum critical threshold, ${SC}^{max}$ —all robots will be added to the socially excluded group, thus forming a single swarm. This was established as the switching criterion that allows switching from an exploring behavior to an inspection behavior, that is, $N_{excluded} = N$ , wherein $N_{excluded}$ and $N$ correspond to the number of robots within the excluded group and the total number of robots in the SAR mission, respectively.

Within the context of a firefighting operation occurring in a large basement garage, the reconnaissance phase will correspond to exploring the scenario while robots perform cooperative SLAM.¹ Hence, the objective function of the team may be defined as a cost function in which robots need to minimize the map’s entropy.²² Therefore, during exploration, each robot $n$ computes its best frontier cell as follows:

m_{i}^{s} = [ψ (x_{t}, m_{i}) | | \vec{\nabla} H (m_{i} | |)]

(10)

wherein $N (x_{n} [t], r_{s})$ represents the set of frontier cells located in the neighborhood of robot $n$ with sensing radius $r_{s}$ . The coefficient $ψ (x_{n} [t], m_{i}) \in [0; 1]$ measures whether the cell $m_{i}$ is in line-of-sight from a position $x_{n} [t]$ , which also implies that cell $m_{i}$ is likely to be empty. Moreover, the entropy of the cell $m_{i}$ is represented by $H (m_{i})$ and may be calculated as follows:

H (m_{i}) = - p (m_{i}) \log \log [p (m_{i})] - (1 - p (m_{i})) [1 - p (m_{i})]

(11)

where $p (m_{i})$ is the probability that a grid cell is occupied. In brief, when a robot is unable to find empty cells during a threshold ${SC}^{max}$ , it will get excluded from its current active group. When all robots are excluded, that is, $N_{excluded} = N$ , the collective inspection phase starts.

4.2. Rescue

In the rescue phase, the mobile robot team should cover the whole scenario, identifying the location of any remaining victims and the possible evolution of the disaster, for example, monitor the fire evolution in a firefighting operation and transmit this information to the CC. The problem proposed in this phase finds similarities with the multi-robot coverage problem, where agents should visit and clear the whole environment. More particularly, it resembles an inspection or patrolling mission, where the map is already available to the robots and important locations are defined. Therefore, the team of agents is expected to reorganize and visit all of these locations in an effective way and report new situations back to the CC, such as undetected victims or new fire outbreaks.

In this work we will adopt a modified version of the State Exchange Bayesian Strategy (SEBS) for multi-robot patrolling.²³ This strategy was chosen due to the recognized results obtained when compared to other state-of-the art strategies and due to the specificities of the problem. Briefly, the idea behind the SEBS is to have robots coordinating themselves in a distributed way, while promoting efficiency in the patrolling mission. Robots in the team consider two variables when deciding their moves, the Gain (G_i) that each robot expects to earn when moving to a location $v_{i}$ , and the State (S_i), which guarantees the coordination between teammates and prevents different agents visiting the same location. The Gain variable is a function of the idle time that a location v_i spends without being visited, that is, the idleness $I_{v_{i}}$ and the distance $| e_{0 i} |$ between the current location v₀ and v_i. It is given by the following:

G_{i} (t) = c \cdot (\frac{I_{v_{i}} (t) - I_{v_{i}} (t + Δ t)}{| e_{0 i} |})

(12)

wherein c represents constant robot speed. The State variable keeps track of teammates’ moves and future immediate goals to ensure the coordination of robots. More particularly, the State S_i is a discrete variable that represents the number of robots that intend to visit a given location v_i involved in the decision process of a robot, which is currently located in v₀:

S_{i} \in N_{0} \cap [0, R - 1], R > 1

(13)

wherein R represents the number of robots in the team. Robots communicate not only their current location, but also inform other robots where they have decided to move next. Using this approach, robots are capable of computing the state directly by collecting other robot’s intentions and checking the locations involved in their decision process. Each robot calculates the probability of moving to a given location recurring to each variables’ statistical distribution. For the Gain variable, the distribution is given by the following:

F (g) = L \cdot \exp (\frac{\ln (\frac{1}{L})}{M} \cdot g)

(14)

wherein L and M are constants. More specifically, L controls the probability values for lower gains and M is the gain saturation, beyond which the probability values are maximum: $F (g \geq M) = 1$ . Here, L is usually set to a value close to 0, while M is calculated using an upper bound of $I_{v_{i}}$ . Note that in this distribution, higher values of gain become rapidly more influential on the robot’s decision. On the other hand, for the State’s discrete probability distribution, we use the following:

f (s) = P (S_{i} = s) = \frac{2^{R - (s + 1)}}{2^{R} - 1}

(15)

which means that with a greater the number of teammates in the vicinity of a robot, it becomes increasingly unlikely for the robot to move in that direction. Finally, in order to obtain the probability of moving to a given location v_i, the authors make use of Bayes’ rule:

\begin{matrix} P (move (G_{i}, S_{i}) \propto P (move (v_{i}) \cdot P (G_{i} | move (v_{i})) \\ \cdot P (S_{i} | move (v_{i})) \end{matrix}

(16)

wherein $P (move (v_{i})$ represents prior knowledge and a priori restrictions to the problem, and $P (G_{i} | move (v_{i})$ and $P (S_{i} | move (v_{i}))$ are given by $F (g)$ and $f (s)$ , respectively. For more details on the approach, the reader should refer to Portugal and Rocha.²³

In this work, robots may not be within communication reach. In such situations, robots will not make completely informed decisions, which will have an influence on the results, as seen in the next section.

Beyond adapting the SEBS to the communication constraints inherent in SAR scenarios, it is important to point out that the original strategy assumes a static topological map (i.e., an undirected graph $G (V, E)$ ) with vertices representing important locations in the scenarios and edges representing the cost of moving between two neighbor vertices. However, in this context, there is no graph per se. Robots are in the presence of a dynamic grid-like map and only have knowledge of important locations that they should visit. Therefore, they maintain a list of locations that need to be visited (similar to the vertices in a graph) and consider pseudo-costs between these locations by calculating their Euclidean distances. In this way, they will not need to maintain a whole graph.

5. Simulation results

This section shows the potential of employing a robotic team alongside first responders, to assist in the exploration of the environment and maintain the robustness of the communication network, which consequently increases the efficiency and performance in fulfilling the rescue mission. This section explores the accuracy of the mathematical modeling of a traditional firefighting mission in response to an urban fire in a large basement garage. This allows the drawing of some conclusions regarding the time required to save a certain number of victims (i.e., mission completeness) solely with human teams and posteriorly a comparison with human–robot cooperation within such missions.

5.1. Human teams

To evaluate the performance of firefighters in the rescue operation, 30 simulation trials for each different configuration were conducted in the same basement garage environment represented in Section 3 (cf., Figure 5 and 6). Experiments were carried out with one and two rescue teams, that is, two and four firefighters, denoted as 2F and 4F, respectively, for four and eight victims, denoted as 4V and 8V, respectively. This then encompasses a total of 120 trials, divided into four different configurations with 30 trials each. The victims were randomly positioned in the scenario at the beginning of each trial. Moreover, 10–20 fire active cells were randomly spawned at the beginning of each trial.

Figure 5.

Evolution of rescue during a simulation trial: (a) behavior of a team of two firefighters using common wall-following exploring methods in a scenario with four victims; (b) timings for the rescue of victims in a scenario with a single team of two human agents. (Color online only.)

Figure 6.

Evolution of rescue during a simulation trial; (a) behavior of a team of two firefighters assisted by four robots in a scenario with four victims; (b) timings for the rescue of victims in a scenario with a team of two humans and four robots. (Color online only.)

In agreement with Section 3.3 and the Portuguese Fire Department indications, rescue missions (i.e., trials) were limited to 15 minutes and the CC was placed near the only exit and the rescue teams’ starting positions were defined within its vicinities.

Let us, at first, provide a temporal, and somewhat detailed, example of how a mission with teams of humans is fulfilled. Figure 5(a) depicts the trajectory carried out by a single team of two firefighters while trying to rescue four victims. In this specific example, the team was only able to rescue three victims within the 15 minutes. The vertical colored lines in the temporal chart (Figure 5(b)) for each victim represents, in that order, the moment firefighters heard the victim (red line), the moment the master was able to reach it (yellow line), the moment the master reached the slave (green line), and finally, the moment the team reached the CC with the victim (blue line).

In view of these preliminary results, the next section introduces a team of robots into the same setup to be compared and to assess the potentialities of such human–robot cooperation.

5.2. Mixed human–robot teams

To evaluate the performance of human firefighters assisted by mobile robots in the rescue operation, 30 simulation trials for the same configurations described in the previous section were conducted. The only difference in the experiments described in the previous section was the addition of a team of four robots initially located near the CC.

Similar to before, let us analyze the mission fulfilment with teams of humans and robots. Figure 6(a) depicts the trajectory carried out by a single team of two firefighters while trying to rescue four victims. In this example, the team was able to rescue all victims within the 15 minutes. Note that robots are able to inform human rescuers of victims’ possible locations. Moreover, since robots build the map quicker over time than firefighters are able to navigate in the scenario (due to the lack of visibility), both master and slave are able to stay closer to each other in most situations, navigate more accurately using the map provided by robots, and in a less constrained way, that is, without being necessarily restricted to wall following.

Henceforth, only three vertical colored lines for each victim are represented in the temporal chart (Figure 6(b)): the first one corresponds to the moment robots or firefighters heard the victim (red line), the second one to the moment the human team is able to reach it (green line), and finally, the third line corresponds to the moment the team reached the CC with the victim (blue line). It is interesting to observe that firefighters become aware of victims’ locations due to robots’ exploration before they can get the current rescued victim to the CC. In fact, in some situations, they even become aware of other victim’s locations before they can reach the victim they are trying to rescue at that time. This means that the mobile robot team is able to inform the team of firefighters about victims’ locations considerably faster than they can rescue them. In other words, rescue resources (i.e., firefighters) when using only a single team are not enough for the outcome of the robotic team. Looking closely at the results obtained in Figure 6, it can be seen that the number of victims rescued in each situation generally increased with the assistance of the robotic team, compared with Figure 5.

To promote a deeper analysis of the outcome of the rescue operation, boxplot charts are used to graphically represent the number of rescued victims for each different configuration (Figure 7). The ends of the blue boxes and the horizontal red line in between correspond to the first and third quartiles and the median values, respectively. It must be emphasized, once again, that teams may be unable to save all victims within the first 15 minutes of the whole mission (rescue phase) since they do not know their location. Usually, the remaining victims are rescued during the second phase while fighting fire outbreaks.

Figure 7.

Number of rescued victims for one and two teams of firefighters: (a) without robot cooperation; (b) cooperating with four robots. (Color online only.)

As one may observe in Figure 7(a), for a single team of firefighters, they can only rescue approximately half of the victims placed on the scenario. In fact, the best outcome arising from a team of two firefighters was observed in a single trial wherein they were able to rescue five of the eight victims in 15 minutes, that is, an outstanding result of 3 minutes on average for each victim. Such outcome is greatly overcome by adding a new team of firefighters. With four victims to save, only once they were unable to rescue the four available victims (outlier). Regarding the worst-case scenario with eight victims, the two teams were able to save all of them in only 10% of the trials (three trials). The median value stood around six rescued victims. Although the rescue mission completeness using only teams of firefighters does not attain 100% efficiency, the results are still commendable, varying between approximately 45% (2F8V) and 99% (4F4V).

On the other hand, with the assistance of four robotic agents, it can be observed in Figure 7(b) that there is an increase of rescued victims, except in the 4F4V case, which retained exactly the same result of 99% rescue. In the team configuration 2F4V, it occurred that, in 25% of the trials, all four victims were successfully rescued, which never happened in the same configuration without robots. In the 2F8V case, 50% of rescues occurred with between four and five victims. The most denotative result in these trials was the 4F8V configuration, in which the median value stood around eight rescued victims, as opposed to the six victims without robot assistance. In view of these rescue results, it is noteworthy that human resources (firefighters) are constraining the mission’s success when only one team is used (2F4V and 2F8V). Nevertheless, one should explore the possible reason, or reasons, behind such incompleteness. Although the main reason may reside in the dynamical constraints of human agents under such scenarios (e.g., lack of visibility), communication failures between agents and the CC may also contribute to a decreased performance.

As previously stated, the communication between agents and the CC is considered to be crucial and the absence of it may delay the mission. One of the methods to evaluate the communication cost consists of the processing time to handle the communication procedure (i.e., pause time) for each team, over the 15 minutes of each trial. Regarding the pause time inherent to the whole communication procedure, a timer was used to measure the turnaround time of communications between agents and the CC. It is noteworthy that during that time, agents from that team are unable to perform any other action while waiting for the CC reply. A maximum pause time of 10 seconds was considered for each communication procedure (cf., Section 3.4).

Let us once again appeal to boxplots to graphically represent the pause time from the RF communication (Figure 8). As one may observe in Figure 8(a), for a single team of two firefighters the median value of pause time due to communication failures stays around 10 seconds. This corresponds to approximately 1% of the rescue mission time, which may be considered insignificant. It is, however noteworthy, that in approximately one third of the trials the pause time may reach 20 seconds or, in some minor occasions, it may even reach 30 seconds. Half a minute represents a significant interference in the overall mission. Nevertheless, worse situations may be observed when one resorts to two teams of firefighters. Under such situation, the worst-case scenario may go above 30 seconds to up to 50 seconds (almost 1 minute). Despite seeming like significant values, 1 minute corresponds to 1/15th of the rescue mission time in which a firefighter may travel an average distance of 60 meters (more than the scenario’s east wall length). In addition, it is also clear that the assistance given by robotic agents (Figure 8(b)) in the maintenance of the ad hoc network resulted in the persistent connectivity of the network during all missions, meaning that no messages were dropped in the communications between agents and the CC.

Figure 8.

Pause time inherent in the radio frequency communication procedure for one and two teams of firefighters: (a) without robot cooperation; (b) cooperating with four robots.

To outline the comparison, Figure 9 shows an illustrative bar chart of the firefighting operation efficiency. Such efficiency $η_{T}$ was computed as a variation of the Single Robot Path Length Ratio metric proposed by Navarro and Matía²⁴ for each victim $n$ rescued by firefighter team $T$ :

η_{n} = \frac{| | {\vec{C}}_{T} [t_{n}^{1}] - {\vec{C}}_{T} [t_{n}^{0}] | |}{\frac{1}{2} \sum_{i = 1}^{2} (\sum_{t = t_{n}^{0}}^{t_{n}^{1}} (| | {\vec{x}}_{i} [t] - {\vec{x}}_{i} [t - 1] | |))} \cdot \frac{N_{res}}{N_{tot}}

(17)

wherein ${\vec{C}}_{T}$ is the center of mass position of the firefighter team $T$ and ${\vec{x}}_{i}$ is the position of the agent. This metric takes into consideration the ratio between the distance of a straight line from the instant $t_{n}^{0}$ firefighters from team $T$ start looking for victim $n$ (in the CC vicinities) to the instant $t_{n}^{1}$ in which they find it, and the real path length of each firefighter from team $T$ between these same instants. Therefore, it provides a normalized value that represents a notion of how far firefighters had to travel to save each victim. The second term $(\frac{N_{res}}{N_{tot}})$ represents the ratio of victims rescued $N_{res}$ over the total number of victims $N_{tot}$ in the scenario.

Figure 9.

Rescue efficiency by firefighter per victim during the mission: (a) without robot cooperation; (b) cooperating with four robots.

As one may observe in Figure 9, without the assistance of the robotic agents, the rescue efficiency is worse for every team configuration. This happens due to the early knowledge of victims’ location provided by the robots. It can be observed that teams of firefighters without robot support are only sufficient when no more than four victims is involved. However, for scenarios with more victims, the presence of the robots is critical. For instance, it was possible to achieve a superior efficiency of approximately 15% for the 4F8V configuration. Since the number of rescued victims increases by adding robotic agents to the firefighting operation (cf. Figure 7), the efficiency is always superior when resorting to human–robot cooperation.

Despite the positive return from human–robot cooperation observed in Figures 7 –9, the most notable feature is the decrease in the time necessary to detect victims (see Figures 10 and 11). In Figure 10, for every case with a total of four victims, the average time instants when each victim is detected by the rescuing team are illustrated. For instance, with four firefighters and no robots (4F4V case), the second victim was detected on average after 2 minutes. Note again that each mission was limited to 15 minutes, as described in Section 3.3. Hence, a maximum value of 15 minutes was considered when it was not possible to detect a given victim. For instance, in the 2F4V case the fourth victim is often left undetected by the end of the mission. In fact, when using only teams of humans, the third victim is usually detected late in the mission and cannot be rescued in time. However, when employing robotic teams in the rescue mission, the time necessary to detect the third victim decreases significantly by 83%. The same analysis is conducted for the trials with a total of eight victims (see Figure 11). Once again, the large decrease in the awareness time of victims’ locations is verified. However, robots seem to have a larger impact in this case. When using only a team of firefighters, the fifth victim is often not detected in time, while adding a team of robots decreases to 9 minutes the average time necessary to detect all eight victims. This result clearly shows the potential of using robots in cooperation with human teams in rescue operations. However, even though the cooperation between human and robots is beneficial in that robots are able to detect all the victims within the mission time and inform firefighters about their location, an increase in the number of firefighters is still necessary in order to rescue them afterwards.

Figure 10.

Average awareness time of the victims’ locations (trials with a total of four victims). Two firefighters: (a) without robot cooperation; (b) cooperating with four robots. Four firefighters: (c) without robot cooperation; (d) cooperating with four robots.

Figure 11.

Average awareness time of the victims’ locations (trials with a total of eight victims). Two firefighters: (a) without robot cooperation; (b) cooperating with four robots. Four firefighters: (c) without robot cooperation; (d) cooperating with four robots.

Overall, these results show the potential of employing a robotic team alongside first responders to assist in the exploration of the environment and maintain the robustness of the communication network. As a consequence, the efficiency and performance in fulfilling the rescue mission increase.

6. Conclusion

In this article, the importance of human–robot teaming in SAR scenarios has been demonstrated by proposing and extracting results via a simulation and modeling framework that integrates several phenomena observed in an urban fire SAR mission. Based on the behavior of firefighters and victims, and modeling important variables, such as fire spreading, RF communication, and voice propagation, it was shown that human rescuers face extreme difficulties and timing constraints at distinct stages of the mission. By fostering cooperation with mobile robot teams with specific capabilities, the potential to assist humans and gain effectiveness in this context has been proved by intensive simulations in a large urban fire scenario under a set of different configurations. A first step has been made in this article to quantify the improvement of the rescue mission, obtained when using collaborating robots alongside human rescue teams. In the future, field experiments would be valuable to verify and/or adjust the models used, as well as to present and validate real-world solutions for human–robot cooperation teams in urban firefighting missions.

Footnotes

Funding

This paper reports research work carried out within the CHOPIN R&D project, funded by the Portuguese Foundation for Science and Technology (FCT) with (ref. PTDC/EEA-CRO/119000/2010). The first two authors were also supported by PhD scholarships granted by FCT (SFRH/BD/73382/2010 and SFRH/BD/64426/2009).

ORCID iD

Micael S Couceiro

Author biographies

Micael S Couceiro obtained the PhD degree in Electrical and Computer Engineering at the University of Coimbra. Over the past 10 years, he has been conducting scientific research on several areas, including robotics, computer vision, sports engineering, and others. This resulted on more than 50 articles in international impact factor journals, more than 60 articles at international conferences, and 3 books. He is currently an Associate Researcher at the Institute of Systems and Robotics and the co-founder and CEO of Ingeniarius, Ltd. (Portugal).

David Portugal completed his PhD degree on Robotics and Multi-Agent Systems at the University of Coimbra in Portugal, in March 2014. His main areas of expertise are cooperative robotics, multi-agent systems, simultaneous localization and mapping, field robotics, human-robot interaction, sensor fusion, metaheuristics, and graph theory. After his PhD, he spent 4 years outside academia, and he is currently working as an Assistant Researcher at the University of Coimbra since 2019. He has been involved in several local and EU-funded research projects in Robotics and Ambient Assisted Living, such as CHOPIN, TIRAMISU, Social Robot, CogniWin, GrowMeUp, STOP, CORE, SEMFIRE and WoW. He has co-authored over 70 research articles included in international journals, conferences and scientific books.

Rui P Rocha is an Assistant Professor (tenure position) in the Department of Electrical and Computer Engineering and a permanent researcher at the Institute of Systems and Robotics, both at the University of Coimbra, Portugal. His research interests include cooperative mobile robotics, cooperative perception, multi-robot systems, human-robot team cooperation, distributed control, ambient assisted living, social robots, and autonomous robots. Rocha received a PhD in electrical and computer engineering from the University of Porto in May 2006. He has been involved in several FP6 and FP7 European funded projects developed in consortium for the past few years, and he is a senior member of IEEE and of the IEEE RAS Technical Committee on Multi-Robot Systems.

André Araújo received the MSc degree in Electrical and Computer Engineering from the University of Coimbra (UC) in July 2012, with a MSc dissertation entitled “ROSint - Integration of a mobile robot in ROS architecture”. Afterwards, he held a research fellowship in the CHOPIN (Cooperation between Human and rObotic teams in catastroPhic INcidents) R&D project, at the Institute of Systems and Robotics (ISR-UC), and a research fellowship in the PRODUTECH project (Production Technologies Cluster) R&D initiative, at the Centre of Robotics in Industry and Intelligent Systems (CROB), INESCTEC, Porto. His main research interests are Mobile Robotics, Unmmanned Aerial Vehicles, Internet of Things, Mechatronics, and Low-level Control. He is a co-founder of Ingeniarius, Ltd. (Portugal), currently performing as CTO and CFO, and a PhD student at the University of Coimbra.

References

Couceiro

Portugal

Rocha

. A collective robotic architecture in search and rescue scenarios. In: Proceedings of the 28th annual ACM symposium on applied computing, Coimbra, Portugal, 18–22 March 2013, pp.64–69. ACM.

Witkowski

El Habbal

MAM

Herbrechtsmeier

, et al. Ad-hoc network communication infrastructure for multi-robot systems in disaster scenarios. In: Proceedings of IARP/EURON workshop on robotics for risky interventions and environmental surveillance (RISE 2008), Benicassim, Spain, 7–8 January 2008.

Gandhi

Ganz

Mullett

. FIREGUIDE: firefighter guide and tracker. In: 2010 Annual international conference of the IEEE engineering in medicine and biology, Buenos Aires, Argentina, 31 August–4 September 2010, pp.2037–2040. IEEE.

Pascucci

Panzieri

Marsella

, et al. A REference implementation of interoperable indoor location & communication systems for First REsponders: the REFIRE project. In: 2012 IEEE international symposium on safety, security, and rescue robotics (SSRR), College Station, TX, 5–8 November 2012, pp.1–5. IEEE.

Gancet

Motard

Naghsh

, et al. User interfaces for human robot interactions with a swarm of robots in support to firefighters. In: 2010 IEEE international conference on robotics and automation (ICRA), Anchorage, AK, 4–8 May 2010, pp.2846–2851. IEEE.

Marjovi

Marques

Penders

. Guardians robot swarm exploration and firefighter assistance. In: Workshop on NRS in IEEE/RSJ international conference on intelligent robots and systems (IROS), St Louis, MO, 11–15 October 2009. IEEE.

Kruijff

GJM

Janíček

Keshavdas

, et al. (2014). Experience in system design for human-robot teaming in urban search and rescue. In: Field and service robotics. Berlin Heidelberg: Springer, 2014, pp. 111–125.

Kruijff-Korbayová

Colas

Hindriks

, et al. TRADR project: long-term human-robot teaming for robot assisted disaster response. KI - Künstliche Intelligenz 2015; 29: 193–201.

Murphy

Tadokoro

Nardi

, et al. Search and rescue robotics. In: Khatib

Siciliano

(eds) Springer handbook of robotics. Berlin Heidelberg: Springer, 2008, pp.1151–1173.

10.

Kim

Kang

Sun

, et al. Design and implementation of user-friendly remote controllers for rescue robots used at fire sites. In: Proceedings of SICE Annual Conference, Taipei, Taiwan, 18–21 August 2010, pp.875–880. IEEE.

11.

Doroodgar

Ficocelli

Mobedi

, et al. The search for survivors: cooperative human-robot interaction in search and rescue environments using semi-autonomous robots. In: 2010 IEEE international conference on robotics and automation (ICRA), Anchorage, AK, 4–8 May 2010, pp.2858–2863. IEEE.

12.

Ohgai

Gohnai

Ikaruga

, et al. Cellular automata modeling for fire spreading as a tool to aid community-based planning for disaster mitigation. In: Timmermans

van Leeuwen

(eds) Recent advances in design and decision support systems in architecture and urban planning. Dordrecht: Springer Netherlands, 2004, pp.193–209.

13.

Wakamatsu

. The risk of fire. Arch Outline 1978; 21: 279–298.

14.

Heliövaara

. Computational models for human behavior in fire evacuations. Department of Engineering Physics and Mathematics, Helsinki University of Technology, 2007.

15.

De Luca

Mazzenga

Monti

, et al. Performance evaluation of indoor localization techniques based on RF power measurements from active or passive devices. EURASIP J Appl Sign Proc 2006; 2006: 160–160.

16.

Potter Electric Signal Company LLC. Fire alarm training manual. St. Louis, MO, 2010.

17.

Truax

(ed.). Handbook for acoustic ecology. The World Soundscape Project. Burnaby, Canada: Simon Fraser University, 1978.

18.

Howard

Multi-robot simultaneous localization and mapping using particle filters. Int J Robot Res 2006; 25:, 1243–1256.

19.

Machado Santos

Couceiro

Portugal

, et al. A sensor fusion layer to cope with reduced visibility in SLAM. J Intell Robot Syst 2015; 80: 401–422.

20.

Couceiro

Rocha

Ferreira

. (2011). A novel multi-robot exploration approach based on particle swarm optimization algorithms. In: 2011 IEEE international symposium on safety, security, and rescue robotics (SSRR), Kyoto, Japan, 1–5 November 2011, pp.327–332. IEEE.

21.

Tillett

Rao

Sahin

, et al. Darwinian Particle Swarm Optimization. In: Proceedings of the 2nd Indian international conference on artificial intelligence (IICAI-05), Pune, India, 20–22 December 2005, pp.1474–1487.

22.

Eberhart

Kennedy

. A new optimizer using particle swarm theory. In: Proceedings of the sixth international symposium on micro machine and human science, 1995 (MHS’95), Nagoya, Japan, 4–6 October 1995, pp.39–43. IEEE.

23.

Portugal

Rocha

RP.

Distributed multi-robot patrol: a scalable and fault-tolerant framework. Robot Auton Syst 2013; 61: 1572–1587.

24.

Navarro

Matía

. A proposal of a set of metrics for collective movement of robots. In: Proceedings of the workshop on good experimental methodology in robotics, at RSS 2009, Seattle, WA, 28 June 2009.