Identification of crowd behaviour patterns using stability analysis

Abstract

Crowd behaviour analysis and management have become a significant research problem for the last few years because of the substantial growth in the world population and their security requirements. There are numerous unsolved problems like crowd flow modelling and crowd behaviour detection, which are still open in this area, seeking great attention from the research community. Crowd flow modelling is one of such problems, and it is also an integral part of an intelligent surveillance system. Modelling of crowd flow has now become a vital concern in the development of intelligent surveillance systems. Real-time analysis of crowd behavior needs accurate models that represent crowded scenarios. An intelligent surveillance system supporting a good crowd flow model will help identify the risks in a wide range of emergencies and facilitate human safety. Mathematical models of crowd flow developed from real-time video sequences enable further analysis and decision making. A novel method identifying eight possible crowd flow behaviours commonly seen in the crowd video sequences is explained in this paper. The proposed method uses crowd flow localisation using the Gunnar-Farneback optical flow method. The Jacobian and Hessian matrix analysis along with corresponding eigenvalues helps to find stability points identifying the flow patterns. This work is carried out on 80 videos taken from UCF crowd and CUHK video datasets. Comparison with existing works from the literature proves our method yields better results.

Keywords

Crowd flow surveillance optical flow crowd model stability analysis

1 Introduction

The increased population density in cities and the socio-economic structure make security an essential concern of any government. Large gatherings occur in many human-related activities, and effective management of its mobility leads to the quality of human experience. Security systems implemented in the public spaces facilitate monitoring crowds and further analysis of security concerns. However, most video surveillance systems use human-in-loop in decision-making and hence have many limitations leading to disasters. A system capable of automatically estimating crowd size, behaviour, and deviation from expected behaviour would assist better crowd management. The area of intelligent surveillance systems is now attracting more research attention, and it will become capable of overriding traditional surveillance systems soon. An intelligent surveillance system uses strategies and algorithms from many potential research areas like computer vision, machine learning, pattern recognition, and artificial intelligence [35]. There has been significant interest in employing computer vision methods to monitor crowds. It is possible to model the crowd from video and apply efficient algorithms for crowd flow detection, localization, and movement.

An intelligent surveillance system contains several steps, including crowd flow modelling, motion pattern identification, action recognition, object identification and object tracking. Each of these steps is an individual research problem, and crowd flow modelling is vital in creating an intelligent surveillance system. The crowd flow modelling is a mathematical representation of the crowd flow patterns observed in the crowd videos. Crowd flow patterns precisely give us a detailed description of the entire crowd’s flow in the video scene, which accurately models the crowd’s possible usual behaviours and helps the system to identify abnormal flows quickly. Group dynamics is also an essential factor that effects crowd flow motion. The study of group dynamics includes different fields varies from psychology to computer science. The effective representation of these group dynamics is a challenging issue in crowd flow modelling. People show some interesting properties in their movement while moving as part of the crowd. Those movements are not a random movement, and they can be characterized. The characterization of these dynamic properties can be identified using stability analysis, and the proposed work is based on this concept.

In this work, a mathematical model to represent the crowd flow behaviour is proposed. We were able to classify the given input crowd video into one of the eight categories called bottleneck, two variations of lane motion, dispersion, diversion, circular and two types of spiral movements. All these movements are calculated based on the Hessian matrix’s determinant and the Jacobian matrix’s determinant extracted from the Region of Interest (ROI). We have used both the Hessian matrix and the Jacobian matrix to cross-validate the obtained results. A new method to identify the ROI using the optical flow method is also proposed in this work.

The rest of the paper is organised as follows. The literature review is included in Section 2, which deals with the related works and Section 3 describe the steps involved in the proposed method. Section 4 discusses experiments and results and, conclusion and future works are presented in Section 5.

2 Literature survey

Many works are reported in the literature in this area for the last few years. Crowd flow modelling can be classified into different categories based on various parameters [59] that affects the crowd flow motion. Most of them deal with low and medium-density crowds, claiming that they will also work for the high-density crowd. The parameters are different according to the scenario such as view level, the structure of the flow, nature of the background and many more. Crowd flow modelling can be categorised into two classes based on how the modelling is performed. They are micro-level flow modelling and macro-level flow modelling [25]. The micro-level analysis is also known as object-based method and in which each pedestrian or object can be considered as a separate entity. The macro-level analysis is also known as the holistic method. The entire flow can be viewed as a single global entity, and we can analyse and track the features of that global motion present in the crowd. Object-based methods consider local features and holistic approaches to recognise global behaviours to identify the crowd flow present in the video. Microscopic flows are further categorised into four based on the strategy that they are following. They are Force based Models, Cellular Automata Models, Agent-Based Models and Hybrid Models.

Force-based methods calculate individual motion using the force coefficients that can be calculated using different force equations. Force-based models use the continues position representation of individuals to represent the crowd flow. However, it is failed to incorporate the continuum flow laws and theorems available in physics and mathematics to the model. The Boids Model proposed by Reynolds [38] in 1987, which uses the bird flocks representation to model the crowd flow behaviour. This method was able to shape three types of actions like separation, alignment and cohesion. This model paves the path for later research in force-based models. Social Force model [16] proposed by Dirk Helbing and Peter Molnar in 1995, which uses the equation of motion to model the pedestrian behaviours. Social force model considered different features which influence pedestrian behaviour. Some of the crucial factors are the desired path of a pedestrian to the destination, the real way to the target, desired velocity, actual velocity, obstacles, the attractive force between pedestrians etc. This model is considered as the milestone invention in the research of crowd modelling. A modified version of the social force model which can handle panic situations was proposed by Helbing in 2000 [17]. This method considered nine socio-psychological features in addition to all social force aspects. This model was useful up to some extent in low-density crowds. Nevertheless, it is failed to address panic situations in high-density scenarios. Another modification of the social force model was proposed by Zheng [55] in 2002, which considered the individual personalities like patience and impatience to get a more accurate result. Heigeas et al. proposed a physically-based particle model of the crowd behaviour [14], which considers the interaction between two pedestrians defined by linear functions. Modified particle swarm optimisation was proposed by Cheng [5], in which crowd is viewed as a particle system constitutes from individual particles. These models may be failed to calculate the homogeneous behaviour of the flow system [34]. Many variations of Social force models are still being used to model and simulate the crowd behaviour. Qu et al. [37] proposed a modified version of the social force model to analyse the pedestrian dynamics. The proposed model is applied to analyse pedestrian dynamics in a corridor scenario with either unidirectional or bidirectional flow. In the model, the video is divided into a series of Voronoi cells, and the behaviours are calculated using the social force model in each cell.

Cellular automata model is a grid-based approach consisting of some cells. Usually, each compartment accommodates a single pedestrian, and at each step, the pedestrian moves to the one of the neighbouring grid. Grids are typically equal in size, and there is one assumption that all the pedestrian have the same speed. However, in the real-world scenario, the walker’s size and speed may change from person to person. Cellular automata models primarily lie on some predefined rules that guide the entire system. We can predict the individual behaviours using these rules. The cell states may be either empty or filled. If a person travels from one cell to another cell, the first cell’s state becomes vacant, and the second cell’s state becomes filled. Many methods using this concept have been proposed so far. The main difference between those methods is the difference in defining the rules for updating the states. Kirchner and Schadschneider introduced a bionic inspired cellular automata model [26] based on transition probabilities to determine the state changes. This method considers only four adjacent cells instead of eight neighbours. Cellular automata model for the bidirectional flow was proposed by Yeu [54] in 2010, which considers eight neighbours to models the pedestrian flow in two directions. Fang et al. [28] proposed an extended version of the cellular automata model, which uses a potential cost field to represent the pedestrian flow patterns. They also explored the influence of different factors which effects crowd flow and evacuation. Ji et al. [23] proposed a cellular automata model for the high-density crowd’s evacuation in panic situations. They have used the concept of triangle grids, which help the system adapt to the spatial information better. The fuzzy version of cellular automata model was proposed by Gerakakis et al. [10] in 2019.

In the agent-based approach, each pedestrian is considered as a single separate entity called agents. Each agent is not only autonomous but also interactive, intelligent and individual. The agent-based method considers each pedestrian as a single agent, and the system can calculate collective behaviour from the combination of these agent behaviours and each agent must possess the ability to learn and adapt the environment [2]. Most of the methods based on the agent-based system lie on a concept called Belief-Desire-Intention. In this concept, each agent must have a belief, a desire and an intention. Belief means the information processed by the agent, and desire means the aim of the agent and intention is the future task going to be performed by the agent [2, 41]. The main advantage of the agent-based model is the ability to consider individual behaviour heterogeneity. However, this system requires high computational power to maintain these heterogeneities. Because in the agent-based model, we need to calculate each agent or pedestrian’s behaviour in the crowd. The agent-based model can be applied where high-level accuracy is required. Zhou et al. [56] use the dynamic pedestrian agent to teach the pedestrians collective behaviour patterns from the crowded scene. Luo et al. [31] proposed a novel addition to the agent-based model to avoid the pedestrians’ collision. They have investigated two proactive steering behaviour of the crowd, namely gap seeking and following. Zia et al. [58] have incorporated the idea of a grid system with the original agent-based model to improve the model’s efficiency. These models concentrate more on individual behaviours rather than group dynamics [27].

Hybrid models use the combinations of the models mentioned above, and they can perform better than individual models. Some of them are agent-based cellular automata model, force-based agent model and force-based cellular automata model. Agent-based cellular automata model combines the benefits of the agent-based model as well as cellular automata model. Each pedestrian is considered as an agent moving over the grids called cells [18, 30]. Grids are used to locate the pedestrian’s position as agents. It will help the system to access the area of the crowd more precisely. Force-based agent models combine the advantages of both force-based models and agent-based models. These methods make the decision based on agents’ intelligence produced by agent-based models, and corresponding actions are calculated using the force-based models. This method can address both the homogeneous and heterogeneous behaviour of the crowd [19 , 36]. Very few studies are conducted in the area of force-based cellular automata model. It is challenging to combine the force-based and cellular automata models because force-based methods and cellular automata methods follow different crowd representations. One pioneer work based on this concept was proposed by Song et al. [51], which can handle emergencies.

The Macroscopic models consider the crowd as a single global entity instead of considering the crowd as pedestrians with individual behaviour. Macroscopic flows can be classified into four categories. They are Fluid dynamics Models, Regression-based Models, Route Choice Models and Queuing Models. The crowd flow is almost similar to the fluid flow if we watch the flow in a macroscopic way, and this concept is used to model fluid dynamics crowd flow models. The idea of thinking fluid was proposed by Hughes et al. [22], which is considered as the first crowd behaviour analysis based on fluid dynamics. Helbing et al. presented a fluid dynamics based crowd evacuation model, which considers the crowd motion as a fluid motion [15]. Habib et al. [47] used the crowd dynamics model for the coherency detection of the crowd under surveillance. Farooq et al. [9] estimated the global motions present in high-density crowd scenarios using fluid dynamics. All these models are using the same approach, similar to the gas kinetic models.

The regression-based model calculates the mathematical relation between the flow variables using regression methods. Milazzo et al. [33] propose the first crowd flow method based on regression analysis. One of the latest work using this approach was presented by Duieves et al. [7] in 2017. This method calculates the regression between two parameters, like the pedestrians’ interaction and their walking behaviour. In the route choice model, the crowd can select a path, which can maximise their needs. The crowd sets their path based on different influential parameters like comfort, travel time, travel distance etc. The crowd always chooses the way which can maximise the utility [21].

Any system with limited space is usually known as a queueing system. Each environment is defined as the nodes, and the path between the environments are defined using edges. Markov chain model can be used in queuing models to describe the pedestrian’s movement from one node to another. The queuing theory can be used primarily in crowd scenarios to solve the congestion scenes like evacuation scenarios. Lovas et al. [29] propose a pedestrian model based on queuing theory, which models the crowded environment as a network of walkway sections. Razieh et al. [11] use the queuing theorem for the modelling of crowd flow during the pilgrimage at Masjiudul Haram.

Crowd flow is also divided into the structured flow, and unstructured flow based on the crowd movements [12 , 40]. If all the pedestrian in the crowd uniquely flows in the same direction, we can call it a structured motion, and if it is not in the same direction, we can classify it as unstructured motion. It is effortless to identify the crowd behaviours from the structured action because it may contain only one movement type. At the same time, unstructured nature may include different types of behaviours in the same crowd.

The crowd flow is again categorised into two categories based on the background present in the video. They are the flow of the static environment and the stream with a dynamic context. Usually, the surveillance videos may contain static backgrounds in both indoor videos and outdoor videos. However, in some outdoor videos like the videos taken from the seashore, both the foreground and background may be dynamic [46]. An excellent crowd flow method must have the capacity to handle both the static and dynamic scenarios.

Some models in the literature cannot be restricted into any of the categories as mentioned above. Solmaz et al. [44] proposed the crowd behaviour analysis behaviour using Jacobian matrix analysis.They have contributed five different flow patterns namely Bottleneck, Fountainhead, Lanes, Arches and blocking. Chen et al. [4] proposed a method to detect unusual event detection with the help of optical flow and clustering algorithms. In this paper, The optical flow from the crowd video is estimated using the Lukas Kande Optical flow algorithm, and the crowd clusters are detected using adjacency matrix-based clustering. Then these crowd clusters are modelled using the force field method. Hang su et al. [45] uses the concept of Spatio-temporal viscous fluid fields to analyse human behaviour. hey have proposed a Spatio-temporal matrix to analyse the video using the local difference between relative pixels. They have used eigenvalue analysis to classify the crowd behaviour patterns. Ali et al. [1] proposed a system that analyses the flow instability of the system using Lagrangian Particle. A flow map is constructed to set up a deformation tensor. The maximum eigenvalue of the tensor is used to construct a finite-Time Lyapunov Exponent (FTLE) field, which reveals the Lagrangian Coherent Structures (LCS) present in the underlying flow. Cong et al. [6] proposed a crowd flow model using sparse reconstruction cost. Features are extracted using a Multi-scale Histogram of Optical Flow, and a dictionary of crowd behaviours are created using sparse reconstruction cost. Jing Shao [42] considers different group descriptor parameters such as collectiveness, stability, uniformity and conflict to model the crowd flow. Wu et al. [53] proposed a collective density clustering approach, which can detect both the macroscopic and microscopic motions. Shang Wu et al. [52] first uses curl and divergence for crowd behaviour analysis. Wang et al. [48] propose a texture-based approach for real-time crowd abnormal behaviour detection.

The concept of deep neural networks drives recent research in this area [49]. Deep learning frameworks are used to detect abnormalities rather than crowd flow modelling [13, 24]. Hasan et al. [13] proposed a fully-connected encoder with hand-crafted features as input to the system. Zhou et al. [57] used spatio-temporal information for classifying normal and abnormal events. Wang et al. [50] proposed modified generative neural networks for crowd anomaly detection.

The proposed system follows a macroscopic approach, in which the system considers the entire crowd as a single entity.

3 Proposed system

The architecture of the proposed crowd flow modelling is given in Fig. 1. The different steps involved in this proposed crowd modelling system are crowd flow detection, Crowd flow localization, Crowd movement analysis using Hessian matrix and Eigen matrix, and crow flow pattern identification. The proposed intelligent surveillance system takes surveillance videos as the input and identifies the flow region from the footage using The Gunnar-Farneback optical flow Method [8]. The optical flow method gives us the video’s flow fields, and these flow fields can represent the crowd motions present in the surveillance video. The optical flow also results in the extraction of motion coordinates. Some filters, like median filters and morphological filters, are applied to remove the video’s noise. SURF [3] feature extraction method is used to identify the interest points in the video, and these interest points are used to localize the crowd flow. Localization gives us the region where the motion presents and Hessian matrix and Jacobian Matrix are calculated from the extracted area. Corresponding eigenvectors and eigenvalues are also calculated. Based on these calculated eigenvalues and eigenvectors, stability analysis of the video is performed, and the given video is classified into one of the predefined behaviours. Each step involved in the proposed system is explained in detail in the below sections.

Fig. 1

Architecture - intelligence surveillance system.

Let (x, y, t) be the input video, where x and y are the video’s width and height and t be the length of the video.

3.1 Crowd flow detection

Crowd flow detection is the first step in in the proposed crowd behaviour pattern analysis. First, we need to detect the crowd flow present in the video to analyse the crowd flow. The well known optical flow method is used here to catch the crowd flow. The dense optical flow was calculated here instead of usual sparse optical flow to get more accurate patterns. Dense optical flow calculates the optical flow vectors for all the points in the frame. The dense optical flow was calculated using the Gunners - Farneback optical flow algorithm proposed by Gunner Farneback in 2003 [8].

The input video (x, y, t) is given to the optical flow module. The difference between two consecutive frames is Δx, Δy andΔt. The first step in Farneback optical flow algorithm is the approximation of the neighbourhood using quadratic polynomials. The pixel values in a region of the image are represented by

$f (x) \sim x^{T} Ax + b^{T} x + c$ (1)

Where A is the symmetric matrix, b is a vector and c is a scalar value. So polynomial expansion of at current frame (x, y) can be expressed as

$f_{1} (x) \sim x^{T} A_{1} x + b_{1}^{T} x + c_{1}$ (2)

Suppose d is the displacement for each frame. Then new value f₂ is the polynomial expansion at time t2, and it is calculated using the following equation.

$f_{2} (x) = f_{1} (x - d)$ (3) $= (x - d)^{T} A_{1} (x - d) + b_{1}^{T} (x - d) + c_{1}$ (4) $= x^{T} A_{1} x + (b_{1} - 2 A_{1} d)^{T} + d^{T} A_{1} d - b_{1}^{T} + c_{1}$ (5) $= x^{T} A_{2} x + b_{2}^{T} + c_{2}$ (6)

Equating coefficients in the two polynomials of consecutive frames.

$A_{2} = A_{1}$ (7) $b_{2} = (b_{1} - 2 A_{1} d)$ (8) $c_{2} = d^{T} Ad - b_{1}^{T} d + c_{1}$ (9)

Assuming A is non-singular.

$d = - \frac{1}{2} A_{1}^{- 1} (b_{2} - b_{1})$ (10)

The global displacement vectors are calculated by equating the coefficients from f₁ (x) and f₂ (x) at each point in the frame. These differences are marked as the flow vectors in a coloured manner, and the input frame and corresponding motion detected frames are shown in the Fig. 2 Optical flow calculation step produces the motion vectors and flows fields, and this is used for the remaining process. This step will have coloured pixels locations where some flows are present.

Fig. 2

Crowd flow detection.

3.2 Crowd flow localization

The second step in the proposed system is crowd flow localization. When the crowd flow is detected in the previous step, our interest moves from the entire frame to the area where the motion is present. We have used SURF feature extraction mechanism for Crowd flow Localization. This step’s input is the flow detected frames and frames are represented as (x, y, t), and these key points are obtained only where the flow is present. These key points help us to calculate the ROI.

Speeded up robust features (SURF) [3] is used in this pattern identification system to extract key points from the flow image. SURF is more efficient than other feature detectors like SIFT, blob detector and MSER in our study. SURF algorithm has two parts, The first part of the algorithm is interest point detection, and the second part is the interest point description generation. SURF detects interest points using the Hessian matrix approximation of Integral images. Integral image at a particular location X = (x, y) ^T can be calculated using

$I_{Σ} (X) = \sum_{i = 0}^{i \leq x} \sum_{j = 0}^{j \leq y} I (i, j)$ (11) where I_Σ (X) is the calculated integral image. The determinant of the Hessian matrix is then calculated for the integral images.Hessian matrix at a given point X = (x, y) with scale factor σ is calculated using

$H (x, σ) = [\begin{matrix} L_{xx} (x, σ) & L_{xy} (x, σ) \\ L_{xy} (x, σ) & L_{yy} (x, σ) \end{matrix}]$ (12) where H (x, σ is the Hessian matrix and and L_xx, L_xy and L_yy are Gaussian second-order derivatives. A 3x3 box filter is used to estimate the blob response from the Hessian matrix, and it is denoted as D_xx, D_xy and D_yy. Then the determinant of the Hessian Matrix is calculated as

$\det (H) = D_{xx} D_{xy} - ω D_{yy}$ (13) where ω is the relative weight of responses.

SURF detects interest points in all scales using scale-space representations. Scale-space representation is an essential step in SURF feature detection which can extract interest points in all scales. Scale-space representation method uses octaves and box filters to identify these key points. Then the SURF algorithm needs to localize all these identified key points. SURF localize all these key points using a unique mechanism called non-maximum suppression in a 3 × 3 neighbourhood manner. SURF uses a 64-bit descriptor to describe and represent the SURF key points. The descriptor contains the orientation assignment details and the responses based on the Haar wavelet transform. This information in the SURF descriptors helps the SURF algorithm to match the key points in two different images. This step produces the interest points or key points only at the flow coordinates. These key points will help us to localize the crowd flow efficiently.

Crowd localisation is performed here using the ROI concept. It is defined as the area of an image or sub-image extracted from the original image for a particular purpose. In this problem, the ROI is the sub-image where crowd flow occurs, because our attention is now restricted to analyse crowd flow patterns.

The ROI can be calculated using the SURF keypoints extracted in the previous step. SURF extracts more key points in active areas. So ROI is always nearer to this dynamic area in the video. The centre of ROI is calculated using the global mean of SURF key points. All the SURF key points have x and y values to represent its centre values. The average of all the x and y values gives us the global average of all key points and these global value can be considered as the centre of ROI. A rectangle enclosing most of the key points with the centre as the global centre of the key points will give us the ROI. This ROI help us to localize the crowd flow, and it is shown in Fig. 3.

Fig. 3

Crowd flow localization.

3.3 Flow stability analysis

After localising the crowd behaviour, we need to analyse the crowd videos to get the crowd behaviour patterns. Hessian matrix and Jacobian matrix are used here for crowd behaviour analysis. Hessian Matrix and Jacobian Matrix are calculated only for the ROI, not for the entire frame.

Hessian matrix is a square matrix of second-order partial derivatives of an image. Let I be the ROI extracted from the video, and then the Hessian matrix is calculated using $H_{roi} = [\begin{matrix} \frac{\partial^{2} I}{\partial x^{2}} & \frac{\partial^{2} I}{\partial x \partial y} \\ \frac{\partial^{2} I}{\partial x \partial y} & \frac{\partial^{2} I}{\partial y^{2}} \end{matrix}]$ (14)

Where $\frac{\partial^{2} I}{\partial x^{2}}$ is the second-order partial derivative of ROI with respect to x direction, $\frac{\partial^{2} I}{\partial y^{2}}$ is the second-order partial derivative of ROI with respect to y direction and $\frac{\partial^{2} I}{\partial x \partial y}$ is the second-order partial derivative of ROI with respect to both xy and y directions.

Once the Hessian is calculated, Our system calculates Jacobian of ROI [43]. Jacobian is a square matrix of all first-order derivative of the image. Jacobian Matrix of ROI is calculated using

$J_{roi} = [\begin{matrix} \frac{\partial u}{\partial x} & \frac{\partial u}{\partial y} \\ \frac{\partial v}{\partial x} & \frac{\partial v}{\partial y} \end{matrix}]$ (15)

Where u and v are the optical vectors in horizontal and vertical directions. The crowd flow patterns will be identified based on the Hessian matrix and Jacobian matrix eigenvalues.

Eigenvalues [20] are the set of values derived from linear systems of equations. Eigenvalues are also known as characteristic roots because these values can represent the system’s actual nature or character. The eigenvalues can be mainly used in image processing and computer vision problems to identify the image’s behaviour. The eigenvalues of the Hessian matrix are usually used to describe the behaviour of the matrix. Here the Hessian matrix represents crowd flow, and its eigenvalues can define the structure of the crowd flow.

Let H_roi be the Hessian matrix calculated from the ROI matrix in the previous step. then the eigenvalues are calculated using

$H_{roi} - λ I = 0$ (16)

Where I is the identity matrix, and λ is the eigenvalue matrix. Two eigenvalues are obtained for a 2x2 matrix, and we can denote those values as λ₁ and λ₂. The eigenvalues of the hessian matrix are used to describe the structure of the matrix. Here the structure of the matrix means the structure of the crowd flow. If both the eigenvalues are positive and it represents the concave up behaviour of the matrix. If both the eigenvalues are negative, then it means the concave-down nature of the matrix. If the values are 0, we cannot predict anything from the matrix. If the values are mixed, that means one value is positive, and another value is negative, then it is a saddle point.

Eigenvalues of Jacobian matrix need to be calculated for identifying the motion patterns from the surveillance video. J_roi is the Jacobian matrix calculated in the previous step, and then its eigenvalues are calculated using the equation

$J_{roi} - λ I = 0$ (17)

Where I is the identity matrix, and λ is the eigenvalue matrix. Similar to the Hessian matrix, two eigenvalues are calculated from the Jacobian matrix. The Jacobian matrix’s eigenvalues are also used to describe the structure of the crowd flow obtained from the surveillance video. Based on the values of eigenvalues, we can classify the input video into different behavioural patterns.

3.3.1 Jacobian analysis

We can apply linear stability analysis to continuous-time non-linear dynamical systems. Consider the dynamics of a non-linear differential equation.

$\frac{dx}{dt} = F (x)$ (18)

around its equilibrium point x_eq. By definition, x_eq satisfies

$F (x_{eq}) = 0$ (19)

To analyse the system’s stability around this equilibrium point, we do the same coordinate switch for discrete-time models. Specifically, we apply the following replacement.

$x (t) \Rightarrow x_{eq} + Δ x (t)$ (20)

then we will get

$\frac{d (x_{eq}) + Δ x}{dt} = \frac{Δ x}{dt} = F (x_{eq} + Δ x)$ (21)

Now that we know the non-linear function F on the right-hand side is approximated using the Jacobian matrix, the equation above is approximated as

$\frac{d Δ x}{dt} \approx F (x_{eq}) + J Δ x$ (22)

where J is the Jacobian matrix of F at x = x_eq. By combining the results, We obtain

$\frac{d Δ x}{dt} \approx J Δ x$ (23) Calculate the eigenvalues of F and interpret the results to determine the stability of equilibrium point x_eq.

3.3.2 Hessian analysis

Suppose the function z = f (x, y) has continuous second partial derivative. Let (x₀, y₀) be a critical point of f, and let λ₁ and λ₂ be the eigenvalues of H_f (x₀, y₀).

If λ₁ < 0 and λ₂ < 0, then f has a local maximum value at (x₀, y₀).

If λ₁ < 0 and λ₂ > 0 or λ₁ > 0 and λ₂ < 0, then f has a saddle point at (x₀, y₀)

If λ₁ > 0 and λ₂ > 0, then f has a local minimum value at (x₀, y₀).

If λ₁ = 0 and λ₂ = 0, then no conclusions can be drawn.

3.4 Crowd behaviour pattern identification

Crowd behaviour pattern Identification is the final step in the proposed system. Based on the Hessian Matrix and Jacobian matrix’s eigenvalues, we can classify the crowd behaviour into one of the behaviour patterns shown in the Fig. 4. The eight different categories we have used in our problem are eight different categories named Bottleneck, Dispersion, Lane to a common point, Lane to the opposite directions, Diversion, Circular, Converging spiral and Diverging Spiral. Five of them are proposed by [44], and we newly identify the remaining new three.

Fig. 4

Different types of motion patterns.

We need to interpret the eigenvalues of both the Hessian and the Jacobian to predict the motion patterns. λ_H1 and λ_H2 are the eigenvalues calculated from the Hessian matrix in the previous step. Hessian matrix makes an initial assumption about the flow model, and the Jacobian matrix makes the final classification.

Hessian matrix is the square matrix representation of second-order derivation of the system. We can consider the entire crowd flow as a system, and the Hessian matrix represents the flow system. Eigenvalues are calculated from the flow system to make the first level classification of the given input video. We have eight predefined class in our pattern identification system as given in the 4, and we will make a decision based on the nature of Hessian eigenvalues. The eigenvalues may be either positive, negative or zero. We will get such two eigenvalues for a 2 × 2 Hessian matrix. The real part of the eigenvalues may be negative, positive or mixed. If both of the eigenvalues are positive, then the system is elliptic paraboloid with a local minimum, and the system is stable. So we have two firm motion patterns in our system, and we can say that the given video may follow either a bottleneck, Lane to the same side or converging spiral pattern. Suppose both the eigenvalues of the Hessian matrix are negative. In that case, the system is unstable, and corresponding flow may be the dispersion, lane to the opposite side or diverging sphere. If the eigenvalues are mixed, that means one is positive and other is negative, then the system is in a saddle condition, and the input video pattern may be in any of the two patterns.

The flow patterns are also calculated from the eigenvalues of the Jacobian matrix. λ_J1 and λ_J2 are the eigenvalues calculated from the Jacobian matrix in the previous step. Based on the value and sign of the eigenvalues, we can classify the input surveillance video into eight different crowd motion patterns as given in the Fig. 4.

When both the eigenvalues λ_J1 and λ_J2 are less than zero, that means both of them are a real negative number; then the Jacobian matrix is in a stable state. In a stable system, the flow happens to a centre point from different locations in the scene. It is similar to the crow flow from different directions to a common centre of attraction or a junction. If the flow is too high to a common point without any control, it is complicated to handle that scenario without congestion. That situation can be called as the bottleneck. So when the values λ_J1 < 0 and λ_J2 < 0, we can classify the given video in to bottleneck class.

When both the eigenvalues of the calculated Jacobian matrix are positive and real numbers, the system is unstable. In an unstable system, the force moves from one central point to all outer directions. For example, when a public event or meeting is being happened, the crowd usually gathers near the venue, and the crowd begins to disperse to all directions once the meeting is over. We can call this type of motion as dispersion. So when the values λ_J1 > 0 and λ_J2 > 0, we can classify the given video in to dispersion class.

If one of the eigenvalues is zero, then the direction of force occurs in a line, because only one value is contributing to deciding the flow direction. The direction force is almost like a line motion, but not exactly a straight line. We can call it as a Lane motion. There are two types of lane motions based on the second eigenvalue. If the second value is negative, then the force is to the same direction from the two sides. Otherwise, the force is to the opposite direction in a lane manner.

If the two eigenvalues are there with the different sign, that means one is with the positive sign and another with negative signs; then the system is unstable. However, this unstable system’s structure is entirely different from the structure of unstable system we have discussed in the bottleneck scenario. In this method, the flow comes from different sides and reaches the centre point and divert to the centre’s desired direction. This type of behaviour is known as diversion. The diversion will not create a bottleneck in the junctions. Instead, there is continues flow in the junctions.

Eigenvalues usually in the form α ± iβ, where α is the real part, and β is the complex part of the eigenvalues. when α = 0, then the eigenvalue contains only complex part. If the eigenvalue contains the only real part, then the system follows the circular flow of forces. It is equivalent to the circular motion of crowd around an object an the motion may be in a clockwise or anti-clockwise direction depending on the sign of the complex part of the eigenvalue.

If the eigenvalues contain both the real and complex parts, then the flow is similar to a spiral. The spiral flow can be classified into two categories based on the real part of the eigenvalue value. If the real part of the eigenvalues is negative, then the spiral is converging, and all the flow reached into the centre position after entering into the scene. If the real part of the eigenvalue is positive, then the obtained shape is a diverging spiral and in which the flow starting from the centre and slowly diverged into the outer layers. All the conditions discussed above are summarized and given in the Table 1.

Table 1

Classification rules

Hessian	Jacobian	Behaviour
λ_H1 > 0, λ_H2 > 0	λ_J1 < 0, λ_J2 < 0	Bottleneck
λ_H1 < 0 λ_H2 < 0	λ_J1 > 0, λ_J2 > 0	Dispersion
λ_H1 < 0 λ_H2 < 0	λ_J1 = 0, λ_J2 > 0	Lane to the
		opposite direction
λ_H1 > 0 λ_H2 > 0	λ_J1 = 0, λ_J2 < 0	Lane to the same side
λ_H1 > 0 λ_H2 < 0 or	λ_J1 > 0, λ_J2 < 0 or	Diversion
λ_H1 < 0 λ_H2 > 0	λ_J1 < 0, λ_J2 > 0
λ_H1 > 0 λ_H2 < 0 or	n the form of α + iβ	Circular
λ_H1 < 0 λ_H2 > 0	and α = 0
λ_H1 > 0, λ_H2 > 0	In the form of α + iβ	Converging Spiral
	and α < 0
λ_H1 < 0 λ_H2 < 0	In the form of α + iβ	Diverging Spiral
	and α > 0

Based on the characteristics of eigenvalues of Hessian matrix and Jacobian matrix, we can classify the given input video into eight different categories named Bottleneck, Dispersion, Lane to a common point, Lane to the opposite directions, Diversion, Circular, Converging spiral and Diverging Spiral as we have discussed above.

4 Results and analysis

Experiments are conducted with the videos from UCF crowd video dataset [1], CUHK Crowd Dataset [42]. Ten videos of each class are identified and tested with the proposed algorithm to evaluate the system. The given input video is classified into one of the eight predefined categories based on the eigenvalue conditions shown in Table 1. Optical flow parameters and surf features are extracted from the input video and ROI are calculated from those extracted features. Then, the Hessian matrix and Jacobian matrix’s eigenvalues are calculated, and the decision is made based on those eigenvalues. The decision rules are given in Table 1, and the input video is classified into one of the predefined motion patterns. Some results are shown in Fig. 6.

Fig. 5

Behaviour pattern.

Fig. 6

Classification result.

To evaluate the performance of the proposed work, we compared our results against manually generated ground truth. A confusion matrix is created from the results, and it is given in Table 2. The true positive, true negative, false positive, and false negative values are calculated for each class to evaluate the system. Usually, these are the parameters to assess binary classifier, and we are extending those methods to calculate true positive, true negative, false positive, false negative values of the multi-classifier.

Table 2

Confusion matrix

	Bottle neck	Dispersion	Lane to the opposite direction	Lane to the same Direction	diversion	Circular	Converging Spiral	Diverging Spiral
Bottleneck	9	1	0	0	0	0	0	0
Dispersion	1	9	0	0	0	0	0	0
Lane to the opposite direction	1	0	8	0	1	0	0	0
Lane to the same direction	0	0	1	8	1	0	0	0
Diversion	1	1	0	0	8	0	0	0
Circular	0	0	0	0	0	9	1	1
Converging Spiral	0	0	0	0	0	0	8	2
Diverging Spiral	0	0	0	0	0	0	2	8

Then the Precision, recall, F Score and accuracy values are calculated for each class and given in Table 3. Average precision, Average recall, Average F Score and Average accuracy need to be calculated to evaluate the entire system’s performance. The average values are given in Table 4. We got the Average precision as 0.56, Average recall as 0.56, Average F Score as 0.56 and Average accuracy as 0.89. The performance of each class is diagrammatically represented in the graph given in Fig. 7. The entire system is implemented in Python using OpenCV 3.2. The system runs in Intel Core i5 processor with Quadro GPU.

Table 3

Performance measures

	True Positive	Tru Negative	False Negative	False Positive	Precision	Recall	F-score	Accuracy
Bottleneck	9	67	1	3	0.9	0.75	0.82	0.95
Dispersion	9	68	1	2	0.8	0.82	0.86	0.96
Lane to the opposite direction	8	69	1	2	0.8	0.88	0.84	0.96
Lane to the same direction	8	70	2	0	0.8	1.0	0.89	0.97
Diversion	8	68	2	2	0.8	0.8	0.8	0.95
Circular	9	70	1	0	0.9	1.0	0.94	0.99
Converging Spiral	8	67	2	3	0.8	0.73	0.76	0.94
Diverging Spiral	8	67	2	3	0.8	0.73	0.76	0.94

Table 4

Average performance

Average Precision	Average Recall	Average F Score	Average Accuracy
0.56	0.56	0.56	0.89

Fig. 7

Performance analysis.

We have used a dense optical flow method in our problem to detect the crowd flow motion. If we use sparse optical flow method instead of dense optical flow, we could not detect the entire crowd flow present in the video, and we got only some points that cannot represent the entire crowd. We have also used the SURF feature detection mechanism to extract the crowd features from the video. We have tested our problem with other feature extraction mechanisms like SIFT and MSER. SIFT and MSER are computationally complex than SURF, and those methods take more time to execute. So SURF is preferred here to extract the features.

Our results are compared with existing crowd flow models in the literature. Berkan Solmaz’s [44] crowd flow model with the help of stability analysis is one of the pioneering work in this field. They were able to model only five behaviours. Our work, we have proposed and implemented eight different crowd behaviour patterns. We have compared the average accuracy of all those behaviours with our model’s average accuracy. The average accuracy of their model is 65%, and our method achieves 89% accuracy. Similarly, we have compared our approach with similar works in the literature, and the comparison is displayed in the Table 5. From the table, we can conclude that our method outperforms all the similar works in the literature. Our work is better than all the results we compared because our work uses both Hessian and Jacobian analysis strategies. This two-step analysis helps the system to get a more accurate result and our ROI method helps us to get the output faster than current research.

Table 5

Comparison of accuracy

Methods	Accuracy
Histogram of Optical flow	0.56 [4]
Stability Analysis	0.65 [44]
Spatio - Temporal Features	0.78 [45]
Lagrangian Particle Trajectory	0.81 [1]
Social Force	0.81 [16]
Sparse Reconstruction Cost	0.84 [6]
Force Field	0.85 [37]
Proposed Method	0.89

5 Conclusion and future work

We have presented an architecture for detecting crowd behaviour patterns from videos. Based on the matrix analysis of Jacobian and Hessian, the input videos are assigned into one of the eight categories. Addressing the eight categories from the wide variety of videos is the main advantage of our work. The system could achieve considerably better performance than the works reported in the literature. We have also proposed a new method for ROI calculation. Instead of calculating the crowd behaviour for the entire frame, we have done the analysis only for the calculated ROI. This technique helps us to reduce the number of pixels to be processed. Our architecture can detect only eight behaviours far from the complexities present in the crowd dynamics. The inclusion of more real-world scenarios and crowd flow models will improve the system’s effectiveness. In the future, the proposed model can be combined with deep learning approaches to predict abnormal behaviours present in the surveillance videos.

References

Ali

and Shah

, A lagrangian particle dynamics approach for crowd flow segmentation and stability analysis, 2007 IEEE Conference on Computer Vision and Pattern Recognition, pages 1–6, 2007.

Bandini

, Manzoni

and Vizzari

, Agent based modeling and simulation: An informatics perspective, Journal of Artificial Societies and Social Simulation 12(4) (2009), 4.

Bay

, Ess

, Tuytelaars

and Gool

L.V.

, Speeded-up robust features (surf), Comput Vis Image Underst 110(3) (2008), 346–359.

Chen

D.-Y.

and Huang

P.-C.

, Motion-based unusual event detection in human crowds, Journal of Visual Communication and Image Representation 22(2) (2011), 178–186.

Cheng

, Bo

, Lijun

and Hua

, A modified particle swarm optimization-based human behavior modeling for emergency evacuation simulation system, In 2008 International Conference on Information and Automation, pages 23–28, June 2008.

Cong

, Yuan

and Liu

, Sparse reconstruction cost for abnormal event detection, pages 3449–3456, 07 2011.

Duives

, Daamen

and Hoogendoorn

, Operational walking dynamics of crowds modelled by means of linear regression, page 1, 2017.

Farnebäck

, Two-Frame Motion Estimation Based on Polynomial Expansion, pages 363–370, Springer Berlin Heidelberg, Berlin, Heidelberg, 2003.

Farooq

M.U.

, Saad

M.N.B.M.

, Malik

A.S.

, Ali

Y.S.

and Khan

S.D.

, Motion estimation of high density crowd using fluid dynamics, The Imaging Science Journal 68(3) (2020), 141–155.

10.

Gerakakis

, Gavriilidis

, Dourvas

N.I.

, Georgoudas

I.G.

, Trunfio

G.A.

and Sirakoulis

G.C.

, Accelerating fuzzy cellular automata for modeling crowd dynamics, Journal of Computational Science 32 (2019), 125–140.

11.

Haghighati

and Hassan

, Modeling the flow of crowd during tawaf at masjid al-haram, 2013.

12.

Hao

, Xu

, Wang

, Liu

and Fan

, An approach to detect crowd panic behavior using flow-based feature, 2016 22nd International Conference on Automation and Computing (ICAC), pages 462–466, 2016.

13.

Hasan

, Choi

, Neumann

, Roy-Chowdhury

A.K.

and Davis

L.S.

, Learning temporal regularity in video sequences, CoRR, abs/1604.04574, 2016.

14.

Heigeas

, Luciani

, Thollot

and Castagné

, A physically-based particle model of emergent crowd behaviors, In Graphicon, 2003.

15.

Helbing

, Farkas

, Molnàr

and Vicsek

, Simulation of pedestrian crowds in normal and evacuation situations, In M. Schreckenberg and S.D. Sharma, editors, Pedestrian and Evacuation Dynamics, pages 21–58, Berlin, 2002. Springer.

16.

Helbing

and Molnar

, Social force model for pedestrian dynamics, Physical Review E 51 (1998), 4282–4286.

17.

Helbing

, Farkas

and Vicsek

, Simulating dynamical features of escape panic, Nature, page 2000.

18.

Heliovaara

, Korhonen

, Hostikka

and Ehtamo

, Counterflow model for agent-based simulation of crowd dynamics, Building and Environment 48(Supplement C) (2012), 89–100.

19.

Henein

C.M.

and White

, Agent-Based Modelling of Forces in Crowds, pages 173–184, Springer Berlin Heidelberg, Berlin, Heidelberg, 2005.

20.

Hoffman

and Kunze

, Characteristic Values, Englewood Cliffs, NJ: Prentice-Hall, 1971.

21.

Hoogendoorn

S.P.

and Bovy

P.H.L.

, Pedestrian travel behavior modeling, Networks and Spatial Economics 5(2) (2005), 193–216.

22.

Hughes

R.L.

, The flow of large crowds of pedestrians, Math Comput Simul 53(4-6) (2000), 367–370.

23.

, Lu

, Jin

, Wei

and Ni

, A cellular automata model for high-density crowd evacuation using triangle grids, Physica A: Statistical Mechanics and its Applications 509 (2018), 1034–1045.

24.

Jia

, Shao

and Fu

, Sparse canonical temporal alignment with deep tensor decomposition for action recognition, IEEE Transactions on Image Processing 26(2) (2017), 738–750.

25.

Silveira

J.C.

, Junior

, Musse

S.R.

and Jung

C.R.

, Crowd analysis using computer vision techniques, IEEE Signal Processing Magazine 27(5) (2010), 66–77.

26.

Kirchner

and Schadschneider

, Simulation of evacuation processes using a bionics-inspired cellular automaton model for pedestrian dynamics, Physica A Statistical Mechanics and its Applications 312 (2002), 260–276.

27.

Kountouriotis

, Thomopoulos

S.C.A.

and Papelis

, An agent-based crowd behaviour model for real time crowd behaviour simulation, Pattern Recognition Letters 44(Supplement C) (2014), 30–38. Pattern Recognition and Crowd Analysis.

28.

, Guo

, Kuang

, Geng

and Fan

, An extended cost potential field cellular automaton model for pedestrian evacuation considering the restriction of visual field, Physica A: Statistical Mechanics and its Applications 515 (2019), 47–56.

29.

Lovas

G.G.

, Modeling and simulation of pedestrian traffic flow, Transportation Research Part B: Methodological 28(6) (1994), 429–443.

30.

Lubaś

, Wąs

and Porzycki

, Cellular automata as the basis of effective and realistic agent-based models of crowd behavior, The Journal of Supercomputing 72(6) (2016), 2170–2196.

31.

Luo

, Chai

, Ma

, Zhou

and Cai

, Proactivecrowd: Modelling proactive steering behaviours for agent-based crowd simulation, Computer Graphics Forum 37(1) (2018), 375–388.

32.

Luo

, Chai

, Zhou

and Ma

, Modeling gap seeking behaviors for agent-based crowd simulation, In Proceedings of the 29th International Conference on Computer Animation and Social Agents, CASA ’16, pages 37–43, New York, NY, USA, 2016. ACM.

33.

Milazzo

, Rouphail

, Hummer

and Allen

, Effect of pedestrians on capacity of signalized intersections, Transportation Research Record: Journal of the Transportation Research Board 1646 (1998), 37–46.

34.

Ghani

N.M.

, Selamat

, Khamis

and Ghazalli

S.A.

, Crowd modelling validation for modified social force model, International Journal of Integrated Engineering 12(2) (2020), 10–18.

35.

Qian

, Wu

and Xu

, Intelligent Surveillance Systems, Intelligent Systems, Control and Automation: Science and Engineering. Springer Netherlands, 2011.

36.

Qiu

and Hu

, Modeling group structures in pedestrian crowd simulation, Simulation Modelling Practice and Theory 18(2) (2010), 190–205.

37.

, Xiao

, Wu

, Tang

and Gao

, Modeling detour behavior of pedestrian dynamics under different conditions, Physica A: Statistical Mechanics and its Applications 492 (2018), 1153–1167.

38.

Reynolds

C.W.

, Flocks, herds and schools: A distributed behavioral model, In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’87, pages 25–34, New York, NY, USA, 1987. ACM.

39.

Rodriguez

, Ali

and Kanade

, Tracking in unstructured crowded scenes, In 2009 IEEE 12th International Conference on Computer Vision, pages 1389–1396, Sept 2009.

40.

Roy

, Biswas

, Saha

S.K.

and Chanda

, Classification of moving crowd based on motion pattern, In 2019 IEEE Region 10 Symposium (TENSYMP), pages 102–107, 2019.

41.

Rybokonenko

, Balakhontceva

, Voloshin

and Karbovskii

, Agent-based modeling of crowd dynamics on a moving platform, Procedia Computer Science 66(Supplement C) (2015), 317–327. 4th International Young Scientist Conference on Computational Science.

42.

Shao

, Loy

C.C.

and Wang

, Scene-independent group profiling in crowd, In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’14, pages 2227–2234, Washington, DC, USA, 2014. IEEE Computer Society.

43.

Simon

C.P.

and Blume

, Mathematics for Economists, Norton, 1994.

44.

Solmaz

, Moore

B.E.

and Shah

, Identifying behaviors in crowd scenes using stability analysis for dynamical systems, IEEE Transactions on Pattern Analysis and Machine Intelligence 34(10) (2012), 2064–2070.

45.

, Yang

, Zheng

, Fan

and Wei

, The large-scale crowd behavior perception based on spatio-temporal viscous fluid field, IEEE Transactions on Information Forensics and Security 8(10) (2013), 1575–1589.

46.

Tsai

Y.-T.

, Shih

H.-C.

and Huang

C.-L.

, Multiple human objects tracking in crowded scenes, In, 18th International Conference on Pattern Recognition (ICPR’06) 3 (2006), 51–54.

47.

Ullah

, Uzair

, Ullah

, Khan

, Ahmad

and Khan

, Density independent hydrodynamics model for crowd coherency detection, Neurocomputing 242 (2017), 28–39.

48.

Wang

and Xu

, Spatio-temporal texture modelling for real-time crowd anomaly detection, Computer Vision and Image Understanding 144 (2016), 177–187. Individual and Group Activities in Video Event Analysis.

49.

Wang

, Chen

, Zhang

, Chen

and Snoussi

, Internal transfer learning for improving performance in human action recognition for small datasets, IEEE Access 5 (2017), 17627–17633.

50.

Wang

, Qiao

, Lin

, Li

, Snoussi

, Liu

and Choi

, Generative neural networks for anomaly detection in crowded scenes, IEEE Transactions on Information Forensics and Security 14(5) (2019), 1390–1399.

51.

Wei-Guo

, Yan-Fei

, Bing-Hong

and Wei-Cheng

, Evacuation behaviors at exit in ca model with force essentials: A comparison with social force model, Physica A: Statistical Mechanics and its Applications 371(2) (2006), 658–666.

52.

, Su

, Yang

, Zheng

, Fan

and Zhou

, Bilinear dynamics for crowd video analysis, J Vis Comun Image Represent 48(C) (2017), 461–470.

53.

, Ye

and Zhao

, Coherent motion detection with collective density clustering, In Proceedings of the 23rd ACM International Conference on Multimedia, MM ’15, pages 361–370, New York, NY, USA, 2015. Association for Computing Machinery.

54.

Yue

, Guan

, Zhang

and Shao

, Study on bi-direction pedestrian flow using cellular automata simulation, Physica A: Statistical Mechanics and its Applications 389(3) (2010), 527–539.

55.

Zheng

, Kashimori

and Kambara

, A model describing collective behaviors of pedestrians with various personalities in danger situations, In, Neural Information Processing, 2002. ICONIP ’02. Proceedings of the 9th International Conference on 4 (2002), 2083–2087.

56.

Zhou

, Tang

and Wang

, Learning Collective Crowd Behaviors with Dynamic Pedestrian-Agents, International Journal of Computer Vision 111(1) (2015), 50–68.

57.

Zhou

, Shen

, Zeng

, Fang

, Wei

and Zhang

, Spatial–temporal convolutional neural networks for anomaly detection and localization in crowded scenes, Signal Processing: Image Communication 47 (2016), 358–368.

58.

Zia

and Ferscha

, An agent-based model of crowd evacuation: Combining individual, social and technological aspects, In Proceedings of the 2020 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation, SIGSIM-PADS ’20, pages 129–140, New York, NY, USA, 2020. Association for Computing Machinery.

59.

Zitouni

M.S.

, Bhaskar

, Dias

and Al-Mualla

M.E.

, Advances and trends in visual crowd analysis: A systematic survey and evaluation of crowd modelling techniques, Neurocomputing 186(Supplement C) (2016), 139–159.