Intrusion Detection and Cooperative Tracking Using PTZ Network Thermal Imagers

2.1. Tracking Thread

Modern detectors tend to locate key points by simply examining the intensities of certain pixels around the tentative locations in order to guarantee sufficient number of candidate key points [12–14]. Following the methodology similar to the algorithms proposed in [12], we consider only the intensities along a circle of 16 pixels around each candidate key point.

We locate a candidate key point at a pixel P if the intensities of at least 12 contiguous pixels are all above (negative) or all below (positive) the intensity of P over a certain threshold. Usually, a featureless key point can be rejected quickly without scanning the entire circle.

Once a key point has been located, the intensities of every three contiguous pixels on the discrete circle are weightedly summed, resulting in a total of 16 sums. The 16 sums are utilized to construct a compact descriptor D. In fact, the intensity is substituted by the gradient G from the point on the circle to its center P in order to resist the illumination change. Assuming that R i is a pixel on the circle R, R i cw and R i ccw are the neighboring points in the clockwise and counterclockwise directions of R, respectively. Each element in the descriptor D can be described as

D i = a · G R i cw → P + b · G R i → P + c · G R i ccw → P i ∈ 1,16 .

(1)

The largest sum is chosen as the first element of the descriptor, and the vector is filled with the remaining sums of the circle in a clockwise direction. This simple process is sufficient for rectifying the detected key point with respect to in-plane rotation. Moreover, it considerably reduces the computational cost in calculating the orientation of the small image patch and sorting the descriptor elements using the mean squared deviation (MSD) [15]. Then, the descriptor vector is normalized in order to remove the variance of contrast. The polarity of every descriptor is also recorded such that the key points can be categorized as positive or negative. This partition is efficient for matching since positive features need not be compared with negative ones.

In the matching section of the tracking thread, the hybrid SP-Tree is employed for its high efficiency [16]. Finally, RANSAC, as the effective way of eliminating spurious matches, should not be omitted [17]. The authors do not intend to give a detailed description here, as similar work can be found in [11, 18].

2.2. Mapping Thread

In this thread, a map consists of a collection of M key points located in a world coordinate frame. Each key point represents a locally planar textured patch in the world. The map also contains N key frames. These key frames are the snapshots taken by a thermal imager at various points in time. A map usually contains 2000 to 6000 points and 40 to 120 key frames.

Once an initial map is built using a stereo method, the map is continually refined and expanded as new key frames are added by the tracking thread. This thread is partially realized by PTAM. For more details, please refer to [8, 9].

3.1. Intrusion Detection

The intrusion detection module is used for real-time alerting in case of intrusion into preset surveillance zone. There are many approaches for intrusion detection in a continuous video stream. They are mainly based on motion detection, namely, comparing the current frame with one from the previous frames. We could get a binary image by comparing each pixel between current frame and previous frame. It is possible to count the number of pixels, and if the amount is greater than a predefined alarm level we can signal about a motion event.

It should be noted that moving objects, such as pedestrians and automobiles, usually have higher intensities compared with the background in grey-level pseudocolour mode of thermal imager. According to this phenomenon, the negative impact coming from the environmental change in FOV, such as the changing shadows and the trembling of leaves, can be eliminated. Figure 1(b) shows the reconstructed frame after foreground segmentation of Figure 1(a).

OPEN IN VIEWER

Figure 1

Background segmentation. (a) A frame acquired by the thermal imager and (b) the frame after background segmentation.

After compensating the FOV offset obtained from the real-time FOV matching, it can facilitate a reliable detection of unauthorised accesses under arbitrary weather conditions.

3.2. Cooperative Tracking

When an intrusion event is detected, the related PTZ thermal imager tracks it using an improved particle filtering method and estimates the pose of the imager simultaneously utilizing the information obtained from FOV matching module. Once the intrusion object enters an overlapping FOV of two imagers over the network, the handoff thermal imager is activated immediately according to the spatial relationship built by pose estimation and turns to tracking and pose estimation status. Meanwhile, the previous thermal imager returns to waiting status, waiting for reactivation from a new cooperative tracking event or a new intrusion event. Figure 2 shows the flowchart of the proposed cooperative tracking framework.

OPEN IN VIEWER

Figure 2

Flowchart of cooperative tracking module.

Particle filtering is an effective approach for object tracking [19]. The core of particle filtering is to approximate the relevant distributions with random measures composed of particles. For the reason that the particle swarm optimization algorithm can search the sample area around the last time object position depending on current observation, it can distribute the particles in a high likelihood area even though the dynamic model of the object cannot be obtained. Hence, we adopt an improved particle filtering approach based on particle swarm optimization (abbreviated as PSO-PF) in this module to handle the object tracking [20, 21]. PSO-PF algorithm does not distribute particles like sampling-importance resampling (SIR) for the reason that sampling-importance resampling usually distributes particles in the wrong likelihood area, particularly when tracking objects in a cluttered scene. PSO-PF first finds the sample area by PSO algorithm and then distributes the particles based on two different base points in order to achieve diversity and convergence in feature matching. For more details, please refer to [7].