Abstract
Early detection of defects and anomalies is important for safety and efficient management of structural elements. Brillouin scattering–based optical fiber sensors provide distributed sensing capabilities by monitoring the strain and temperature over large distances in structural elements. Their use has been limited to oil and gas explorations, mainly due to the inherent low signal-to-noise ratio in such systems, preventing detection of microcracks in structural monitoring applications. This study introduces a method based on the visual saliency approach through which the digital images acquired by the distributed strain data are employed for the detection of surface microcracks. When using this method, strain data sequences along the entire length of a structural element are sampled with a Brillouin scattering–based optical fiber sensor and then divided into a set of equal-length subsequences. A similarity measure matrix is composed based on the distributed strain data and then converted into a grayscale image. The saliency maps of the acquired grayscale images are calculated and a center-hollowed square template is defined and exploited for convolution with the binarized saliency map as a filter operator. The pixels retained after the filtering correspond to the locations of microcracks. Verification of the method was accomplished by experiments on a 15-m-long steel beam with fabricated defects.
Keywords
Introduction
Safety and efficiency of large engineering structures such as bridges are significantly increased by timely detection of cracks and anomalies at early stages of their development. Non-destructive testing (NDT) methods provide a powerful means for the detection of cracks in structural elements. Detection of defects by NDT approaches are accomplished by a number of methods, including acoustics (i.e. ultrasonics), infrared, X-rays, and others. 1 Point-by-point inspection of structures via NDT methods such as ultrasonics is effective, but time consuming in large structural system applications.2–4 Digital image processing algorithms, in which the length, width, shape, and even the depth of the cracks can be calculated from the acquired images, require fixed reference marks or reference planes, which in practice is hard to obtain and maintain.5–8 Distributed fiber optic sensors, such as those based on Brillouin scattering techniques, that is, Brillouin optical time-domain reflectometer (BOTDR) and Brillouin optical time-domain analyzer (BOTDA), have gained widespread usage for monitoring of strain and temperature in a number of applications including in oil explorations, aeronautics, and civil structural systems. 9 Brillouin scattering occurs when light along the length of an optical fiber interacts with time-dependent density variations in the core of the optical fiber resulting in a frequency shift in the optical signal. The frequency shift is calibrated against strain and/or temperature and employed in structural sensing applications.10,11
Optical fiber sensors possess many attributes including immunity to electric and electromagnetic interferences, resistance to corrosion, conformity to structural shapes, and long-distance high-spatial-resolution distributed sensing. For example, pulse pre-pump Brillouin optical time-domain analyzer (PPP-BOTDA) is capable of monitoring strains along a stretch of 30 km at a spatial resolution (SR) of 1 cm.12,13
Brillouin-based distributed sensors have been used in several structural health monitoring applications that require measurement of strains and detection of damage over long segments of members. Examples include monitoring of cracks in steel beams,14,15 detecting the change in cable forces of cable-stayed Bridges, 16 detecting unusual strain variations and cracks in long-span steel girder bridges, 17 monitoring of rock deformations in geotechnical applications,15,18 and monitoring bridge deck deformation under static loads.19,20 Survey of technical literature reveals that the general methods for distributed detection of cracks have been either direct and through identifying the localization of strains, that is, strain peaks along the monitoring length, or by way of the physical parameters of the Brillouin system, such as the change in Brillouin frequency with strain (BFS), or changes to the Brillouin gain spectrum width (BGS). The physical parameters of the Brillouin system are then related to the crack opening displacements (CODs). 21 While such approaches have been successful for larger cracks, they have been ineffective for the detection of microcracks.
The crack detection capability from the measured distributed strains depends on the SR of the Brillouin scattering system, system noise as well as close proximity to the crack. SR corresponds to the interval over which the distributed sensor averages the strain along the length of the optical fiber. Systems that average the strains over shorter lengths produce higher resolution strain measurements along the length of the optical fiber. Strain measurements along the lengths of structural elements are achieved by resolving the average strains within the SR of the system at preprogrammed intervals, or sample points. The capability of the sensing system for the interpretation of strain distribution is dependent on its SR, sampling frequency, system noise, and the strain gradient (i.e. COD) over the sensing segment. 22 The relationship between the SR and system noise is however inverse. Measurements at lower SRs are associated with less noise, and vice versa, where measurements at higher resolutions are associated with more noise. In addition to system noise, other issues such as environmental and traffic noise as well as the interference of the neighboring cracks cumulatively affect the output frequency shifts of the Brillouin system.21,23
Recent research activities on the subject of noise reduction in Brillouin systems have concentrated on a number of signal analysis techniques, including wavelets and vector decomposition methods.24–26 These approaches have been effective for the reduction of noise in less complex environments. Further research is necessary to examine the efficiency of the wavelet and signal decomposition methods in more complex loading conditions and damage patterns. Wavelet-based approaches require the selection of specific wavelet functions for different loading patterns and signal distribution geometries. The uncertainty and high degree of non-linearity associated with system and environmental noise increase the computational complexities and parameter sensitivity problems in such approaches.27–30
Recently, progress has been made in the use of image-based methods, such as image restoration techniques for enhancing the signal-to-noise ratio and intensification of Brillouin data. For example, Soto et al. 31 were successful in developing two- and three-dimensional image restoration techniques for intensification of Brillouin distributed data. The research reported herein, pertains to the development of a surface crack detection method based on visual saliency principles, which transforms the crack detection into a visual saliency problem in digital images. This image processing technique is not unique to Brillouin systems. Other distributed methods such as Rayleigh and draw tower fiber Bragg gratings (FBGs) also provide better SRs than BOTDA. This method is based on the hypothesis that the positions of salient pixels in the grayscale images have higher possibility to represent the locations where the cracks occur. Validation of the method involved laboratory tests of a 15-m-long wide-flange beam on a large test bed. The beam was configured with special splice points at two locations for controlled simulation of small CODs (microcracks) during the experiments. The experimental program included multiple tests to validate the hypothesis and the microcrack resolving capability of the proposed method for detecting the surface cracks. The experiments involved testing the hypothesis with a range of CODs at two different SRs in order to determine the crack detection resolution of the method in terms of crack size and SR. This approach is described next followed by the experimental program.
Proposed approach
The proposed approach involves the sequence of processes outlined in Figure 1. A typical sequence begins by the conversion of the distributed sensor data to grayscale pixels through computation of the similarity measure matrix. The distributed strain data acquired from a typical Brillouin system, such as PPP-BOTDA, correspond to the difference between the strain and reference measurements. The grayscale image is enhanced next using Gamma transformation, a technique for the amplification of pixel differences in the grayscale. Saliency map is formed next, which corresponds to the non-trivial parts of the pixels, signifying the features of the individual pixels within the image. Binarization of the grayscale salience map results in separating the salient and non-salient pixels. The binarized saliency map is then convoluted with a defined template, such as a center-hollowed square. The locations of the pixels corresponding to the maximum convolution values in the convoluted map are attributed to the possible crack locations. The frequency of the detected cracks based on this approach is employed for the finalization of crack locations.

Sequence of processes involved in the proposed approach.
In short, the overall objective for use of the image processing approach was to distinguish with a good degree of confidence the development of smaller microcracks, above and beyond the signal-to-noise limitations of the Brillouin systems. The various components of the image processing system to accomplish this include use of the techniques such as visual saliency and similarity matrix. The visual saliency approach is generally employed for distinguishing objects hidden by noise and/or other perturbations within images. In particular application to the distributed strains, the method enhanced the detection of microcracks (hidden objects), by creating foreground–background contrast between the crack and the rest of the data (i.e. the rest of the strain profile). Generally, similarity matrix is employed for graphical representation of similar data. In image processing and the method introduced herein, the similarity matrix is employed for the identification of similar strains within the acquired distributed strains. Once that is accomplished, it is possible to establish a grayscale image of the data. The contrast in the grayscale images is further enhanced using the visual saliency described herein. A detailed description of these processes is provided in the subsequent sections of this article.
Grayscale conversion of distributed strains
A strain sequence, S, is a sequence of strain values,

Typical distributed strain sequence with spatial distribution converted to sampling points.
The similarity measure
where
Once the similarity measure matrix is calculated, it is normalized to the interval [0,1], and the normalized matrix acquired is the grayscale image, O, shown in Figure 3.

Grayscale image obtained from similarity measure matrix.
Visual saliency approach
A typical image enhancement method, Gamma transformation, is employed for enhancement of the details of the image in the sections with high and low gray areas through exponential transformation of its gray values. 33 The mathematical description of the Gamma transformation employed herein for highlighting the contrasts between the pixels of different grayscale values is given by
where

Enhanced image based on Gamma transformation.
The spectral residual–based saliency detection method is employed for the generation of saliency map, Q, based on the enhanced grayscale image, M, acquired from the Gamma transformation mapping of the grayscale. 34 The principle behind the spectral residual–based saliency detection method is that a local linearity exists in the averaged logarithmic amplitude spectrum of pseudo-natural images. Comparing with the average, the statistical singularities in the logarithmic amplitude spectrum are responsible for the anomalous regions, that is, the salient regions, in the individual image. The computation takes the following form 35
where
Moreover, R is the spectral residual and Q is the saliency map, representing the reconstructed spatial domain image of the grayscale. The salient map of the grayscale image is shown in Figure 5.

Salient map calculated from the enhanced grayscale image.
Image binarization
Binarization of the grayscale salience map results in separating the salient and non-salient pixels. Therefore, the salient map, Q, is binarized to another image, B, by a dynamic threshold, namely
where

Binarized salient map B.
Filtering using center-hollowed square template
The binarized image, B, needs to be further filtered, since not all the preserved salient pixels correspond to real cracks. It should be noted that the similarity measure between the two subsequences containing the cracks is usually higher than that between the subsequences containing cracks and the subsequences on normal surfaces. These similarity measures usually correspond to the local maximums. It is assumed that a specific geometric shape like a center-hollowed square exists for the pixels corresponding to real cracks and their surroundings. Based on this assumption, a template is defined and used to further filter the preserved binary image, B, in order to refine the preserved salient pixels.
As shown in Figure 7, the template is defined as a center-hollowed square with a size of

Center-hollowed square template.
The preserved binary image, B, is further filtered using the convolution matrix, H, which is the defined center-hollowed square template. The filtered image is depicted in Figure 8.

Filtered binarized image using center-hollowed square template, Cd.
Each pixel of the resulting image,
Experimental program
The experimental program was designed to accommodate a longer span beam on a test bed with five steel frames on steel wide-flange beam foundations. This allowed for testing of a 15-m-long (49.2-ft) steel I-beam. The steel frames were employed for support and application of four-point bending loads to the beam. The 15-m (49.2-ft) span of the beam provided sufficient length for the constant moment region of the beam. This arrangement facilitated the evaluation of the method with SR of the PPP-BOTDA as one of the study parameters. The beam was constructed by splicing three steel segments of 4.57 (15 ft), 5.85 (19.2 ft), and 4.57 m (15 ft) in length at two splice points along the span length. Instead of double splice plates on beam flanges to carry the moments across the splice points, a single bolted plate was employed to create a discontinuity, reduce the stiffness at the splices, and simulate damage in the form of defect growth or COD during the experiments. The openings of the simulated defects (CODs) were controlled by loosening the bolts at the splice points. The beam was supported at two intermediary locations and loaded at the ends by hydraulic actuators in a four-point bending configuration. The upper flange of the beam was placed in tension by loading the beam at its ends. The beam support locations were intentionally selected to position the splice points in the constant maximum moment region of the beam. A schematic layout of the beam with the loading locations and the typical moment diagram are shown in Figure 9. The beam and its spliced regions are shown in Figure 10.

(a) Schematic layout of the steel beam and (b) typical moment diagram of the beam under the point loads.

(a) Test frame, (b) steel beam, and (c) splice joint.
A commercially available PPP-BOTDA system was employed in the experiments. As shown in Figure 11, PPP-BOTDA employs two light sources, that is, pump and probe, entering the fiber from opposing directions. A single-mode fiber, Corning SMF-28, was used as the sensing fiber over the 15 m span length of the beam. The optical fiber out of the probe end of the BOTDA was slightly prestressed for stability and adhered to the top surface of the beam by an epoxy resin, coiled at the span end, and returned to the BOTDA at the pump light. These parallel fiber optic segments were protected by another layer of polyimide tape. The experiments were performed under isothermal conditions, and the coiled loose end of the fiber acted as a temperature sensor for thermal compensation in the eventuality of thermal fluctuations. FBG displacement sensors were employed for monitoring the COD at the two simulated defect zones of the beam during the experiments. The gauge length of the displacement sensors was 17.8 cm with the resolution of 1 μm. Their operational principles are described elsewhere. 36 The beam was loaded at 177-N (40-lbs) increments at each load end from 0 to 711 N (160 lbs) for straining the entire optical fiber along the length of the beam and to increase the COD at the two simulated defect locations. Two different loading mechanisms were employed at each load point, a mechanical device on one side and a hydraulic jack on the other. The hydraulic jack had internal oil leakage problem, and the leakage intensified for subsequent larger loads. This led to a decrease in strain on one side.

BOTDA system with distributed sensing fiber.
The experimental program is provided in Table 1. As shown in this table, the experiments involved tests at two different SRs of 10 and 20 cm, respectively. The ranges of simulated CODs considered in this study were from 23 to 69 µm for joint A and 57 to 304 µm for joint B for each SR. The distributed strain measurements were made over a sampling interval of 5 cm in all the experiments. Two crack sensors were employed at each joint for independent measurement of CODs. Experimental results are described next.
10 test cases for steel beam test in laboratory.
Experimental results
Table 1 provides information about the applied loads and the corresponding measured CODs for the 10 experiments performed in this study. Typical distributed strain data along the length of the beam are shown in Figures 12 and 13. Figure 12 pertains to the measurements at an SR of 10 cm, and Figure 13 corresponds to measurements at an SR of 20 cm. Comparison of Figures 12 and 13 indicates that the measurements at lower resolutions contained less system noise, and at an SR of 20 cm it was actually possible to detect the location of the defects, especially at larger CODs without further processing. As discussed earlier, the relationship between the higher resolution measurements (i.e. SR = 10 cm) and the system noise is inverse. The BOTDA averaging process is performed over a longer distance at lower resolution SRs, that is, 20 cm for SR = 20 cm as opposed to 10 cm for SR = 10 cm. Therefore, as shown in Figure 13, lower SR measurements result in smoother distributed strains over the length of the beam. On the other hand, for closely spaced multiple cracks, the averaging process at lower SRs diminishes the intensity of the smaller CODs, resulting in detection error.

Distributed strains of the steel beam for Cases 2 and 3 (SR = 10 cm).

Distributed strains of the steel beam for Cases 7 and 8 (SR = 20 cm).
Considering the experimental results from this as well as previous studies, it is understood that the limitations on crack detection in distributed measurements are the size of the crack or COD, SR, and interference of multiple cracks within close proximity to each other.
23
To examine the capability of the visual saliency approach presented herein, binarized images of the distributed strains were computed and filtered using the center-hollowed square template. The threshold for the indicator r, for measurements in all the experiments, was set to be 10 (Tr = 10). The number of points exceeding this threshold is

Final possible detected crack locations.
In the same figure, the detected crack locations based on the computed visual saliency method are shown by circular tick marks. Experiment numbers 1 and 6 correspond to the control cases in which there were no loads applied and CODs were zero. It is however possible to evaluate the crack location detection capability of the computational approach for increasing CODs in experiments 2–5 and 7–10 for the SRs of 10 and 20 cm, respectively. For the high-resolution measurements at SR = 10 cm, all four cracks are detected in experiments 4 and 5. For the lower resolution measurements, that is, SR = 20 cm, all four cracks in experiments 8–10 are detected. Note that in some cases the computed results provide two locations near each other at the crack locations. Considering the near misses as false positive, the performance of the visual saliency approach can be quantitatively evaluated by introducing the missing rate (MR) and false-positive rate (FPR) indexes defined as
where
Missing and false-positive rates of the experiments.
MR: missing rate; FPR: false-positive rate.
The variation in MR with respect to FPR for both experimental groups is depicted in Figure 15. For the points closer to the lower left edge of Figure 15, crack locations were more accurately detected. These points are also associated with larger CODs.

Missing rate with respect to false-positive rate.
It can be seen from Figure 15 that for lower resolution SRs (SR = 20 cm) it was possible to detect the location of all the cracks (MR = 0), whereas for the measurements at higher resolution SRs (SR = 10 cm) in experiments 3 and 4 location of one crack was missed, even though in these experiments the CODs were the same as those in experiments 8 and 9.
Conclusion
A visual saliency–based surface crack detection method is proposed by distributed measurement of strain with a fiber optic Brillouin scattering sensor. The method involved sequential clustering of the distributed strain data acquired from PPP-BOTDA, grayscale binarization of data, establishment of the saliency map of the grayscale image, filtering using a center-hollowed square template, and computation of the coordinate values of the salient pixels in the saliency map representing the locations of the microcracks. The effectiveness of the method was verified by an experimental program involving load tests of a steel beam with simulated defects.
The experimental results indicated that for the same CODs the accuracy of the method was dependent on the SR of the measurements. For instance, the minimum detectable COD at an SR of 10 cm was computed to be 69 μm, whereas it was possible to detect a COD of 32 μm at SR = 20 cm. Despite these results, the better performance of lower SRs cannot be generalized based on the results of this study. The effects of multiple cracks within the proximity of each other may alter these results because of interference in the averaging and noise canceling processes involved in measurements at lower SRs. Further experiments are necessary with multiple cracks within the SR length of the measurements, that is, two cracks within the 20 cm SR. Moreover, additional work is necessary for exploring and designing better convolution templates and visual saliency detection algorithms in reducing the proportion of the false positives in the computations.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
Financial support for Prof. Q. Song’s period of study at the University of Illinois, Chicago was provided in part by China Scholarship Council (CSC) and also by the Fundamental Research Funds for the Central Universities, Chang’an University (Grant No. 310824162022), which is greatly appreciated.
