Damage detection in a novel deep-learning framework: a robust method for feature extraction

Abstract

Extracting damage features precisely while overcoming the adverse interferences of measurement noise and incomplete data is a problem demanding prompt solution in structural health monitoring (SHM). In this article, we present a deep-learning-based method that can extract the damage features from mode shapes without utilizing any hand-engineered feature or prior knowledge. To meet various requirements of the damage scenarios, we use convolutional neural network (CNN) algorithm and design a new network architecture: a multi-scale module, which helps in extracting features at various scales that can reduce the interference of contaminated data; stacked residual learning modules, which help in accelerating the network convergence; and a global average pooling layer, which helps in reducing the consumption of computing resources and obtaining a regression performance. An extensive evaluation of the proposed method is conducted by using datasets based on numerical simulations, along with two datasets based on laboratory measurements. The transferring parameter methodology is introduced to reduce retraining requirement without any decreases in precision. Furthermore, we plot the feature vectors of each layer to discuss the damage features learned at these layers and additionally provide the basis for explaining the working principle of the neural network. The results show that our proposed method has accuracy improvements of at least 10% over other network architectures.

Keywords

Deep learning convolutional neural network feature extraction contaminated data damage detection

Introduction

Several large-scale structures such as bridges, buildings, aircraft, and defense systems are becoming irreplaceable in modern society. The safety and stability of these structures are fundamental issues that cannot be ignored owing to congenital and acquired damage. Structural health monitoring (SHM), a promising technique, has made progress in implementing damage detection strategies. The ultimate goal of the technique is to identify the presence, severity, and location of the damaged areas by processing measurement signals in various forms such as modal participation ratio,¹ frequency response function,² and wavelet coefficients.³

In the research field of SHM particularly for large-scale structures, the approach that translating the measurement vibration signals into meaningful indices to reflect the actual condition of the structure is one of the most adopted approaches.⁴ Indicators (i.e. hand-engineered features) based on various vibration signals such as mode shape–based,^5,6 frequency-based,⁷ frequency response function–based,⁸ and vibration response–based^9,10 features have been extensively studied. Among these methods, mode shape–based methods are particularly suitable for the issue of damage localization owing to their spatial arrangement characteristic. As an early-proposed hand-engineered feature to represent damage, the mode shape curvature has a definite physical meaning and has been applied to a variety of structures.^11–13 This damage representation utilizes the second-order derivative of mode shape to reflect the damage location, and the center difference method¹⁴ is the most popular method to calculate the mode shape curvature. In addition, detecting damage via this hand-engineered feature does not require the data in undamaged structure. Flexibility,¹⁵ which is the inverse of the stiffness matrix, is another mode shape–based damage indicator. The methods based on this indicator mainly utilize the change in flexibility, before and after damage, to reflect the damage location. This indicator has the advantage in that it can extract the damage features from multiple modes and, therefore, is suitable for multiple damage scenarios and noisy conditions.¹⁶ The capabilities of these methods have been validated via laboratory experiments or real structures. However, developing a hand-engineered feature to represent damage requires not only to study the change in characteristic before and after the damage occurred but also to resolve the problem of robustness to operational and environmental variations.^17–19 Moreover, a hand-engineered feature is difficult to apply to a variety of structures, because a certain type of structure (including its material, geometry, and service environment) has specific vibration characteristics. Consequently, a reliable hand-engineered feature relies heavily on expert knowledge, including vibration analysis and signal processing. For example, it is difficult to obtain the data by deploying sensors in a rotating component when health monitoring is conducted on a mechanical system; thus, we cannot obtain all the data of the system. Besides, for the same structure, the detection results maybe different due to random character of the measurement noise. Therefore, a method which has robustness to noise and ability to overcome the interference of data missing is needed.

Deep learning²⁰ empowers many aspects of modern society, such as object identification, semantic understanding, and image processing. Convolutional neural network (CNN), as an important branch of deep learning, discovers an intricate feature in large datasets by using the error-backpropagation algorithm^21,22 to indicate how a computational model should change the internal parameters used to compute the representation in each layer from the representation in the previous layer. Furthermore, deep learning can avoid the shortcomings of the aforementioned hand-engineered features, that is, a deep-learning-based method allows a computation model to be fed with raw data and to automatically discover the representations needed for the target task, and this is the fundamental distinction between the two approaches as shown in Figure 1. Because it can reduce not only the research time but also expert knowledge of physical problems, to date, several research groups have proposed CNN-based methods for damage detection,^23–26 and the process adopted in them is the same: first, generating the sample set via vibration testing or numerical simulation; then, selecting the architecture of the network; and finally, training and testing the network. However, the computational model is a black-box, that is, it is hard to understand how it works.²⁷ Thus, a trained deep-learning model may not adopt to other tasks. To get a reliable estimation, it is vital for us to get qualitative interpretations to these models.^24,28 Besides, considerable amount of time and expert knowledge are required to design feature extractors or classifiers.

Figure 1.

Comparison of computational flow between hand-engineered-feature-based methods and deep-learning-based methods: the deep-learning-based method is a feature extractor that learns representations of data with multiple levels of abstraction via recomposing multiple processing layers.

The main contribution of this study is the introduction of a novel framework of the deep-learning model to realize damage detection in a variety of conditions. In the designing and training strategy of the network, scenarios such as noise interference, measurement data missing, and multiple damage are considered. Then, we take corresponding measures including multi-scale representation learning, residual learning, and dropout technique to solve these problems. A network model needs a large amount of data for training; however, it is difficult to meet this requirement only by sensor measurements. To solve this limitation, we adopt an alternative strategy, which is generating the mode shapes of various beams as training samples via numerical simulation. The results of numerical and laboratory experiments demonstrate that the proposed method can not only learn damage representation from the raw data but also achieve an acceptable accuracy under various conditions. Besides, the knowledge transfer between related tasks can reduce training costs effectively. The transferring parameter methodology is introduced to re-use some parameters in the per-trained network for other target tasks. Moreover, experimental results show that this methodology can reduce training cost without decreasing accuracy. In addition, we discuss what the network learned in each layer and use it to analyze its working principle.

Proposed approach

Function of each module

Most architectures of CNN models mainly contain the convolutional and pooling layers. The convolution computation is the major process of feature extraction and learning. Let us consider a one-dimensional CNN with L hidden layers, and $l \in {1, . . ., L}$ index the hidden layers of the network. Given an input vector with n components, $x = (x_{1}, x_{2}, . . ., x_{n}) \in ℜ^{n}$ , the ith unit at the lth layer, $y_{i}^{[l]}$ , can be described as

z_{i}^{[l]} = \sum_{j \in N} W_{j}^{[l]} x_{j} + b_{i}^{[l]}

(1)

y_{i}^{[l]} = σ (z_{i}^{[l]})

(2)

where $z_{i}^{[l]}$ is the ith unit of the convolution result at layer l, N is a set of indices of neighbors, $W_{j}^{[l]}$ is the jth unit of the convolutional kernel at layer l, $b_{i}^{[l]}$ is the ith unit of bias at layer l, and $σ (\cdot)$ is the activation function. Here, we choose the parametric rectified linear unit (PReLU)²⁹ as the activation function and can be described as follows

σ (z_{i}^{[l]}) = {\begin{matrix} z_{i}^{[l]}, z_{i}^{[l]} > 0 \\ α z_{i}^{[l]}, z_{i}^{[l]} \leq 0 \end{matrix}

(3)

where $α$ is a coefficient controlling the slope.

The pooling layer is another commonly used layer type, which contains three functions: reduction of feature dimension, feature invariance, and overfitting prevention. In this study, average pooling and max pooling are employed, and they can respectively be described as

y_{j}^{[l]} = \frac{1}{s} \sum_{1 \leq k \leq s} y_{s (j - 1) + k}^{[l - 1]}

(4)

y_{j}^{[l]} = max_{1 \leq k \leq s} y_{s (j - 1) + k}^{[l - 1]}

(5)

where $y_{i}^{[l]}$ is the jth output unit of the pooling layer and s is the length of the pooling stride.

In this study, a novel network architecture is designed to solve the problems of noise interference and data missing. Concretely, the architecture consists of the following three modules.

Multi-scale module

Stacking multiple convolutional layers on top of each other can give the network an inherent feature of abstracting details in deeper layers.^30–32 However, this imposes more burden on the network for learning to reconstruct details from an abstract representation of the signal. Moreover, such a network requires a large number of layers and, hence, is more difficult to train. To circumvent this, this article introduces a multi-scale module to extract as many features as possible from the input signal in the first layer itself. The architecture of this module is first proposed in GoogLeNet.³¹ The main idea of this module is based on finding out how an optimal local sparse structure in a CNN can be approximated and covered by readily available dense components. The module contains multiple sets of convolutional filters, and each set has a special filter size. Figure 2 illustrates the architecture of the multi-scale module. In this study, we select four types of convolution kernels, with size $1 \times 1$ , $1 \times 3$ , $1 \times 5$ , and $1 \times 7$ . As displayed in the figure, the features are extracted by convolving the input signal with these convolutional kernels, and the resulting activations from all the convolutional layers are stacked together as the output of the module. Owing to the different sizes of convolution kernels have various receptive fields and the output of the module has the multi-scale property, that is, it has different representations of the same signal, as shown in Figure 2(b). In addition, the property is helpful to extract useful features from a noisy signal.^33–36

Figure 2.

(a) Architecture and (b) schematic of the multi-scale module. Here, the black-dotted bordered rectangles denote the receptive fields of the convolutional kernels, Conv: $W \times L$ denotes a convolution kernel, with width W and length L. The green block represents an activation layer.

Residual learning modules (RLM)

Deeper neural networks are difficult to train, and the residual learning framework is more applicable to these networks because this framework can solve the problem of vanishing or exploding gradients. Furthermore, empirical evidences show that residual networks are easier to optimize.^32,37 In this study, we adopt two types of RLMs to every few stacked layers, and the modules can be described as follows

y = σ (F (x, {W_{i}}) + x)

(6)

y = σ (F (x, {W_{i}}) + W_{s} x)

(7)

where ${W_{i}}$ represents multiple weights and $W_{s}$ represents a linear projection vector by the shortcut connections to match the dimensions, with size $1 \times 1$ . $F (x, {W_{i}})$ is the residual mapping to be learned and can represent multiple convolutional layers. The detailed architectures of the two types of RLMs are shown in Figure 3. The architecture contains stacked nonlinear layers (i.e. the residual mapping) and a shortcut connection. We employ these two modules alternately followed by the multi-scale module to extract more abstract features.

Figure 3.

Detailed architectures of two types of RLMs, namely, (a) RLM-A and (b) RLM-B.

Global average pooling

For conventional CNN architectures, the output feature of the last convolution layer is vectorized and fed into fully connected layers.^23,24 This connection mode bridges the convolutional layers with fully connected layers. Here, the convolutional and pooling layers are treated as feature extractors, and the fully connected layers are treated as target classifiers or regressions. However, the architecture of the fully connected layers has two shortcomings: first, the weight matrices contain a large number of parameters that require optimization, and the process employs excessive computing resources; second, a large number of parameters may lead to model overfitting, thus hampering the generalization ability of the overall network. Hence, we utilize the global average pooling strategy³⁸ to replace the traditional fully connected layers. The idea is to generate one feature vector for each corresponding category of the classification or regression task in the last convolutional layer. Instead of adding fully connected layers on top of the feature maps, we take the average of each feature map. The great benefit is that there is no parameter to optimize in the global average pooling, and thus, overfitting is avoided.

Architecture of the neural network

We tried several versions of networks during our verification stage. The most successful particular instance is illustrated in Figure 4, with the detailed configuration listed in Table 1.

Figure 4.

Overall schema for the proposed network architecture.

Table 1.

Detailed configuration of the proposed neural network architecture.

Type	Kernel size (Stride)	Kernel number (with activation)	Output size	Padding
Multi-scale module	$1 \times 1$ (1)	32 (True)	$1 \times 39 \times 32$	Same
	$1 \times 3$ (1)	32 (True)	$1 \times 39 \times 32$	Same
	$1 \times 5$ (1)	32 (True)	$1 \times 39 \times 32$	Same
	$1 \times 7$ (1)	32 (True)	$1 \times 39 \times 32$	Same
Max pooling	$1 \times 3$ (2)	None	$1 \times 20 \times 128$	Valid
RLM-A1	$1 \times 1$ (1)	64 (True)	$1 \times 20 \times 64$	Same
	$1 \times 3$ (1)	64 (True)	$1 \times 20 \times 64$	Same
	$1 \times 1$ (1)	128 (True)	$1 \times 20 \times 128$	Same
RLM-B1	$1 \times 1$ (1)	128 (True)	$1 \times 20 \times 128$	Same
	$1 \times 3$ (2)	128 (True)	$1 \times 10 \times 128$	Same
	$1 \times 1$ (1)	256 (False)	$1 \times 10 \times 256$	Same
	$1 \times 1$ (2)	256 (False)	$1 \times 10 \times 256$	Same
RLM-A2	$1 \times 1$ (1)	128 (True)	$1 \times 10 \times 128$	Same
	$1 \times 3$ (1)	128 (True)	$1 \times 10 \times 128$	Same
	$1 \times 1$ (1)	256 (False)	$1 \times 10 \times 256$	Same
RLM-B2	$1 \times 1$ (1)	256 (True)	$1 \times 10 \times 128$	Same
	$1 \times 3$ (2)	256 (True)	$1 \times 5 \times 128$	Same
	$1 \times 1$ (1)	512 (False)	$1 \times 5 \times 512$	Same
	$1 \times 1$ (2)	512 (False)	$1 \times 5 \times 512$	Same
RLM-A3	$1 \times 1$ (1)	256 (True)	$1 \times 5 \times 128$	Same
	$1 \times 3$ (1)	256 (True)	$1 \times 5 \times 128$	Same
	$1 \times 1$ (1)	512 (False)	$1 \times 5 \times 512$	Same
Convolutional	$1 \times 3$ (1)	1024 (True)	$1 \times 5 \times 1024$	Same
Convolutional	$1 \times 3$ (1)	2048 (True)	$1 \times 5 \times 2048$	Same
Convolutional	$1 \times 1$ (1)	38 (False)	$1 \times 5 \times 38$	Same
Global average pooling	$1 \times 5$ (1)	None	$1 \times 1 \times 38$	Valid

RLM: residual learning modules.

When a dataset consisting of mode shape is given, the network first uses the multi-scale module to learn damage features at various scales, and a max pooling layer is followed to down sample the feature vectors. Then, three RLMs A and two RLMs B are repeatedly stacked to learn the high-level features. Next, we designed three convolutional layers to extract features from the previous layer, and the third convolutional layer can reshape the output dimension. It should be mentioned that the size of the last convolutional layer is $1 \times 1$ and has no activation operation because it can avoid the blurring of the damage features.³⁹ Finally, a global average pooling layer is stacked at the end of the network, and it directly outputs the damage probability of each measurement point after the flattened processing. Furthermore, a dropout layer is introduced at the end of the network, and the function of the dropout layer is to randomly delete certain points in the output of the previous layer and retain the input and output of this layer with the same dimension. Moreover, the introduction of this layer can prevent overfitting effectively.⁴⁰

Training strategy of the network

Generation of samples and labels

The finite element method (FEM) is adopted to model the Euler–Bernoulli beams with single or multiple damage. Then, the modal computation is conducted to obtain the mode shape data as training samples, as shown in Figure 5. Moreover, local damage is introduced via the stiffness degradation method, which can be described as

E_{k}^{d} = (1 - α) E_{k}^{u}

(8)

where $E_{d}$ and $E_{u}$ are the Young’s modulus of the kth element in damage and undamaged scenarios, respectively, and $α$ is the damage coefficient.

Figure 5.

FEM model of a damaged beam: the damage locations (marked in gray), beam dimension, and material properties are randomly chosen.

Moreover, various boundary conditions are considered, including pinned–pinned (P–P), clamped–free (C–F), and clamped–clamped (C–C). In this way, the network can learn more types of damage representations thereby improving the applicability. The strategy for generating the samples is described as follows: (1) randomly choose the boundary condition, dimension, and material parameters of beams; (2) randomly choose p elements to decay their stiffness, with p ranging from 1 to 3; (3) randomly choose p damage coefficients for selected damaged elements, with $α$ range (0, 1); (4) calculate the mode shape and add Gaussian white noise with a random signal-to-noise ratio (SNR) of 60–100 dB to the pure mode shape; (5) pre-process the noisy mode shape via normalization processing to ensure that all the data are at the same scale, and the processing can be described as

Φ^{*} = \frac{Φ - \bar{Φ}}{std (Φ)}

(9)

where $Φ^{*}$ is the normalized mode shape vector, $Φ$ and $\bar{Φ}$ are the mode shape vector and its mean value, respectively, and $std (\cdot)$ represents the standard deviation of the vector.

Processes (1)–(5) are repeated to obtain the sample set, and that contains $1.5 \times 10^{6}$ data for training, $5 \times 10^{4}$ for validation, and $10^{4}$ for testing. The dataset contains noisy mode shapes and its corresponding labels. The label is designed as a vector consisting of n elements (n is equal to the element number of the beam model). Each element value represents the damage degree of the corresponding beam element. For example, if a label vector has all zero elements except that the second element is 0.1 and the fourth is 0.3, it implies that the second and fourth element of the beam are damaged with damage levels of 10% and 30%, respectively, and the others are intact.

Loss function

Regression and classification are two fundamental tasks in supervised learning, which are used to predict the relationship between input and output variables. Concretely, a regression problem is equivalent to a function fitting that choose a function curve to fit the given data while predicting the unknown, and the output is continuous. The classification model can map the regression results to (0, 1) via a function (e.g. SoftMax)²¹ and then define a threshold to classify the input. For damage detection issues, particularly when multiple damage occurs in a structure and the damage degrees need to be predicted, the regression model might be more suitable.

To minimize the output of the objective function, an algorithm for first-order gradient-based optimization of mini-batch stochastic gradient descent, called Adam,⁴¹ is employed as the optimization algorithm. This method is based on adaptive estimates of lower order moments and is well suited for problems that are large in terms of parameters. Furthermore, we choose the mean squared error (L2 loss) as the loss function, and it can be written as

L (P, \bar{P}) = \frac{1}{M} \sum_{m = 1}^{M} {(P (m) - \bar{P} (m))}^{2}

(10)

where $L$ is the loss value, $P$ and $\bar{P}$ are the prediction value and true label, respectively, and $M$ is the dimension of the label vector. Here, the dimensions of the prediction and label are equal to the number of elements.

Training results

The training and testing of the proposed network are performed on the same software and hardware platform: the processor and graphics card types are Inter Xeon E5-2623 and Nvidia GTX 1080Ti, respectively. We adopt the Python packages NumPy and SciPy to program the FEM model of the damaged beams, and then calculate the mode shapes, that is, generate the samples and labels. Then, the deep-learning framework TensorFlow⁴² is employed to establish, train, and test the network model. The graphics processing unit (GPU) acceleration technology is used, which can improve the training speed of the network remarkably.

We trained the proposed network with the generated dataset for 80 epochs, and it required approximately 8 h. The detection accuracy and loss value of each batch (a batch contains 512 samples) are recorded and displayed in Figure 6. As demonstrated in plots, the accuracy rate increases slowly and finally stabilizes around 0.8, and the curves of loss values have the same varying tendency. In a comparison of the performance in the training and test sets, the network achieves the same level of accuracy in training and testing sets. Moreover, the fluctuation of accuracy and loss in the training set is less than it is in the testing set, because the testing data are completely novel for the network. Therefore, the performance in the testing set is relatively unsteady.

Figure 6.

(a) Detection accuracy and (b) loss evolution during the training process.

Network evaluation via numerical simulations

Damage detection in noisy scenarios

A simply supported beam with a rectangular cross section of length 0.7 m, width 0.05 m, and thickness 0.01 m is considered as the test specimen. The Young modulus and density are 210 GPa and 7860 kg/m³, respectively. Rayleigh damping is used to simulate the damping effect which can be expressed as $C = α_{0} M + α_{1} K$ , where $α_{0} = 7 \times 10^{- 6} s$ and $α_{1} = 1.0 s^{- 1}$ . The beam is divided into 40 Euler–Bernoulli beam elements, the sixth node is subjected to a single-point momentary excitation. In the 10th and 30th elements, 10% and 12% reductions of stiffness are introduced, respectively. Besides, the noise is added on the time domain responses referring to⁴³

y_{n} (t) = y_{n}^{0} (t) + β λ max (y_{n}^{0} (t))

(11)

where $y_{n} (t)$ and $y_{n}^{0} (t)$ represent the noisy and non-noisy displacement responses of node n at time t, respectively. $λ$ is the random parameter, with its continuous distribution following a Gaussian distribution with zero mean and unitary standard. The level of noise $β$ is set to be 0.02.

Figure 7 displays the time domain responses of the fifth node and the first–third mode shapes extracted using the modal identify technique. The curves are smooth and no abnormality at the preset damage locations is observed; thus, it is difficult to locate the damage directly from the noisy mode shapes as the damage degree is small. The SNR, the ratio between the average power of the noisy mode shape $P_{s}$ and the corresponding noise $P_{n}$ , is defined as $SNR = 10 \log_{10} (P_{s} / P_{n})$ . The details of the simulation are listed in Table 2.

Figure 7.

(a) Time domain responses of the fifth node in damaged (top) and undamaged (bottom) conditions and (b–d) the first–third noisy mode shapes.

Table 2.

Different orders of modal frequencies and the corresponding SNRs in damaged and undamaged (indicated in brackets) conditions.

Modal order	Natural frequency (Hz)	SNR (dB)
1	47.585 (47.833)	78.393 (80.721)
2	190.170 (191.334)	71.888 (71.489)
3	429.323 (430.502)	62.929 (66.305)

SNR: signal-to-noise ratio.

Figure 8 shows the predictions of the network; it can be observed that obvious damage features (i.e. singular peaks) appear around the preset damage locations. The results demonstrate that the proposed method can identify the damage based on the mode shape data directly and has robustness against measurement noise. Moreover, the predictions of undamaged mode shapes are almost zero; it means that the network has learned the damage feature and can process a scenario which we have never seen before.

Figure 8.

Damage detection results obtained by (a–c) first–third mode shapes (marked in black line) and the corresponding undamaged conditions (marked in blue dots).

In addition, we utilize the Monte Carlo simulation to separately evaluate the detection capability of the trained network in various noise environments. Each sub-simulation contains $10^{5}$ samples, including different dimensions of beams, damage numbers and locations, damage degrees, and noise levels. Figure 9 shows the accuracy assessments of the single and multiple damage scenarios, and the input data are polluted by different noise intensities. According to the figure, the accuracies obtained by higher order mode shapes are significantly better than the lower orders, and the results obtained by the first mode shape are more seriously affected by noise interference. Moreover, the proposed method can steadily maintain a higher accuracy (more than 90% and 75% in single and multiple damage scenarios, respectively), with SNR ranging from 90 to 120 dB, and then, it gradually decreases. The measurement noise is more likely to mask the damage features with the increase of noise intensity, which makes it difficult to extract the damage features. Consequently, the recognition accuracy decreases with the decreasing SNR. Overall, the proposed method can obtain acceptable results, particularly based on higher mode shapes (the results obtained by second–fourth mode shapes can maintain more than 80% and 60% accuracies in single and multiple damage scenarios, respectively).

Figure 9.

Detection accuracies of single (left) and multiple (right) damage scenarios in various noisy environments obtained by Monte Carlo statistic.

Data missing test

In real applications, the problem of data missing is unavoidable because of the limitation of measurement and improper data preservation. Figure 10 displays Three data missing scenarios and the corresponding detection results. The incomplete signals are based on the mode shapes mentioned above (as shown Figure 7(b)–(d)), about 10%, 20%, and 30% of the total. We utilize the zero value to fill in the missing fields in the computation to maintain the continuity of data, and the noise interferences are also added. According to the detection results, the proposed method can extract damage features from the incomplete input, which demonstrates that proposed method based on the deep learning has robustness to data missing and noise interference.

Figure 10.

(a–c) Noisy mode shapes with missing data and (d–f) the corresponding detection results. Here, blue boxes denote the locations of missing data.

To validate the capability of the proposed method in data missing and noisy scenarios, the Monte Carlo evaluation is conducted again and the results are displayed in Figure 11. Here, each sub-simulation contains $10^{5}$ noise-polluted and defective mode shape data. Compared with the results as shown in Figure 9, all the accuracies are decreased. However, the proposed method can still obtain acceptable accuracies (more than 75%) in the single and multiple damage scenarios with 40% and 30% missing data, respectively.

Figure 11.

Assessments of single (left) and multiple (right) damage detection accuracies in data missing and noisy scenarios.

Experimental validations

This section describes the laboratory experiments conducted to validate the proposed deep-learning-based method. Two types of beam-like structures and two verification means are selected to demonstrate the validity of the proposed method. Concretely, the experiments are organized as follows:

Selection of single observation data to test the damage identification ability of the network in actual scenarios.

Selection of the cracked rotating shafts as test specimens and measurement of the mode shape 100 times to test the detection accuracy of the network.

Note that the network used here is the same trained network used in “Network evaluation via numerical simulations” 4, and the network is trained via numerical data.

Damage detection for cracked beams

The experimental investigations are based on the WavStructDamAs benchmark provided by Professor Katunin on his website.^44,45 The benchmark consists of measurement data obtained from vibration-based testing of composite structures. The data we choose to verify the detection capability of the proposed method are mode shapes of beams with single and multiple cracks. The length, width, and thickness of the test beam are 250, 25, and 5.28 mm, respectively. Moreover, the beams are made of a 24-layered glass fiber–reinforced epoxy-based laminated composite. The mode shapes are measured by scanning LDV Polytec PSV 400.

The measurement mode shapes contain 39 sampling points, and the locations of the two cracks (both depths are 1 mm) are, respectively, 86 and 149 mm from the left fixed end, and the first- to third-order mode shapes are shown as Figure 12(a) to (c). As can be seen from the diagrams, there are no abnormal changes around the preset crack locations, and the mode shape curves remain smooth. These measurement results demonstrate that it is difficult to detect the local damage from the mode shape directly. By contrast, the damage features can be extracted completely by the trained network, except the result based on the first mode shape. Moreover, the trained network eliminates the noise interference while extracting the damage features, and this result proves that the proposed method has the capability of resisting the measurement noise.

Figure 12.

Measurement mode shapes of the cracked beam and detection results: (a–c) the first–third order mode shapes and (d–f) the detection results of the proposed network.

Another set of measurement data is chosen, and the test specimen contains two cracks and their locations are, respectively, 50 mm (depth 1 mm) and 119 mm (depth 2 mm) from the left fixed end. Here, we remove certain data artificially to verify the detection capability of the proposed method via experimental means, as shown in Figure 13(a)–(c). The locations of the missing data are selected by the authors; the original measurement data are complete. The detection results show that the network can extract damage features from incomplete data, whereas the missing data and measurement noise have no obvious influence on the feature recognition.

Figure 13.

(a–c) Measurement mode shapes of the cracked beam with missing data and (d–f) detection results.

Transferring parameter methodology

In practical applications of SHM, large amounts of measuring signals are difficult to maintain a consistent length, and the number of predicted damage locations also vary. For a trained network, when the length of the input signal and the output prediction are different from the training samples, the traditional way is training a novel network to fit the new scenario; however, it implies a waste of computing resources and time. Therefore, we utilize the transferring parameter methodology⁴⁶ to make the trained network parameters adapt to novel input–output relations. The key idea of the transferring parameter methodology is that the internal layers of the CNN can act as a generic extractor of a mid-level signal representation, and then be re-used in other target tasks. Figure 14 illustrates the flowchart of the transferring parameter approach. First, the network is trained on the source task with a large amount of available labeled mode shape data, as shown in the top. Pre-trained parameters of the internal layers of the network are then transferred to the new target tasks, as shown at the bottom. To compensate for the different of the source and target data, we re-train the last two convolutional layers on the labeled data of the target task.

Figure 14.

Transferring parameters of the trained network.

Furthermore, benchmark data are selected to demonstrate the detection capability of the network. Figure 15(a)–(d) displays the first- to fourth-order mode shapes of a single cracked beam, and the crack is 150 mm from the left fixed end. Each of the mode shapes contains 44 sampling points. We retrain a network via the transferring parameter technology and use these benchmark data to test the detection capability (as shown in Figure 15(e)–(h)). The retrained network can detect all damage locations according to the measurement mode shape data, except for the second order. It should be mentioned that we just generate $6 \times 10^{5}$ training samples ( $1.2 \times 10^{6}$ samples were used for the original network) to train the novel network, and it requires only about 45 min (8 h were required for the original network). In view of the results, the transferring parameter technology can not only solve the problem of inconsistency between the application scenario and the training scenario but also save a large amount of computing time.

Figure 15.

The first- to fourth-order mode shapes of the cracked beam and corresponding detection results: (a–d) mode shapes and (e–h) detection results.

To validate the damage detection capacity for other beam-like structures, we conducted an experiment on cracked rotating shafts. The experiment is designed on a Bently Nevada RK4 Rotor Kit as displayed in Figure 16. Two cracked rotating shafts are used as the test specimen, and we, respectively, introduce single and double cracks (all with depths of 2 mm) via wire-electrode cutting. The diameter and length of the shafts are 10 and 550 mm, respectively. The crack locations are as follows: for single case, 310 mm from the left bearing, and for the double crack case, 180 mm and 310 mm. We totally measured each sampling point in two speeds 100 times and obtained 100 mode shape data in each revolving speed. The speed control voltages are selected as 1 and 1.5 V, which correspond to the speeds 1920 and 2910 rpm, respectively. Owing to the limitations of the measurement instruments, we used three eddy current displacement transducers, Bently Nevada 3300 Xl, to capture the displacement when the shaft is rotating. Concretely, the transducers are moved by hand within an effective area, with about 432.5 mm, and measurement mode shapes contain 18 sampling points.

Figure 16.

Setup of the cracked rotating shaft experiments: (a) experimental setup and (b) measurement details.

We adopt the transferring parameter method, as shown in Figure 14, to retrain a novel network for this experiment. Then, the trained network is used to detect the crack location, and the statistical results are evaluated as shown in Figure 17. Here, we only select the displacement mode shapes at the first critical speed to verify the proposed method owing to the limitation of the experimental platform. According to the figure, the proposed method can obtain reliable accuracies, of more than 70% in single-crack scenarios and more than 50% in multiple-crack scenarios.

Figure 17.

Accuracies of the single and multiple cracked rotating shafts at different speeds.

Discussion

Comparing with existing hand-engineered-based methods

To demonstrate the prowess of our proposed deep-learning-based method, we compare the performance of our method to four existing algorithms (mode shape curvature,¹⁴ flexibility,¹⁵ modal strain energy (MSE),⁴⁷ and a wavelet-based method).³ The specimen used for comparison is a simply supported beam with 40 elements. A 10% stiffness reduction is introduced in the 24th element and the first mode shape is displayed in Figure 18(a). The mode shape curvature method cannot detect the damage under a noisy condition (Figure 18(b)), as a result of the amplification of noise. Although the other three methods can detect the damage (Figure 18(c)–(e)), some drawbacks still exist: (1) the data before and after the occurrence of structural damage are needed when we use the flexibility and MSE; (2) MSE needs to know the knowledge of material properties and boundary conditions; and (3) for the wavelet-based method, the boundary effect of wavelet on a limited signal may cause the abnormal wavelet coefficients at the ends of results which will narrow the detection range. However, our proposed method can detect the damage directly and accurately without any drawbacks as mentioned above (Figure 18(f)).

Figure 18.

(a) The specimen mode shape and (b) its corresponding results via mode shape curvature, (c) flexibility, (d) modal strain energy, (e) wavelet-based method, and (f) the proposed deep-learning method.

Comparing with other deep-learning architectures

Deep learning has a series of breakthroughs for SHM. Recent evidences reveal that deep CNN can extract damage feature from the vibration signals. These methods make use of deep-learning model to detect damage according to various forms of vibration signals such as the time domain waveforms^9,28,48 and the frequency domain waveforms.⁴⁹ To show the advantage of our method, we compare the performance of the detection capability of two CNN architectures^28,49 with our proposed method. During the comparison, all the architectures are trained via the same training set (including 0%–15% data missing). Figure 19 displays the accuracy of the three different CNN architectures in noisy conditions, with SNR ranging from 60 to 120 dB. The accuracy is the average accuracy of $10^{5}$ samples, and each sample contains 0%–15% missing data. It shows that the proposed CNN architecture has the best performance among the three architectures. Two reasons contribute to better performance: (1) the proposed architecture has a deeper structure which can extract more abstract features, and (2) the introductions of multi-scale module and residual module are helpful to solve the measurement noise and data-missing problems.

Figure 19.

Assessments of the damage detection accuracies by using different CNN architectures.

Application in continuous beams

Furthermore, we retrain the network for continuous beams by using the parameters transferring methodology. A continuous beams data set with $1 0^{5}$ samples via FEM is generated. Each sample in data set is contaminated by noise and has 0%–30% missing data. Here, we select three representative scenarios and the details are listed in Table 3.

Table 3.

Damage scenarios of continuous beams.

Test No.	Dimension ( $l \times m \times n$ )	Damaged element No.	Damage degree	Modal order	SNR (dB)
1	$0.7 \times 0.05 \times 0.01$	25	8%	1	80
2	$0.7 \times 0.03 \times 0.01$	10, 30	10%, 12%	2	70
3	$0.5 \times 0.05 \times 0.01$	15, 25	8%, 10%	3	60

SNR: signal-to-noise ratio.

Figure 20 displays the noisy mode shapes of three structure types and the corresponding detection results. It can be found that obvious crests appear at the preset damage locations. This result demonstrates that a trained network can also be applied to other similar structure types by using the parameters transferring methodology, with a relatively less training cost.

Figure 20.

(a–c) Mode shapes of continuous beams with different boundary conditions and (d–f) the corresponding detection results. The damaged elements are marked in black in FEM models.

Learned feature in visualization

Although deep learning has achieved a remarkable development in many fields, how a network model works is still inscrutable. This problem is particularly severe when designing the architecture of a network and analyzing the network reliability. Therefore, study of explaining the output of each module and layer is significant as it helps us to analyze the working principles of the target task.^24,28

Figure 21 displays the features of the convolution kernels with different sizes in the multi-scale module. The first mode shape of the experimental beam, as shown in Figure 13(a), is selected as the input data. Here, we re-select the 28th measurement point as the missing data to demonstrate the change regularity, the location of the point is approximately 155 mm from the left fixed end. It is apparent from the figure that a singular peak appears at the location of data missing, and the shape characteristic of each channel is similar to that of the input signal. Hence, we argue that the feature learned in this module is the shape characteristic of the input signal. Because different convolutional kernels have various receptive fields, the learned features of different convolutional kernels based on the same input vary.

Figure 21.

Features that different convolution kernels learned in the multi-scale module.

Figure 22 displays the learned features of other modules and layers. It can be seen that with the deepening of the network, the learned features become increasingly abstract. In the last three convolutional layers, the fluctuations of each feature vector become gradual, and two singular peaks (i.e. damage features) appear in the feature map of the last convolutional layers.

Figure 22.

Features that different layers learned.

The analysis of the feature vectors shows that a deep-learning model is composed of multiple layers, which can learn representations of data with multiple levels of abstraction. Moreover, convolutional kernels with different sizes can learn various features from the same input. As the network becomes deeper, the learned features become increasingly abstract, and the mapping between the input and the target gradually emerges in the last few layers. Although the proposed network shows satisfactory performance in damage detection tasks, the physical interpretations of the abstract output in the middle of the framework still cannot be found according to current information. We will focus on this problem in our future study.

Conclusion

A deep-learning model based on CNN is proposed to deal with the problem of damage identification without any hand-engineered features. This model can extract damage features from noisy and incomplete mode shape data, and its detection capability is validated via numerical simulations and laboratory experiments. A comparison of the results from our proposed method with other network architecture shows that the proposed method has accuracy improvements of 10% over the other network architectures.

In the framework of this computation model, multi-scale representations of damage are extracted via a multi-scale learning module. In addition, a set of stacked RLMs are used to learn more abstract features while optimizing the training processes. Finally, the global average pooling technique is utilized to realize the regression process of damage identification. In designing the architecture and training strategy of the network, scenarios such as noise interference, measurement data missing, and multiple damage are considered; we take corresponding measures, including multi-scale representation learning, residual learning, and dropout technique. Furthermore, to discuss what representations the network learns, a measurement signal is used to illustrate the feature extracted in each layer/module.

At this stage, the proposed method can achieve a predictive effect, but the results are based on the data from numerical simulations and laboratory experiments. As the next step of this research, we will focus on actual structures and discuss more complex scenarios.

Footnotes

Acknowledgements

Authors would like to express gratitude to Prof. A. Katunin for sharing the benchmark data in the website. Z.X. acknowledges the support of the National Natural Science Foundation of China (NO. 51275385).

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Park

. Damage detection of building structures under ambient excitation through the analysis of the relationship between the modal participation ratio and story stiffness. J Sound Vib 2018; 418: 122–143.

Wodecki

Michalak

Zimroz

. Optimal filter design with progressive genetic algorithm for local damage detection in rolling bearings. Mech Syst Sig Pr 2018; 102: 102–116.

Cao

M-S

Ren

W-X

, et al. A concept of complex-wavelet modal curvature for detecting multiple cracks in beams under noisy conditions. Mech Syst Sig Pr 2016; 76–77: 555–575.

Yan

Cheng

, et al. Development in vibration-based structural damage detection technique. Mech Syst Sig Pr 2007; 21(5): 2198–2211.

Shi

Qiao

. A new surface fractal dimension for displacement mode shape-based damage identification of plate-type structures. Mech Syst Sig Pr 2018; 103: 139–161.

Shahsavari

Chouinard

Bastien

. Wavelet-based analysis of mode shapes for statistical detection and localization of damage in beams using likelihood ratio test. Eng Struct 2017; 132: 494–507.

Salawu

. Detection of structural damage through changes in frequency: a review. Eng Struct 1997; 19(9): 718–723.

Maia

NMM

Silva

JMM

Almas

EAM

, et al. Damage detection in structures: from mode shape to frequency response function methods. Mech Syst Sig Pr 2003; 17(3): 489–498.

Abdeljaber

Avci

Kiranyaz

, et al. Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks. J Sound Vib 2017; 388: 154–170.

10.

Lei

Jiang

. Structural damage detection with limited input and output measurement signals. Mech Syst Sig Pr 2012; 28: 229–243.

11.

Cao

Ostachowicz

, et al. Two-dimensional curvature mode shape method based on wavelets and Teager energy for damage detection in plates. J Sound Vib 2015; 347: 266–278.

12.

Qiao

Lestari

, et al. Curvature mode shape-based damage detection in composite laminated plates. Compos Struct 2007; 80(3): 409–428.

13.

Feng

. Output-only damage detection using vehicle-induced displacement response and mode shape curvature index. Struct Control Hlth 2016; 23(8): 1088–1107.

14.

Pandey

Biswas

Samman

. Damage detection from changes in curvature mode shapes. J Sound Vib 1991; 145(2): 321–332.

15.

Pandey

Biswas

. Damage detection in structures using changes in flexibility. J Sound Vib 1994; 169(1): 3–17.

16.

Grande

Imbimbo

. A multi-stage approach for damage detection in structural systems based on flexibility. Mech Syst Sig Pr 2016; 76–77: 455–475.

17.

Farrar

Doebling

Cornwell

, et al. Variability of modal parameters measured on the Alamosa Canyon bridge. In: Proceedings of the 15th international modal analysis conference, Orlando, FL, 3–6 February 1997, pp. 257–263. Los Alamos, NM: Los Alamos National Laboratory.

18.

Reynders

Wursten

De Roeck

. Output-only structural health monitoring in changing environmental conditions by means of nonlinear system identification. Struct Health Monit 2014; 13(1): 82–93.

19.

Erazo

Sen

Nagarajaiah

, et al. Vibration-based structural health monitoring under changing environmental conditions using Kalman filtering. Mech Syst Sig Pr 2019; 117: 1–15.

20.

LeCun

Bengio

Hinton

. Deep learning. Nature 2015; 521: 436–444.

21.

Bishop

. Pattern recognition and machine learning. New York: Springer, 2006.

22.

Lecun

Boser

Denker

, et al. Handwritten digit recognition with a back-propagation network. In: Touretzky

(ed.) Advances in neural information processing systems 2. San Francisco, CA: Morgan Kaufmann, 1990, pp. 396–404.

23.

Cha

Choi

Büyüköztürk

. Deep learning-based crack damage detection using convolutional neural networks. Comput-Aided Civ Inf 2017; 32(5): 361–378.

24.

Jia

Lei

, et al. Deep normalized convolutional neural network for imbalanced fault classification of machinery and its understanding via visualization. Mech Syst Sig Pr 2018; 110: 349–367.

25.

Shao

Jiang

Zhang

, et al. Rolling bearing fault feature learning using improved convolutional deep belief network with compressed sensing. Mech Syst Sig Pr 2018; 100: 743–765.

26.

Padil

Bakhary

Hao

. The use of a non-probabilistic artificial neural network to consider uncertainties in vibration-based-damage detection. Mech Syst Sig Pr 2017; 83: 194–209.

27.

Zeiler

Fergus

. Visualizing and understanding convolutional networks. In: Proceedings of the 13th European conference on computer vision, Zürich, 6–12 September 2014, pp. 818–833. Cham: Springer.

28.

Lin

Nie

. Structural damage detection with automatic feature-extraction through deep learning. Comput-Aided Civ Inf 2017; 32(12): 1025–1046.

29.

Zhang

Ren

, et al. Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the 2015 IEEE international conference on computer vision (ICCV), Santiago, 7–13 December 2015, pp. 1026–1034. New York:IEEE.

30.

Gatys

Ecker

Bethge

. A neural algorithm of artistic style. Nat Commun 2015, https://arxiv.org/abs/1508.06576

31.

Szegedy

Liu

Jia

, et al. Going deeper with convolutions. In: Proceeding of the 2015 IEEE conference on computer vision and pattern recognition, 7–12 June 2015, pp. 1–9. New York: IEEE.

32.

Szegedy

Ioffe

Vanhoucke

, et al. Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the 31st AAIA conference on artificial intelligence, San Francisco, CA, 4–9 February 2017, pp. 4278–4284. Palo Alto, CA: Association for the Advancement of Artificial Intelligence (AAIA).

33.

Tian

Zili

. Structural health monitoring using dog multi-scale space: an approach for analyzing damage characteristics. Smart Mater Struct 2018; 27(3): 035009.

34.

Cao

Ostachowicz

, et al. Damage identification for beams in noisy conditions based on Teager energy operator-wavelet transform modal curvature. J Sound Vib 2014; 333(6): 1543–1553.

35.

Lindeberg

. Scale-space theory: a basic tool for analyzing structures at different scales. J Appl Stat 1994; 21(1): 225–270.

36.

Guo

. Data fusion of multi-scale representations for structural damage detection. Mech Syst Sig Pr 2018; 98: 1020–1033.

37.

Zhang

Ren

, et al. Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR), Las Vegas, NV, 27–30 June 2016, pp. 770–778. New York: IEEE.

38.

Lin

Chen

Yan

. Network in network. In: Proceedings of the international conference on learning representations, 2014, https://arxiv.org/pdf/1312.4400.pdf

39.

Divakar

Babu

. Image denoising via CNNs: an adversarial approach. In: Proceedings of the new trends in image restoration and enhancement, computer vision and pattern recognition workshops, 2017, pp. 1076–1083, http://openaccess.thecvf.com/content_cvpr_2017_workshops/w12/papers/Divakar_Image_Denoising_via_CVPR_2017_paper.pdf

40.

Srivastava

Hinton

Krizhevsky

, et al. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 2014; 15(1): 1929–1958.

41.

Kingma

. Adam: a method for stochastic optimization. In: Proceedings of the international conference on learning representations, 2015, https://arxiv.org/pdf/1412.6980.pdf

42.

Abadi

Agarwal

Barham

, et al. Tensorflow: large-scale machine learning on heterogeneous distributed systems. In: Proceedings of the Arxiv: distributed, parallel, and cluster computing, 2015, https://arxiv.org/abs/1603.04467

43.

Tondreau

Deraemaeker

. Numerical and experimental analysis of uncertainty on modal parameters estimated with the stochastic subspace method. J Sound Vib 2014; 333(18): 4376–4401.

44.

Katunin

. The construction of high-order B-spline wavelets and their decomposition relations for fault detection and localisation in composite beams. Sci Prob Mach Oper Maint 2011; 46: 43–59.

45.

Katunin

. Identification of multiple cracks in composite beams using discrete wavelet transform. Sci Prob Mach Oper Maint 2010; 45: 41–52.

46.

Oquab

Bottou

Laptev

, et al. Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the 2014 IEEE conference on computer vision and pattern recognition, Columbus, OH, 23–28 June 2014, pp. 1717–1724. New York: IEEE.

47.

Shi

Law

Zhang

. Structural damage localization from modal strain energy change. J Sound Vib 1998; 218(5): 825–844.

48.

Pathirage

CSN

, et al. Development and application of a deep learning–based sparse autoencoder framework for structural damage identification. Struct Health Monit 2018; 18: 103–122.

49.

Wang

, et al. A novel deep learning-based method for damage identification of smart building structures. Struct Health Monit 2019; 18(1): 143–163.