A novel structural damage identification method based on the acceleration responses under ambient vibration and an optimized deep residual algorithm

Abstract

Recently, structural health monitoring (SHM) methods for civil structures have been investigated widely, especially Deep Learning (DL)-based methods. However, it is usually difficult to fully train a deep neural network, and thus, typical DL-based SHM methods are limited in terms of performance. While addressing these issues, in this paper, a novel methodology is proposed for smart damage identification of frame structures. The newly proposed SHM method is based on raw time-domain structural response signals and deep residual network (DRN). The introduced DRN algorithm has been designed and tested in an effective way for extracting and learning the optimum features of the 1D raw ambient vibration acceleration signals, without any need for engineered features. Also, the network’s performance has been optimized using Bayesian optimization, which clearly enhances the network’s accuracy and information flow across it. Next, the outputs of DRNs are further utilized through new methods for damage size estimation and damage localization. The proposed methodology has been evaluated using the datasets of numerical and experimental frames of the SHM benchmark problem and the dataset of a real-world full-scale truss bridge. The results show that the proposed method is capable of detecting, localizing, and quantifying structural damage accurately for all of the simulated cases of the two examples. Furthermore, conducted comparison studies have approved that the new approach is more efficient than other machine learning-based methods, and it can overcome the major limitations of Artificial intelligence-based SHM models.

Keywords

Deep residual network convolution neural network deep learning structural damage detection structural health monitoring ambient vibration

Introduction

Civil infrastructure is an essential part of a nation’s economy. They are susceptible to gradual deterioration and structural damage throughout their lifetime due to numerous factors, such as natural disasters. Therefore, it is vital to assess structural status and assure structural safety. Structural health monitoring (SHM) is a field that deals with damage identification and condition evaluation of structural systems. Structural health monitoring evaluations are often conducted by recording samples of the structural responses during the service-life or after an unexpected accident that probably impacts the structure’s functionality.¹

SHM methods

Generally, structural condition evaluation has been investigated using many evaluation methods; most of the applied methods are non-destructive evaluations (NDE). The main limitations of NDE methods can be the high dependency on experts, in addition to the difficulty to access the critical parts of the structural systems.² On the other hand, structural dynamic analysis was applied widely for the mentioned purpose, which is generally through the use of structural mathematical or physical models. The issue of complicated nonlinearity has been raised in most real-life implementations. This nonlinearity can be initiated from different factors, such as materials and geometries, which creates serious complexity and high cost for the used analytical model. As a result, many researchers moved to focus on NDE methods, which are based on the structural responses acquired by utilizing a network of sensors from a structure under different structural conditions.^3–5

In recent 30 years, vibration-based methods have been developed for structural damage detection.⁶ These methods have been used for damage detection in different structural systems, and even for large and sophisticated structures.⁷ Generally, vibration-based methods are based on the concept that it is possible to consider the structural system as a dynamic system, having structural parameters such as stiffness, mass, and damping. Accordingly, if any damage occurs to the structural system, its structural parameters will certainly be affected. Similarly, the frequency-response function and modal parameters of the structural system will also be affected. Therefore, the initial damage occurrence can be monitored by using the variation of the structural modal parameters as a damage indicator.^8–15

However, vibration-based structural damage detection methods are divided into: (a) conventional category, which includes the structural damage detection methods that just use the mechanical parameters of the structure itself such as natural frequency, damping ratio, mode shape, etc. These methods mostly require performing modal analysis experimentally or measuring the transfer function. Thus, it is not suitable for online damage detection, and not well-fitting for in-service structures. Moreover, manual procedures or instrumentations are required for acquiring the essential measurements for these methods. For this reason, the researchers have not paid much attention to these methods recently.¹⁶ The other vibration-based structural damage detection method is (b) intelligence-based category, which includes the structural damage detection methods that utilize the online structural response signals of in-service structures. Accordingly, it is reasonable and possible to construct automated structural damage detection for in-service structures by getting the benefit of these methods realization.¹⁷ Artificial neural network (ANN) is a popular sort of intelligence-based method.^18–20 This research field has recently turned into a significant hot area due to the achieved results.^21,22

DL in vibration-based SHM

The existing intelligence-based methods are using the concept of meaningful features extraction and selection to diagnose the different patterns, leading to the need for advanced signal processing tools, which is considered as a main shortcoming.^23–25 Thus, a serious need to develop a powerful model continues, and the model is capable of extracting and learning the features of the raw response signals to build an intelligent tool for damage identification.

On another note, deep learning (DL) is considered the era trend in machine learning (ML), and it comes under the field of artificial intelligence (AI). Deep learning models have the potential power to overcome the potential weakness of the traditional machine learning methods and the conventional intelligence-based SHM methods.²⁶ In recent years, some essential applications of DL are in computer vision, natural language processing, SHM, etc. Deep learning models have been studied extensively, particularly the convolution neural network (CNN).^27–29

Convolution neural network is a purely supervised DL model that combines feature extraction and learning in one model. Different from the conventional ANNs, neurons of CNN’s layer can be organized along three dimensions: length, width, and height. Therefore, the CNN has been used in several fields such as power engine fault detection,^30,31 electrocardiogram signals classification,^32,33 and image object detection.³⁴ The results presented in the references^35–42 show CNN model’superiority in terms of performance accuracy and speed compared to the standard methods. Recently, CNN has been applied in the SHM tasks and achieved some recognized achievements.^43–52

Despite the abundant achievement that has been accomplished by CNN, it is generally challenging to fully train a deep model due to the gradient vanishing or exploding during back-propagation. Deep residual network (DRN) has arisen as a part of extremely deep architectures providing competitive precision and outstanding performances.⁵³ Deep residual network has achieved state-of-the-art performance on widely developed ML tasks. The unit skip connection is presented in the DRN architecture, which lets the data maps transmit directly across the network and the chances of optimal feature extraction are higher in this way. Therefore, residual learning is a highly promising signal processing tool. The DRN was successfully applied in machine fault diagnosis and approved to be better than other DL algorithms,^54–56 also, it was successfully used in the image-based SHM,^57–59 non-image SHM methods after conducting modal analysis, and has shown to be more efficient than traditional DL algorithms.^60,61

However, in the AI-based and vibration-based SHM, many issues still need to be explored and solved, for example, (1) the artificial source of excitation has to be used to excite the monitored structure; (2) the noise influences on the measured response signals for small structural damage identification; (3) learning process of the structural damage features is very adaptable and inconstant; (4) the training process of AI-based SHM algorithm requires a large dataset; (5) AI-based SHM algorithm is computationally costly and depends tightly on the graphical processing unit (GPU); (6) it is generally challenging to fully train a deep model due to the gradient vanishing or exploding during back-propagation; and (7) rare study has been carried out on deep DRN for vibration-based SHM applications, especially for damage recognition based on the raw ambient vibration acceleration signals acquired from an external complex structure.

Motivated by the issues above, a novel SHM methodology has been developed for accurate damage identification. Regarding evaluation for the efficiency and robustness of the proposed methodology, comprehensive experiments involve numerical and experimental studies are conducted. In the numerical studies, acceleration response signals generated by a finite element model are used as the input.⁶² In the experimental studies, firstly, the SHM benchmark problem phase II is utilized, which had been presented by International Association for Structural Control (IASC)-American Society of Civil Engineers (ASCE) SHM Task Group, and publicly published in 2003 to offer an integrated evaluation bed for testing the new approaches,⁶³ secondly, the vibration data of a steel truss bridge were used, which were acquired from a progressive damage test conducted on a Japanese bridge subjected to an ambient vibration excitation.⁶⁴

In this work, we intend to eliminate the mentioned limitations by (1) developing DRN model for effective SHM using the raw vibration signals, and (2) optimizing DRN performance through a precise method of dataset arrangement and model training method.

Contributions of the present study

The key contributions of this paper can be summarized as follows:

• This work introduces a novel damage identification method. The method utilizes DRN model for feature extracting and learning from the raw acceleration response signals, without any need for engineered features, or signal prepossessing tools.

• Full training and investment are promoted for the network’s power in feature extraction, whereas the convolutional filters are enabled to move over the 1D data in two directions and harvest the optimal features. The entire computational process is achieved without any need for GPU.

• A practical solution for the issue of excitation resources in the vibration-based SHM method. The proposed approach works on the raw acceleration signals measured from a structure under ambient excitations, which consist of wind, pedestrians, and traffic.

• A fixed solution for the difficulty of large dataset size requirements. The new method needs to perform only a single measurement session for each structural condition, which is a reasonable size of the training and testing dataset.

• The proposed method can identify the structural condition, damage sizes and locations integrated with damage index (DI). Both the method and DI are based on the DRN classification outputs.

After highlighting the contributions of our current work. In the Methodology and DRN-based SHM Framework sections, the proposed method is described in detail. In the Numerical simulations and Experimental verification sections, descriptions for the SHM benchmark studies I, II, and steel truss bridge, and the achieved results are provided. In the Discussions and Comparison study section, the experimental results are analyzed and discussed, and the proposed approach is compared with the state-of-art methods. Finally, conclusions are shown in the Conclusions section. Table 1 lists the notations used in this work.

Table 1.

List of notations used in this work.

Symbol	Description
X	Input signal vector
Y	Target class
N	Number of samples in the dataset
$f (\cdot)$	An activation function
$O_{j}^{l - 1}$	The output of the l−th layer
$W_{i j}^{l}$	The input into the next layer through the feature map i
$O_{j}^{l}$	The jth feature map of the l−th layer
*	The convolution operator
D _in	The side length of the l−th input layer
C	The side length of the filter
D _out	The side length of the lth output layer
S _conv	The stride (both in height and width)
$b_{j}^{l}$	An additive bias to each output map
M	The size of the input
Q	The size of output
S	The stride
LF	The length of the filter
LP	The left padding
RP	The right padding
S (o)	The maximizing function of every class with a specific size in the feature map named max pooling
$δ_{j}^{l}$	The deviation in the multiplier for the jth feature map of the l layer
$P_{p o o l}$	The side length of the pooling operator
$S_{p o o l}$	The stride (both in height and width)
$x_{j}^{l}$	The jth neuron in fully connected layer l
$x_{i_{1} i_{2} i_{3}}^{l - 1}$	A neuron in the preceding 3D feature map $l - 1$ , $i_{1}, i_{2}, i_{3}$ refer to height, width, and channel coordinate, respectively
$W_{i_{1} i_{2} i_{3}}^{l}$	The weight of neuron $x_{i_{1} i_{2} i_{3}}^{l - 1}$ between $l - 1$ and l layers
$y_{k}$	The ouput of softmax layer which is the probability for a specific class k
K	The class value
$W_{k}^{l}$	The $k - t h$ row weight vector
$l_{k}^{n}$	The corresponding $k - t h$ label of a sample n
$\emptyset_{j}^{l}$	The sensitivity of neuron
$P o_{(x_{i})}$	The affiliation probability of each sample to a specific class of the dataset
SoD	The size of damage
D _j	The specified size of damage for the jth damage class
DI	Damage index
a	The number of samples classified as a healthy class
b	The number of samples classified to belong to any of the damage classes

Methodology

Feed-forward operation

Let the samples of the vibration data be ${X, Y}$ , assume $X$ and $Y$ are the input signal vector and the target class, respectively. The structural conditions are called as the target classes. The dataset involves number of samples which is $N$ .⁶⁵ During the feed-forward operation, the data set will pass-through the network’s layers. The important advantage of deep networks such as DRN is its ability to extract features of signals automatically through its layers one after one. The feature of each of the network’s layers is considered as a feature map for the next layer. The feature extractors are weights and bias linking two layers. Then, the spatial property of the last feature is based on the neuron number of the final layer of the network. The weights and bias for all of the layers are updated using gradient descent to achieve the highest performance accuracy. The configuration and the working concept of the DRN’s main layers have been explained in the following sections.

Convolution layer

Convolution layer as per its functionality generates a new representation of features called features maps, the kernel (filter) convolves the feature maps in the mentioned layer.⁶⁶ Convolution of several input feature results in the output map. The definition of feature map progress in a convolution layer is

O_{j}^{l} = f (\sum_{i \in M_{j}} σ_{i j} O_{i}^{l - 1} * W_{i j}^{l} + b_{j}^{l})

(1)

where

O_{i}^{l - 1}

is the output of the

(l - 1) t h

layer,

W_{i j}^{l}

is the input into the next layer through the feature map i, and

O_{j}^{l}

is the jth feature map of the lth layer.

f (\cdot)

for the sigmoid function, M _j is a selection of input maps and is the kernel weight linking the

(l - 1) t h

feature map of the

(l - 1) t h

layer with the jth feature map of the convolution layer, and * denotes the convolution operator. An additive bias

b

is given to each output map. Through several kernels, the input feature maps are related to a specific output feature map convolved. During the filter convolution on an input, a multiplication conducted element wisely through the input receptive field, then found productions summation. Next, the filter moves according to the stride option, repeating the same procedure. The in-plane size of the output feature maps is determined through

D_{o u t} = \frac{D_{i n} - C}{S_{c o n v}} + 1

(2)

where

D_{i n}

is the side length of the

(l - 1) t h

input layer; C is the side length of the filter;

D_{o u t}

is the side length of the lth output layer; and

S_{c o n v}

is the stride (both in height and width). For example, in this work, the input D _in = 100, the input layer convolved with a 7 × 1 filter with stride 1, then D _out = 94. Figure 1 summarizes the convolution operation of this study, demonstrating a convolution between the input layer of 5 × 1× 1 (height, width, channel) and N filters of size 2 × 1 × 1.

Zero-padding method

Since it is crucial to control the size of the feature, the basic method used to manage the feature dimension is the padding technique. In this work, the zero-padding method is used to let the convolutional filters move over the data in two directions and to control the dimension, preventing the dimension loss. The left padding $L P$ and right padding $R P$ numbers are computed using the following equations

N = c e i l (\frac{M}{C})

(3)

P T = (Q - 1) \times S + L F - M

(4)

L P = f l o o r (\frac{P T}{2})

(5)

R P = P T - L P

(6)

where M is the size of the input, Q is the size of output, LF is the length of the filter, and S is the stride. Note that

c e i l (\cdot)

is the ceil function and

f l o o r (\cdot)

is the floor function. When applying the zero-padding method, the convolution operation would be performed over the data and the zeros are added around the data. In this study, the zero-padding method enhances the feature extraction process significantly in addition to the dimension control. A simple example is explaining the one dimension zero paddings in Figure 2. The parameters are

M = 6

S = 2

L F = 3

, then the padding result will be

L P = 1

R P = 1

Q = 4

Batch normalization, and rectified linear unit

Since the convolution operation is a linear process, it bounds the capability of learning input’s complicated nonlinearity. An activation function is used on the intermediary extracted features of a convolution layer to present nonlinear behavior and produce the proper feature maps. In this context, the rectified linear unit has shown as an effective activation function for network training, which prevents the gradient vanishing issue with sigmoid functions.⁶⁷ Accordingly, due to the introduced nonlinearity, as the model goes deeper the operation slows down. To speed up training, a batch normalization layer is located between the convolution layer and ReLU layer (nonlinearity).⁶⁸ Firstly, it normalizes the input’s activation by finding the mean and divide it by the standard deviation of each mini-batch. Then, the input shifts and scales by offset and scale factor γ and Β, respectively. During the training process, the model learns and updates parameters γ and Β. Note that the batch normalization layer normalizes the activations and gradients propagating through the model, which simplifies the network training optimization issue.

Pooling

The pooling layer aims to decrease the size of feature maps using the sub-sampling in a particular method. This method results in more in-depth subtracted representations and overfits prevention. The following formula is for producing the features maps of the sub-sampling layers

O_{j}^{l} = f (δ_{j}^{l} S (O_{j}^{l - 1} + b_{j}^{l}))

(7)

where

S (o)

is the maximizing function of every class with a specific size in the feature map named max pooling, and

δ_{j}^{l}

is the deviation in the multiplier for the jth feature map of the lth layer. The in plane-size of the output feature maps is determined through

D_{o u t} = \frac{D_{i n} - P_{p o o l}}{S_{p o o l}} + 1

(8)

where

P_{p o o l}

is the side length of the pooling operator and

S_{p o o l}

is the stride (both in height and width).

Classification unit

The classification unit backbone is two layers (fully connected and softmax). First, the fully connected layer has full connection to the activation of the preceding layer through neurons having the ability to reshape their link from 3D to 1D connection as

x_{j}^{l} = f (\sum_{i_{1} = 1}^{H e i g h t} \sum_{i_{2} = 1}^{W i d t h} \sum_{i_{3} = 1}^{C h a n n e l} x_{i_{1} i_{2} i_{3}}^{l - 1} W_{i_{1} i_{2} i_{3}}^{l} + b_{j}^{l})

(9)

where

x_{j}^{l}

is the jth neuron in fully connected layer l,

x_{i_{1} i_{2} i_{3}}^{l - 1}

is a neuron in the preceding 3D feature map

l - 1

i_{1}, i_{2}, i_{3}

refer to height, width, and channel coordinate, respectively.

W_{i_{1} i_{2} i_{3}}^{l}

is the weight of neuron

x_{i_{1} i_{2} i_{3}}^{l - 1}

between

l - 1

and l layers;

b_{j}^{l}

is the bias of input to

x_{j}^{l}

; and

f (o)

refers to an activation function.

Second, the feature vectors extracted by a fully connected layer are passed into the softmax layer. Their formulations are

y_{k} = [P (p r e d i c t i o n = k | x^{l - 1}; W_{k}^{l})] = \frac{e^{W_{k}^{l} x^{l - 1}}}{\sum_{m = 1}^{K} e^{W_{m}^{l} x^{l - 1}}}, k \in [1, K]

(10)

x^{l - 1} = {[x_{1}^{l - 1}, \dots x_{i}^{l - 1}, \dots x_{I}^{l - 1}]}^{T}

(11)

W_{k}^{l} = [W_{k 1}^{l}, \dots W_{k i}^{l}, \dots W_{k I}^{l}], W^{l} = {[W_{1}^{l}, \dots W_{k}^{l}, \dots W_{K}^{l}]}^{T}

(12)

Let layer l be the softmax layer. Its ouput $y_{k}$ is the probability for a specific class k. Accordingly, K is the class value. $\sum_{k = 1}^{K} y_{k} = 1$ . $x^{l - 1}$ is the feature vector of size $I \times 1$ , and $W_{k}^{1}$ is the kth row weight vector of size $1 \times I$ . Figure 3 presents an illustration of the process.

Back-propagation and gradient descent

Back-propagation is used to compute the loss function gradient for weights of all the layers, and the objective function is specified using a loss function of squared-error (SE) in this study. Generally, starting with a good initialization and performing grid search for the optimal hyper-parameters, the deep model performs better when using the SE loss function, and could beat the same model using cross-entropy (CE).⁶⁹ Since we adopted the methods of He et al. for initialization⁷⁰ and Bayesian optimization for finding the optimal parameters,⁷¹ therefore, we have used SE as loss function here. This problem is multi-class with m classes and N dataset samples. As an explanation, the loss function is presented in equation (13), after training one sample, n. The overall loss function is obtained by summing the loss functions of the different samples, as follows

E^{n} - 1 / 2 {\sum_{k = 1}^{m} (l_{k}^{n} - y_{k}^{n})}^{2}

(13)

where n is the training sample index, and k is the label index. Here, in a sample n the corresponding kth label is

l_{k}^{n}

, and the output layer unit value is

y_{k}^{n}

In problems of classification of multi-class, a vector presents the output mostly, the positive is the class output node related to the dimension of the input only. The remaining nodes of the class are negative or zero, depending on the activation function of the output layer. The tanh function is 1 and the sigmoid function is 0. Subsequently, in a layer from neurons u to neurons v, the sensitivity of neuron $\emptyset_{j}^{l}$ is computed, and the function of loss gradient for the kernel weights is calculated by back-propagation as follows

g r d = \frac{\partial E}{\partial W_{i j}^{l}} = \sum {(\emptyset_{j}^{l})}_{u, y} (O_{j}^{l - 1} W_{i j}^{l})

(14)

Residual learning

He et al.⁵³ first suggested the residual network in his work . In recent years, DRN has achieved considerable progress in image processing.⁷² The residual network typically has three important characteristics. The first is the implementation of identity skip links, which allows data to flow directly from other layers to subsequent layers. Second, with skip links, the depth of the network structure can be largely expanded. Finally, a single layer has a slight influence on the testing performance of DRN. A residual learning block is shown in Figure 4.

Considering an input x which passes through a limited stacked nonlinear layers of the conventional CNN, as illustrated in Figure 4(left), the requested essential mapping can be defined as $H (x)$ . By presenting a skip connection as shown in Figure 4 also, the network can directly use an identity mapping x, whereas the stacked nonlinear layers can fit a so-called residual mapping $F (x) = H (x) - x$ , assuming that $H (x)$ and x have the same dimension. Then, the original mapping $H (x)$ can be reformatted as $F (x) = H (x) + x$ . In other words, the layers in a traditional CNN are learning the true output $H (x)$ , while the layers in a DRN are learning the residual $F (x)$ . It is observed that it is easier to optimize the residual mapping $F (x)$ than the original mapping in practice. It should be noted that the dimensions of $F (x)$ and x are supposed to be the same. When they are not matched, a linear projection of x can be used for the addition.⁷²

Formally, the residual block presented in Figure 2 can be defined as

y_{l} = X_{l} + f_{l} (X_{l}, W_{l})

(15)

X_{l + 1} = f (y_{l})

(16)

where

X_{l + 1}

and

X_{l}

are the input and the output of the lth block in the network, F is also the residual function, W is the parameter to be learned of the block, and

f (o)

is the nonlinear activation, for example, the ReLU function. One is able to construct a deep DRN, through sequentially stacking residual blocks.

Training of the DRN-SHM model

After deciding the neural network configuration, the DRN must be trained using the labeled dataset to obtain internal weights and biases. Then, if the network is trained, it can be used to assess the structural condition and to identify damage. However, a DL model relies heavily on deciding the ideal hyper-parameters, and it is vital for DL model optimization. In this context, Bayesian optimization is an important way of solving computationally costly functions.⁷¹ It can be used to address a dilemma without the expression of a closed-form. It may also be applied to work with computing costly, for example, derivatives are hard to determine or the function is non-convex. In this work, the aim of the optimization process is to achieve the optimum value (accuracy) of an unknown function $f (o)$ at the sampling point using Bayesian optimization

X^{+} = \underset{X \in A}{\arg \max f (X)}

(17)

where A denotes the search space of x. Bayesian optimization originates from Bayes’ theorem,^71,73 that is, given evidence data E, the posterior probability

P (M | E)

of a model M is proportional to the likelihood

P (E | M)

of over-serving E given model M multiplied by the prior probability of

P (M)

P (M / E) \propto P (E / M) P (M)

(18)

The previous formula returns the basic concept of Bayesian optimization. Also, gradient-based optimization algorithms typically rely on the initial value θ, and the incorrect initial values continue to induce poor convergence and are stuck optimally at the local level to increase the consistency of the initial stage. The initialization technology used determines the input and output neuron numbers in the initialization scale and preserves the same input and output variance to avoid the gradient explosion and eruption.⁷⁰ In summary, the specific flow chart for training the DRN-based SHM model can be shown in Figure 5.

DRN-based SHM framework

Deep learning models have better performance in terms of accuracy and generalization, compared to conventional machine learning techniques such as back-propagation neural network and support vector machine (SVM). Deep learning approaches can automatically extract nonlinear features. In this context, DRN is one of the promising deep neural network structures. It was firstly introduced for image recognition.⁵³ Later on, DRN was extensively utilized in several pattern recognition tasks. In contrast to other DL techniques such as ordinary CNN, the benefit of the DRN consists of a faster training process, easier gradient transmission and a deeper neural network with less gradient vanishing or explosion, among others. Deeper DRN-based networks may be used to produce a better score for both regression and classification tasks. Hence, DRN is explored in this paper as a multi-classifier for detecting and diagnosing healthy and damaged structural conditions.

Outline of the proposed method

In this work, the goal is to develop an advanced method for structural damage identification based on the acceleration response signals induced by ambient vibration and the DRN algorithm. Figure 6 presents an outline of the proposed method, which includes five steps, as explained in the following sub-sections.

Step 1: Collecting the numerical or experimental acceleration response signals of a civil structure under ambient excitation using a certain data acquisition system;

Step 2: Dataset preparations, which include the process of dividing the signals into subdivisions (segments) and store it in a suitable form, and then feed it as DRN inputs;

Step 3: A DRN model designing, training, testing, and optimizing its main parameters and properties, to achieve the highest possible accuracy;

Step 4: Structural condition evaluation based on the trained DRN model and the structural response signals;

Step 5: If structural damage occurs in the previous step, an estimation for damage size and location needs to be performed using the developed method in this work.

Data preparations

Usually, the traditional vibration-based SHM methods require manual data preprocessing. The typical method is to visualize a subdivision of dataset after representing it in time and frequency domains, respectively. Admittedly, this process is time-consuming and costly. To avoid this procedure, we directly used the raw time-domain SHM data as input, rather than inputting a pre-processed time series data to the designed model.

DRN is a supervised DL model that combined feature extraction and learning in one body. To get full advantage of the DRN and its superiority of learning features, in this work, we efficiently used 2D DRN model, as explained later in detail. Figure 7 illustrates the procedure of dataset preparations described as follows:

1. Let the length of the acquired signals from a structure at a structural condition N be n samples;

2. Divide each signal into S segments, the length of each segment is L;

3. All of the divided features are formed in a 4-D array as images (matrixes) with dimensions of L (length) × W (width) × 1 (channel) to form the DRN input dataset. Additional zeros are added around each of the newly created representations of signals. Note, the structural condition is the corresponding class label;

4. The dataset portioned randomly for two parts; one for training and the other for testing with a percentage of 0.8;

5. The inputs of the DRN model are the created dataset, which has to be fed through the network input layer.

DRN architecture and configurations

Figure 7 demonstrates the computational structure of the proposed DRN. The input is fed to the network and normalized with “zero-center” normalization by the image input layer, which is the first layer. Each input sample is a 1 × 100 vector while the output is the vector from 1 to the number of structural cases. The architecture of the proposed network consists of the number of convolutional and residual blocks, and each block is used for the extraction of features from input data without a manual procedure. With a total of three residual blocks, one is inserted between the first and third convolutional blocks, and two residual blocks are inserted between the third convolutional block and AvgPool layer for additional feature compression following object extraction modules. For the dimension to match before the linear classifier, an additional AvgPool is added. Lastly, a linear classification (Linear) consisting of an FC layer and a Softmax output layer is used for the classification. In addition, the configuration details of each residual and convolution block are listed in Table 2.

Table 2.

Detailed configurations of the layers for each residual and convolution block.

Layer name	Function	Parameters
Conv2d	2D-Convolution	77 7 × 1 × 1 convolutions with stride [1 1] and padding [0 0 0 0]
BatchNorm	Batch normalization	Batch normalization with 77 channels
ReLU	Rectified linear unit	ReLU activation
Conv2d	2D-Convolution	77 7 × 1 × 1 convolutions with stride [1 1] and padding [0 0 0 0]
BachNorm	Batch normalization	Batch normalization with 77 channels
ReLU	Rectified linear unit	ReLU activation

The proposed damage identification method

As mentioned earlier, this work aims to develop a method for evaluating the structural condition. In principle, the occurrence of damage in any structure system affects the structural behavior of that system. As result, at the damaged location, more energy absorption locally is expected. Likewise, to identify damage growth, more compressive and tensile displacements (or relative positions) are expected. In this study, a new damage identification method is introduced, which needs to construct and train DRN model. The responsibility of this DRN is to assess the structural status using the captured response signals by accelerometer transducers placed on the structure. In this way, using this new damage identification method, any changes in the structural condition are discoverable. It is also summarized as follows:

1. Training DRN for classifying N structural conditions of the benchmark study. In other words, enabling the trained DRN to extract the optimal features from the raw ambient vibration acceleration signals. Note that the dataset preparations are explained in the Data preparations section.

2. The extracted outputs of classification from just trained DRN are a specific class and the vectors of belonging probability to the N structural status in the benchmark study.

3. Once a damage case has been detected, an estimation for the size and location of the damage will be performed, as per the following sub-sections.

Damage size quantification method

1. The proposed damage severity estimation method is based on the DRN’s classification results for each segment of the response signals. These results are the classes of segments (Healthy, Damage1, …, Damage N), and the vectors of belonging probability to these classes.

2. The labels have been noted as 1, 2, N for a case that has N classes. The softmax method is utilized for calculating the affiliation probability $P o_{x_{i}}$ of each sample to a specific class of the classes of the dataset

P o_{(x_{i})} = {P o_{1}, P o_{2}, P o_{N}}

(19)

3. A probability threshold value is set to be 97%, in other words, only the sample classified with higher than 97% probability of belonging to jth class of the N classes has to be considered.

4. Mean of probabilities that equal to or higher than the set threshold value is calculated as

M e a n P o_{j} = \frac{\sum (P o_{j} \geq 97 %)}{N}

(20)

where

M e a n P o_{j}

is the affiliation probability of the classified samples to the jth damage class.

5. Consequently, to calculate the size of the damage (SoD) as

S o D = (D_{j} \times M e a n P o_{j}) \pm 0.01

(21)

where

D_{j}

is the specified size of damage for the jth damage class, and ±0.01 is a correction factor which is equal to the loss value.

Damage localization method

1. The damage localization method is also utilizing DRN to classify the signals acquired from several sensors mounted in different locations at the structure. Generally, the acceleration response plots cannot provide a quantitative approach for damage identification, the quantitative assessment of damage is traditionally made by the use of a scalar damage metric such as root mean square deviation. Root mean square deviation compares the signal of a structure in an unknown or probably damaged condition with respect to a baseline signal which represents the undamaged condition. In this work, an effective statistical DI was presented for damage localization, which is based on the classification results of a fully trained DRN. DI is defined as

D I = \frac{((b) - (a))}{((a) + (b))}

(22)

where a is the number of samples classified as a healthy class, while b is the number of samples classified to belong to any of the damage classes. Note that the probability threshold value is set to be 0.999 for damage localization. That means only the samples classified to belong for specific structural condition (class) with probability of 99.9% or higher are considered.

2. The DI positive sign (+) indicates damage occurrence, and a negative sign (−) indicates a healthy condition.

Numerical simulations

Description of the finite element model

To evaluate the proposed methodology, the first phase in a benchmark SHM problem has been used which is organized under the auspices of the IASC-ASCE SHM Task Group. The scale-model structure adopted for use in this benchmark problem is a four-story 2 × 2 bay steel frame constructed at the Earthquake Engineering Research Laboratory of the University of British Columbia. The frame dimensions are (2.5 m × 2.5 m × 3.6 m) footprint and height, respectively. Lateral bracing was obtained by adding diagonal steel rods. The members are hot-rolled grade 300W steel with a yield stress 300 MPa (42.6 kpsi). The sections are designed for a scale model. The columns are all oriented to be stronger bending toward the x-direction (i.e., about the y-axis). The floor beams are oriented to be stronger bending vertically, that is, about the y-axis (x-axis) for those oriented with longitudinal axis parallel to the x-axis (y-axis). The braces have no bending stiffness, so their orientation is irrelevant. There is one floor slab per bay per floor: four 800 kg slabs at the first level, four 600 kg slabs at each of the second and third levels, and on the fourth floor, either four 400 kg slabs or three 400 kg and one 550 kg to create some asymmetry. More details are available in Ref.[62].

This study has used the analytical structural model of a 120 DOF model, the healthy and damaged cases to be identified are listed in Table 3. In the modeling, element type is Euler beam, number of elements is 116, elasticity modulus is 200 GPa, and the joints are assumed to rigid connects. MATLAB computer codes have been used to generate the response data for the various cases from three accelerometer sensors on each floor.

Table 3.

Descriptions of the analytical scenarios in frame SHM problem phase I.⁶²

Scenario no	Description
1	Healthy
2	Area of 1 brace of the southeast corner in the first floor is reduced to 2/3
3	Southeast corner bay on the first floor is unbraced (no stiffness)
4	Southeast corner bay on the first and third floors are unbraced
5	The first floor is unbraced
6	The first and third floors are unbraced
7	Scenario 6 + bolts at the ends of a beam on the second floor (northwest) are loosened
8	Scenario 7 + bolts at the ends of all of the beams on the first floor are loosened

The nominal integration time step was set to be 0.001 s, approximately 100 s. The acceleration time history responses are available at a 1 kHz sampling rate. A total of eight cases are defined as a part of phase 1 benchmark problem, except for healthy condition, each case includes many damage patterns as shown in Figure 8. The excitations are applied as approximating wind or other ambient excitation, one per floor, and are modeled as independent Gaussian white noise. Besides, subsequent structural cases of the benchmark problem have incorporated into a full-scale framework.

Damage identification results

The achieved results using the developed methodology are introduced in this section. To demonstrate the robustness of the proposed approach, firstly, the used structural responses are the acceleration signals that acquired from the finite element model of benchmark structure ambient vibration. Different eight structural conditions have been considered. Figure 9 shows samples of the response signals collected from the central sensor on the second floor. The presented signals are for the eight simulated structural statuses as in Table 3. Conducting a comparison among these signals, it can easily be observed that the changes in the acceleration response signals are difficult to inspect. For various structural conditions, the incremental accumulative structural damages have produced slight changes in the signals. Therefore, an extra accurate method is required to extract the optimal features of these signals for an effective SHM.

Secondly, following the proposed method for signal preparations discussed in the Data preparations subsection, the signal length for each of the structural conditions consists of 40,000 samples which is used to generate a small training dataset to reduce the computational cost and approve the model superiority. Accordingly, each measured signal is divided into 400 segments with the same length of 100 samples. Then all of the divided signals are stored in the 4-D array as images (matrixes) with dimensions of (100 height × 1 width) and single-channel to form the primary dataset, considering the structural conditions as corresponding classes label. This dataset was portioned randomly for two parts; one for training and the two for testing the DRN model. Table 4 shows DRN’s inputs.

Table 4.

Sample sizes created from each accelerometer response signal at the eight structural conditions of the frame finite element model.

Scenario no	Structural condition	No. of training samples	No. of test samples
1	Healthy (0 damage)	311	89
2	Damage 1	321	79
3	Damage 2	318	82
4	Damage 3	327	73
5	Damage 4	328	72
6	Damage 5	328	72
7	Damage 6	316	84
8	Damage 7	311	89

Next, separate training and testing processes are conducted for 12 DRN models (one DRN model for each accelerometer). Since the variations among the signals are minimal, as shown in Figure 9, the algorithm needs an efficient training method to overcome this issue and bring up the data as well as to get the benefit of those small signal variations and achieve accuracy and reliability in the damage detection method. The proposed method uses different signal subdivision lengths, activation functions, and hyper-parameters during the training and testing processes of each DRN model. The highest achieved accuracies were 100% for both testing and training and all of the 12 DRNs, respectively.

Damage size evaluation results

Regarding evaluation for the newly proposed damage quantification method, Table 5 shows the identified results in damage size. It is observed that the identified damage severity is 0 in Scenario 1 (healthy), 3.23% in Scenario 1, and 73.58% in Scenario 8. With the increase of damage conditions, the identified damage severity is gradually increased. The identification results of all of the 12 DRNs have been certified to well match the simulated evolutional damage conditions with high reliability. Therefore, this approach is appropriate for predicting different severities of damage.

Table 5.

The results of damage detection and size evaluation for the frame finite element model.

Scenario no	Structural condition	Identified damage size (%)
1	Healthy (0 damage)	0
2	Damage 1	3.23
3	Damage 2	6.34
4	Damage 3	13
5	Damage 4	33
6	Damage 5	52.92
7	Damage 6	64.77
8	Damage 7	73.58

Damage localization results

It is crucial that a SHM system be accurate in damage localization. Therefore, in this work, the difficulty factors in the SHM benchmark studies are utilized efficiently to develop a powerful damage localization method. The simulated damage implemented in the benchmark problem phase I are multiple and in various locations, what is more, the sensors are also installed in different locations as described in Table 3. The achieved results listed in Table 6 have proven that different damage locations have been recognized accurately at each floor using any of the trained 12 DRNs. It is also observed that the damage has been detected by the sensor near to damage in addition to the central sensors. Therefore, damage localization can be done accurately by following the proposed method. However, in an effort to explain the results, if we considered scenario 1, results (−1 Negative) for all of the sensors indicate that there is no damage has been detected in any of the monitored areas by 12 sensors. Likewise, for scenario 2, results (+1 Positive) for sensors 1and 2, which indicate that there is damage detected only in the southeast of the first floor only. Similarly, in scenario 8, all the results are (+1 Positive) which means the signals of all of 12 sensors have altered for damage occurrence in the monitored areas by these sensors.

Table 6.

The results obtained by the proposed damage localization method for the frame finite element model.

Sensor no	Sensor installation position	Monitored movement direction	Scenario 1	Scenario 2	Scenario 3	Scenario 4	Scenario 5	Scenario 6	Scenario 7	Scenario 8
1	1st floor, center	E/W	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)
2	1st floor, east	N/S	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)
3	1st floor, west	N/S	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)
4	2nd floor, center	E/W	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)
5	2nd floor, east	N/S	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)
6	2nd floor, west	N/S	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)
7	3rd floor, center	E/W	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)
8	3rd floor, east	N/S	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)
9	3rd floor, west	N/S	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)
10	4th floor, center	E/W	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)
11	4th floor, east	N/S	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)
12	4th floor, west	N/S	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)

Experimental verification

Frame structure

In addition to the dataset of the numerical study, the present study has been verified on the experimental data of the SHM benchmark problem phase II.⁶³ The IASC—ASCE SHM Task Group has published Experimental Phase I and Phase II as benchmark problems.^63,74 The objective of the provided benchmark problems was to offer an integrated testbed for testing new structural damage identification approaches. Different from the Experimental Phase I benchmark problem,⁷⁴ the described structural damage cases of Phase II problem were more practical and complicated.

The benchmark structure is a four-story 2 × 2 bay steel frame constructed at the Earthquake Engineering Research Laboratory of the University of British Columbia. The frame dimensions are (2.5 m × 2.5 m × 3.6 m) footprint and height, respectively. Lateral bracing was obtained by installing diagonal steel rods. The structure has been instrumented with 12 accelerometers. Starting from ground floor, each level was instrumented with three accelerometers: one near the central column, one at the west side, and the third one at the east side. Additional technical details are available in.⁶³

The accelerometers installed at the west and east sides were assigned to record the signals along the NS direction, whereas the central sensors were utilized to capture the signals along the EW direction. Nine structural scenarios have experimentally simulated on the mentioned structure. For each of the simulated scenarios, the acceleration signals were measured by several accelerometers, and the structure was excited using three different excitation resources namely, ambient vibration, impact hummer, and a shaker. A 16-channel DasyLab acquisition systems was used to record the structural responses through the 12 Force Balance Accelerometers (FBA-3). The sensitivity of the sensor is 5 V/g. Anti-aliasing filters of 50 Hz were used in the ambient tests and the data was sampled at 250 Hz.

The structural test scenarios are shown in Table 7 and Figure 10. Damage was initiated in Scenario 2 and increased gradually to severe damage in Scenario 9, while Scenario 1 is undamaged. Two methods were followed to introduce the damage. Damage is introduced through removing the diagonal braces at certain positions in Scenarios 2–7 and through losing bolt at specific joints in Scenarios 8 and 9. It is important to note that to increase the approach reasonability, this work is based on the signals measured from the structure under ambient excitation.

Table 7.

Descriptions of the experimental scenarios in frame SHM problem phase II.⁶³

Scenario no	Description
1	Healthy
2	Southeast corner in one bay on the first floor is unbraced
3	Southeast corner braces in one bay on the first and fourth floors are unbraced
4	Southeast corner in one bay in all floors are unbraced
5	Eastside is unbraced
6	The east face is unbraced, and the north face on the second floor is unbraced
7	All sides are unbraced
8	Scenario 7 + bolts at the ends of a beam on the east face, north sides in first and second floors are loosened
9	Scenario 7 + bolts at the ends of the beam on the east face, north sides in all floors are loosened

Damage identification results

To further show the superiority, this section introduces the achieved results using the developed methodology and the experimental dataset. Firstly, we used the structural responses (acceleration signals acquired under ambient vibration) of the benchmark structure. Nine different structural conditions have been considered. Figure 11 shows samples of the response signals collected from the central sensor on the second floor. Again, the acceleration response signals changed due to the damage occurrence and extension. For various structural conditions, the accumulative structural damages resulted into slight changes in the acceleration signals. That proved for the second time that an extra accurate method is required to extract the optimal features of these signals for accurate SHM.

Secondly, the proposed method was followed for signals preparation as discussed in the Feed-forward operation subsection. The selected signal length for each of the nine structural condition measurements also consists of 40,000 samples, which is the shortest length of signal available in the benchmark dataset. Each measured signal is divided into 400 segments with the same length of 100 samples. Then, all of the divided signals are stored in the 4-D array as images (matrixes) with dimensions of 100 × 1 for each single-channel to form the primary dataset; the structural condition is considered as a corresponding class label. This dataset was portioned randomly in two parts, one consists of 2700 segments (frame) for training and the second 900 segment for testing the DRNs, as shown in Table 8.

Table 8.

Sample sizes created from each accelerometer response signal at the nine experimental frame structural conditions.

Scenario no	Structural condition	No. of training samples	No. of test samples
1	Healthy (0 damage)	323	77
2	Damage 1	314	86
3	Damage 2	319	81
4	Damage 3	318	82
5	Damage 4	328	72
6	Damage 5	324	76
7	Damage 6	325	75
8	Damage 7	322	78
9	Damage 8	307	93

The next step is conducting separate training and testing processes for 12 DRN models (also one DRN for each accelerometer). The achieved testing and training accuracies are similar, and 100% by all of the 12 DRNs. This new methodology has significant efficiency. It is verified again after using the experimental dataset that can overcome feature extraction difficulties and bring up the data.

Damage size evaluation results

To test the newly proposed framework for quantifying damage, Table 9 indicates the significance of the developed method and all 12 DRNs were accepted to balance the actual dimensions of simulated damage with the calculated damage sizes with high reliability. Therefore, to predict the size of damage, this approach is applicable.

Table 9.

The results of damage detection and size evaluation using experimental frame dataset.

Scenario no	Structural condition	Identified damage size (%)
1	Healthy (0 damage)	0
2	Damage 1	9.33
3	Damage 2	16.22
4	Damage 3	22
5	Damage 4	40
6	Damage 5	47.22
7	Damage 6	53.33
8	Damage 7	66.88
9	Damage 8	82.66

Damage localization results

As per the first case study, the simulated damage made in the phase II benchmark analysis is numerous and the sensors are also mounted in different positions. However, as shown in Table 10, different locations of damage were accurately identified on each floor using any of 12 DRNs by following the proposed process. The damage is also detected by the near-to-damage sensor in addition to the central sensors. The central sensors indicate that all damage cases on a particular floor can be sensed, verifying the superiority of the method in localizing damage.

Table 10.

The results obtained by the proposed damage localization method using the experimental frame dataset.

Sensor no	Sensor installation position	Monitored movement direction	Scenario 1	Scenario 2	Scenario 3	Scenario 4	Scenario 5	Scenario 6	Scenario 7	Scenario 8	Scenario 9
1	1st floor, center	E/W	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)
2	1st floor, east	N/S	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)
3	1st floor, west	N/S	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)
4	2nd floor, center	E/W	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)
5	2nd floor, east	N/S	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)
6	2nd floor, west	N/S	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)
7	3rd floor, center	E/W	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)
8	3rd floor, east	N/S	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)
9	3rd floor, west	N/S	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)
10	4th floor, center	E/W	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)
11	4th floor, east	N/S	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)	+1 (positive)
12	4th floor, west	N/S	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	+1 (positive)	+1 (positive)

Steel truss bridge

The dataset was acquired from a progressive damage test conducted on a Japanese bridge that was subjected to ambient vibration and moving vehicle excitations.⁶⁴ The dataset was acquired from a progressive damage test conducted on a Japanese bridge that was subjected to ambient vibration and moving vehicle excitations 64. The bridge was a simply supported steel truss bridge that spanned 59.2 m with a width of 3.6 m and a maximum height of 8m. The structural members of the truss were built from beam elements and were assigned steel material properties (density of 7900 kg/m³ and Young’s modulus of 200 GPa). Similarly, the deck was made of concrete material with a density of 2400 kg/m³ and an elasticity modulus of 21 GPa. It was scheduled to be replaced in 2012. Before that, a progressive damage test was carried out while the bridge was closed to the public. The dynamic response was recorded from eight uniaxial accelerometers, positioned as per Figure 12(b) and measuring the vertical acceleration, with sample rates of 200 Hz. It is important to mention that in this work we used vibration data when the bridge was tested under ambient excitation. Five different scenarios were designed to represent different real-world cases by inducing artificial damage to the truss. The introduced damage scenarios involve severing vertical members of the truss structure at the locations presented in Figure 12(b). The progression of the damage states is described in Table 11 and shown in Figure 13. The damage actions began with the partial intersection of the mid-span vertical member (Damage1), before its full intersection was completed (Damage2). After that, the damaged member was reconnected (RCV) with a welded flange. Finally, the vertical member at 5/8th span was completed severed (Damage3).

Table 11.

Structural conditions conducted on the steel truss bridge.

Scenario no	Description
1	Healthy
2	Half cut in vertical member at mid-span (Damage1)
3	Full cut in vertical member at mid-span (Damage2)
4	Mid-span member reconnected (recovered)
5	Full cut in vertical member at 5/8th span (Damage3)

Damage identification results

Structural health monitoring results for the steel truss bridge using the proposed approach have been introduced in this section. This time, a dataset of a full-scale steel truss bridge is used. The efficiency of the approach is tested using a more challenging real-world problem.

Firstly, we use the structural responses (acceleration signals) that were acquired under ambient vibration excitation using 8 accelerometers. The accelerometers are installed in different locations on the real-world truss bridge, as shown in Figure 12(b). Five different structural scenarios listed in Table 11 have been investigated. In scenario 2, half cut in vertical member at mid-span was implemented, which is called as Damage1. In scenario 3, full cut in vertical member at mid-span was implemented, which is called as Damage2. In scenario 5, full cut in vertical member at 5/8th span was implemented, which is called as Damage3. In different scenarios, samples of the response signals collected from the central sensor (A3) are shown in Figure 14. The acceleration response signals changed due to the occurrence and extension of damage. Again, an extra accurate method is required to utilize these signals for correct SHM.

Secondly, the method discussed earlier for signal preparation is also followed. The selected signal length for each structural condition also consists of 40,000 samples, which is the same as the shortest length of signal available in the benchmark dataset. After dividing, labeling, and storing the signals as per the requirements of the proposed method, the dataset was portioned randomly into two parts. One consists of 1500 segments for training and the second 500 segments for testing the DRNs, as shown in Table 12.

Table 12.

Sample sizes created from each accelerometer response signal in the five structural scenarios of the truss bridge.

Scenario no	Structural condition	No. of training samples	No. of test samples
1	Healthy (0 damage)	298	102
2	Damage 1	300	100
3	Damage 2	302	98
4	RCV (recovered damage)	294	106
5	Damage 3	306	94

As per this approach, a single DRN should be trained and tested for each accelerometer. Thus, eight DRN models are needed. The testing and training accuracies by all of the eight DRNs are similar and achieve 100%. This time, the proposed new methodology has been verified again as a powerful SHM tool even when using a real-world dataset of a full-scale bridge.

Damage size evaluation results

To further verify the damage size evaluation ability of the proposed method, the full-scale truss bridge with progressive damage was considered here. The results for damage size evaluation are listed in Table 13. For health and RCV conditions, the identified results are 0 and 1.11%, respectively, and for Damage1, Damage2, and Damage3 conditions, the results are 9.57%, 17.35%, and 19.33%, respectively. All eight DRNs were acceptable to balance the actual dimensions of simulated damage with predicted damage sizes with high reliability.

Table 13.

The results of damage detection and damage size evaluation for truss bridge.

Scenario no	Structural condition	Identified damage size (%)
1	Healthy (0 damage)	0
2	Damage 1	9.57
3	Damage 2	17.35
4	RCV (recovered damage)	1.11
5	Damage 3	19.33

Damage localization results

Table 14 shows different locations of damage accurately identified on the span of the bridge in different damage scenarios by the proposed method. The results indicate that all damage scenarios on the vertical members along the span of the bridge can be sensed by the sensor which is near to the damaged member, confirming the advantage of the proposed method in localizing damage to different locations in such a large real-world bridge structure.

Table 14.

The results obtained by the proposed damage localization method for the truss bridge.

Scenario no	Sensor A1	Sensor A2	Sensor A3	Sensor A4	Sensor A5	Sensor A6	Sensor A7	Sensor A8
1	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)
2	−1 (negative)	−1 (negative)	+1 (positive)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)
3	−1 (negative)	−1 (negative)	+1 (positive)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)
4	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)
5	−1 (negative)	−1 (negative)	−1 (negative)	+1 (positive)	−1 (negative)	−1 (negative)	−1 (negative)	−1 (negative)

Discussions

By experimental and numerical studies, the results show that the proposed approach can offer an accurate SHM system, which is capable of predicting the overall structural condition, size, and location of damage once it happens. For the three case studies, Figure 15 clearly indicates the convergence of accuracy and loss curves for the DRN model during the training process while using its main hyper-parameters obtained through Bayesian optimization.

To visualize DRN performance in terms of core relations among different classes, Figure 16 presented the confusion matrix. Since the recall (the right entire column) is evaluating the ground truth, it is easy to observe that all of the structural conditions were accurately classified, with precision of 100% for both training and testing datasets, no matter experimental, numerical, and real-world cases.

Next, in Figure 17, the receiver operating characteristic (ROC) curve was obtained from plotting the tpr (true positive rate, i.e., the recall) against the fpr (false positive rate). It usually considers a single class as a positive class and the remaining are negative. The under curve area evaluates the results of all binary classifiers. The presented ROC curves show that the area under the curve for all curves is 1. This confirms the accurate classification performance when classifying all of the structural conditions of the three structures. Furthermore, the result was achieved using only a reasonable training dataset for the DRN without any need for a GPU for the running. This can be considered a superior and fixed solution for practical applications since it depends on providing a training dataset from the available measured signals, which represents the response of the structure. In addition, for the frame and bridge structures, the different damage sizes and locations have been detected accurately using the DRNs assigned to a specific accelerometer, which again confirms the efficiency of the proposed damage identification method.

The superiority of the proposed method arises from these main aspects: (1) the accurate data preparation method gets the most out of DRN efficiency, (2) the optimization method of the DRN hyper-parameters which works to utilizes its full potential, (3) the same DRN can automatically learn the in-depth features from the raw input data without any need for signal preprocessing, and (4) careful selection and utilizations of the DRN outputs.

Visualization of learned features

The efficacy of the proposed damage identification approach is qualitatively demonstrated in this section based on the visualization of learned representation. A powerful technique “t-SNE” is used to simulate data representation in high dimensions by translating samples from the initial data space onto a 2-dimensional map.⁷⁵ Principal component analysis (PCA) is initially used to minimize the dimensionality of the representation to 70, “t-SNE” is then used to convert the 70-dimensional learned feature in a 2D sample. Figure 18 displays the resulting raw data representations and the learned maps of the proposed approach. It can be shown that the features collected through the proposed method cluster are the strongest where all the samples are well separated under various health conditions. An effective classification can be accomplished by the proposed method. The results of the feature presentation correspond with the assumptions from previous sections that the DL architecture of DRN will effectively solve the problem of damage detection. Furthermore, remarkable data samples for class 4 and 5 (numerical) in the cluster regions of other classes are observed, indicating that it is difficult to classify these two structural health conditions. However, it is important to note that the final classification is performed non-linearly in a high-dimensional space. Therefore, acceptable overlapping points for various health conditions are consistent with the high classification accuracy shown in Figure 18 in the visualization.

Computational complexity

Throughout the entire implementation process of this work, the datasets of the SHM benchmark problems I and II have been fed as an input for the designed DRNs after conducting the earlier-mentioned data preparation processes. The training and testing of the DRNs were carried out on a desktop having an Intel C i5-8400, 16 GB of RAM and run on Windows 10 without the necessity of a GPU. The systematic program of the proposed approach has been written and run using the MATLAB R2019b software.⁷⁶ Then, the main hyper-parameters of DRN turn to Bayesian optimization to mitigate the expensive complexity of evaluating our black-box function. The training process of each of the DRNs is repeated 20 times automatically like the obtained optimal hyper-parameters of one of the DRNs, as listed in Table 15. In addition, the maximum number of training epochs was set to 30 epoch, and the rest of the parameters were set to be the default values. A single DRN has been trained for each sensor to identify the structural condition, resulting in a DRN model that can detect, quantify, and localize multiple damage.

Table 15.

Optimal hyper-parameters obtained using Bayesian optimization in the training process.

Hyper-parameter	Value
Minimum batch size	24
Initial learn rate	0.1424
Momentum	0.7701
L2 regularization	3.9950 × 10⁻⁶

Effect of samples dimension

Unlike other works, such as image recognition, the sizes of samples prepared from structure vibration data may be artificially calculated based on precise criteria. Basically, more data points in the samples provide more information about the state of health of the structure, which contributes to better accuracy. However, larger models and a heavier computational burden will be necessary. Therefore, the influence of the sample dimension (segment length) on the performance of the proposed method has also been studied. Figure 19 presents the testing accuracy and the computing time using different sizes of samples, starting from 60 to 140 sampling points, increasing 20 points each time. For all of the implemented cases, the total signal length is fixed to 40,000 sampling point. The number of samples used for training and testing is fixed to 1000 samples.

From Figure 19, it is evident that the testing accuracies are similar in terms of values when the dimensions are from 60 to 100 samples. The testing accuracy reaches 100% when the dimensions are 100 samples. It is also noticeable that testing accuracy decreased significantly for sample dimensions from 120 to 140. However, the computing time decreases as the sample size increase gradually from 60 to 140. In this work, when using the experimental dataset, the higher the dimension of the dataset sample obtained, the lower the testing accuracy and less consuming time for the DRN DL model. Therefore, the selection of sample dimension is an important factor.

Comparison study

To better demonstrate the robustness of the methodology, a comparison has been conducted in terms of performance with high-level pre-trained DL framework (GoogLeNet), and two popular DL methods, CNN and deep auto-encoder (DAE). GoogLeNet is a 22-layer deep convolutional neural network developed by Google researchers. Additional details about GoogLeNet can be found in.²² The specific configurations for CNN and CAE are listed in Tables 16 and 17. The same training and essential hyper-parameters setting of the three models have performed as those of DRN which described in the Training of the DRN-SHM model section. Then, the used activation function is sigmoid, and the maximum number of training epochs of GoogLeNet, CNN, and DAE are set to 30 and 200, respectively, and the rest of the parameters are default values.

Table 16.

Detailed configurations of the compared DAE structure.

DAE = 4 × 3 layer stacked AE
Layer name	Function	Parameters
1 'autoenc1'	Encoder and decoder using sigmoid as encoding and decoding function	Hidden size of 100 unit
2 'autoenc2'	Encoder and decoder using sigmoid as encoding and decoding function	Hidden size of 200 unit
3 'autoenc3'	Encoder and decoder using sigmoid as encoding and decoding function	Hidden size of 200 unit
4 'autoenc4'	Encoder and decoder using sigmoid as encoding and decoding function	Hidden size of 250 unit
5 'softmax'	Softmax activation for classification	Max no. of epoch of 100

Table 17.

Detailed configurations of the compared CNN structure.

CNN = 12 × 1 layer array with layers
Layer name	Function	Parameters
1 'imageinput'	Image Input	100 × 1 × 1 images with 'zerocenter' normalization
2 'conv_1'	Convolution	77 7 × 1 × 1 convolutions with stride [1 1] and padding [0 0 0 0]
3 'batchnorm_1'	Batch normalization	Batch normalization with 77 channels
4 'relu_1'	ReLU	ReLU activation
5 'maxpool_1'	Max pooling	1 × 1 max pooling with stride [10 10] and padding [0 0 0 0]
6 'conv_2'	Convolution	77 7 × 1 × 1 convolutions with stride [1 1] and padding [0 0 0 0]
7 'batchnorm_2'	Batch normalization	Batch normalization with 77 channels
8 'relu_2'	ReLU	ReLU
9 'maxpool_2'	Max pooling	1 × 1 max pooling with stride [10 10] and padding [0 0 0 0]
10 'conv_3'	Convolution	77 7 × 1 × 1 convolutions with stride [1 1] and padding [0 0 0 0]
11 'batchnorm_3'	Batch normalization	Batch normalization with 77 channels
12 'relu_3'	ReLU	ReLU
13 'maxpool_3'	Max pooling	1 × 1 max pooling with stride [10 10] and padding [0 0 0 0]
14 'fc'	Fully Connected	9 fully connected layer
15 'softmax'	Softmax	Softmax
16 'classoutput'	Classification output	Cross-entropy with 'DAMAG1' and 8 other classes

Also, the achieved average testing accuracy and the computing time using DRN are compared with mentioned models, as illustrated in Figure 20. The proposed method repeatedly shows better damage identification performance and stability for five trials, although the computing time of the DAE method is a bit less than the DRN method. Therefore, the proposed method is more accurate and stable than GoogLeNet, conventional CNN and DAE, and can be used for effective SHM.

In addition to the conducted comparisons, the performance of the proposed method has been compared without using Bayesian optimization, to explore the achieved improvement by Bayesian optimization in terms of classification accuracy. Table 18 shows the obtained accuracy during the training process of DRN using a dataset of experimental frame and while performing a manual grid search, and using Bayesian optimization over various parameters like the learning rate, momentum, etc. It is important to mention that the chosen values of these parameters during the grid search are selected to be near to the default values and which are usually used. The results clearly show that the highest accuracy was achieved using the hyper-parameters that were obtained by Bayesian optimization. In addition, another important advantage of using the latter is that it makes DRN’s training process totally automated, which provides an efficient solution for finding the optimal model and for saving time and effort.

Table 18.

Comparison results of performing manual grid search versus using Bayesian optimization for training DRN.

Hyper-parameter	Value
Hyper-parameter	Manual grid search			Bayesian optimization
Minimum batch size	22	24	26	24
Initial learn rate	10 × 10⁻²	10 × 10⁻⁴	10 × 10⁻⁶	0.1424
Momentum	0.6	0.7	0.8	0.7701
L2 regularization	1 × 10⁻⁶	2 × 10⁻⁶	3 × 10⁻⁶	3.9950 × 10⁻⁶
Accuracy (%)	93.9	95.7	98.5	100

Conclusions

In this study, a novel SHM method was proposed using the DRN algorithm. Simulation and experimental datasets on a benchmark SHM were conducted to prove the effectiveness of the proposed method. These datasets consist of raw acceleration signals acquired from frames, and bridge structures subjected to progressive damage under ambient vibration. The DRN is trained to extract and learn features from the raw acceleration signals automatically. Multiple damage quantification and localization are also carried out using the DRN outputs.

The developed method can take full advantage of the designed DRN model. DRN is designed in an improved way to promote filters for moving in a 2D direction over 1D data to learn the in-depth features of the raw 1D acceleration signals. Furthermore, the DRN-SHM method is presented for the first time to work on the raw ambient vibration acceleration signals, without the need for any preprocessing. The algorithm is trained in CPU without the need for a GPU. Even with limited training data, the proposed method can perform well. The proposed DRN outperformed other models such as GoogLeNet, traditional CNN and DAE in terms of maximum accuracy, The Bayesian optimization algorithm has improved neural network efficiency in terms of maximizing the classification accuracy, solving the complexity of high-number of parameters, and automating the training process.

The results show that there is a significant improvement achieved in the sense of the overall achievement rate using the developed DRN-SHM method. In addition to the overall actual structural condition evaluation, the location and severity of damage were identified accurately.

Based on the results, a DRN-SHM algorithm with a hit rate of 100% has been developed. The introduced approach presents a robust and reliable SHM methodology. The authors are eager to continue research in this area in the future.

Footnotes

Acknowledgements

The authors acknowledge the supports of the National Natural Science Fund of China (51278215) and Basic Research Program of China (contract number: 2016YFC0802002).

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Osama Alazzawi

Dansheng Wang

References

Stolz

Neumair

. Structural health monitoring, in-service experience, benefit and way ahead. Struct Heal Monit An Int J 2010; 9: 209–217.

Zhao

Lei

, et al. Vibration-based fault diagnosis of slurry pump impellers using neighbourhood rough set models. Proc Inst Mech Eng Part C J Mech Eng Sci 2010; 224: 995–1006.

Hakim

SJS

Razak

. Modal parameters based structural damage detection using artificial neural networks - a review. Smart Struct Syst 2014; 14: 159–189.

Hoseini

Wang

Zuo

. Estimating ultrasonic time of flight using envelope and quasi maximum likelihood method for damage detection and assessment. Measurement 2012; 45: 2072–2080.

Balageas

Fritzen

Guemes

. Structural Health Monitoring. New Jersey, USA: Wiley Online Library, 2006.

Liu

, et al. Vibration-based condition monitoring. Mech Mach Sci 2013; 11: 431–477.

Van der Auweraer

. International research projects on structural damage detection. Key Eng Mater 2009; 204–205: 97–112.

Shi

Law

Zhang

. Structural damage detection from modal strain energy change. J Eng Mech 2000; 126: 1216–1223.

Lim

. Structural damage detection using modal test data. AIAA J 2008; 29: 2271–2274.

10.

Kawiecki

. Modal damping measurement for damage. Struct Heal Monit 2001; 10: 466–471.

11.

Shi

Law

Zhang

. Improved damage quantification from elemental modal strain energy change. J Eng Mech 2002; 128: 521–529.

12.

Abdo

MA-B

Hori

. A numerical study of structural damage detection using changes in the rotation of mode shapes. J Sound Vib 2002; 251: 227–239.

13.

Yan

Cheng

, et al. Development in vibration-based structural damage detection technique. Mech Syst Signal Process 2007; 21: 2198–2211.

14.

Paul

Jafari

. Recent sdvances in intelligent-based structural health monitoring of civil structures. Adv Sci Technol Eng Syst J 2018; 3: 339–353.

15.

Fan

Qiao

. Vibration-based damage identification methods: a review and comparative study. Struct Heal Monit An Int J 2011; 10: 83–111.

16.

Delgadillo

Casas

. Non-modal vibration-based methods for bridge damage identification. Struct Infrastruct Eng 2020; 16: 676–697.

17.

Yan

W-J

Chronopoulos

Papadimitriou

, et al. Bayesian inference for damage identification based on analytical probabilistic model of scattering coefficient estimators and ultrafast wave scattering simulation scheme. J Sound Vib 2020; 468: 115083.

18.

Lam

H-F

Yuen

K-V

Beck

. Structural health monitoring via measured Ritz vectors utilizing artificial neural networks. Comput Civ Infrastruct Eng 2006; 21: 232–241.

19.

Min

Park

Yun

C-B

. Impedance-based structural health monitoring using neural networks for autonomous frequency range selection. Smart Mater Struct 2010; 19: 125011.

20.

Neves

González

Leander

, et al. Structural health monitoring of bridges: a model-free ANN-based approach to damage detection. J Civ Struct Heal Monit 2017; 7: 689–702.

21.

Gomes

Mendez

YAD

da Silva Lopes Alexandrino

, et al. A review of vibration based inverse methods for damage detection and identification in mechanical structures using optimization algorithms and ANN. Arch Comput Methods Eng 2019; 26: 883–897.

22.

Dong

C-Z

Catbas

. A review of computer vision–based structural health monitoring at local and global levels. Struct Heal Monit 2020; 20: 692–743.

23.

Yan

Yam

Cheng

, et al. FEM modeling method of damage structures for structural damage detection. Compos Struct 2006; 72: 193–199.

24.

Yam

Yan

Jiang

. Vibration-based damage detection for composite structures using wavelet transform and neural network identification. Compos Struct 2003; 60: 403–412.

25.

Yan

Yam

. Detection of delamination damage in composite plates using energy spectrum of structural dynamic responses decomposed by wavelet analysis. Comput Struct 2004; 82: 347–358.

26.

Hinton

Salakhutdinov

. Reducing the dimensionality of data with neural networks. Science 2006; 313: 504–507. DOI: 10.1126/science.1127647.

27.

Jia

Lei

Lin

, et al. Deep neural networks: a promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data. Mech Syst Signal Process 2016; 72–73: 303–315.

28.

Jin

Yun

. A review on deep learning - based structural health monitoring of civil infrastructures. Smart Struct Syst 2019; 24: 567–586.

29.

Liu

Wang

Liu

, et al. A survey of deep neural network architectures and their applications. Neurocomputing 2017; 234: 11–26.

30.

Ince

Kiranyaz

Eren

, et al. Real-time motor fault detection by 1-D convolutional neural networks. IEEE Trans Ind Electron 2016; 63: 7067–7075.

31.

Aljemely

Xuan

, et al. Wise-local response convolutional neural network based on Naïve Bayes theorem for rotating machinery fault classification. Appl Intell 2021; 51: 6932–6950. DOI: 10.1007/s10489-021-02252-2.

32.

Kiranyaz

Ince

Gabbouj

. Personalized monitoring and advance warning system for cardiac arrhythmias. Sci Rep 2017; 7: 1–8.

33.

Kiranyaz

Ince

Gabbouj

. Real-time patient-specific ECG classification by 1-D convolutional neural networks. IEEE Trans Biomed Eng 2016; 63: 664–675.

34.

Balietti

Casoli

Di Stefano

, et al. Ketogenic diets: an historical antiepileptic therapy with promising potentialities for the aging brain. Ageing Res Rev 2010; 9: 273–279.

35.

Wang

Jiang

Luo

. Combination of convolutional and recurrent neural network for sentiment analysis of short texts. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers, Osaka, Japan, Dec 11–17, 2016, pp. 2428–2437.

36.

. Towards dropout training for convolutional neural networks. Neural Networks 2015; 71: 1–10.

37.

Kuo

C-CJ

. Understanding convolutional neural networks with a mathematical model. J Vis Commun Image Represent 2016; 41: 406–413.

38.

Saxena

. Convolutional neural networks: an illustration in TensorFlow. XRDS Crossroads, ACM Mag Students - Cult Comput 2016; 22: 56–58.

39.

Antonellis

Gavras

Panagiotou

, et al. Shake table test of large-scale bridge columns supported on rocking shallow foundations. J Geotech Geoenvironmental Eng 2015; 141: 04015009.

40.

Karpathy

Toderici

Shetty

, et al. Large-scale video classification with convolutional neural networks. In: 2014 IEEE conference on computer vision and pattern recognition, Columbus, OH, June 23–28. IEEE, pp. 1725–1732.

41.

Deshpande

. Understanding convolutional neural networks, 2014, https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks-Part-2/

42.

Tang

Chen

Bao

, et al. Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring. Struct Control Heal Monit 2019; 26: e2296.

43.

Zhang

Biswal

Wang

. SHMnet: condition assessment of bolted connection with beyond human-level performance. Epub ahead of print 2019. DOI: 10.13140/RG.2.2.17510.57929.

44.

Abdeljaber

Avci

Kiranyaz

, et al. Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks. J Sound Vib 2017; 388: 154–170.

45.

Xia

, et al. Fault diagnosis for rotating machinery using multiple sensors and convolutional neural networks. IEEE/ASME Trans Mechatronics 2018; 23: 101–110.

46.

Abdeljaber

Avci

Kiranyaz

, et al. 1-D CNNs for structural damage detection: verification on a structural health monitoring benchmark data. Neurocomputing 2018; 275: 1308–1317.

47.

Chen

F-C

Jahanshahi

. NB-CNN: deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Trans Ind Electron 2018; 65: 4392–4400.

48.

Shao

Jiang

Lin

, et al. A novel method for intelligent fault diagnosis of rolling bearings using ensemble deep auto-encoders. Mech Syst Signal Process 2018; 102: 278–297.

49.

Pathirage

CSN

, et al. Structural damage identification based on autoencoder neural networks and deep learning. Eng Struct 2018; 172: 13–28.

50.

Pathirage

CSN

, et al. Development and application of a deep learning–based sparse autoencoder framework for structural damage identification. Struct Heal Monit 2019; 18: 103–122.

51.

de Oliveira

Monteiro

Filho

. A new structural health monitoring strategy based on PZT sensors and convolutional neural network. Sensors 2018; 18: 2955. DOI: 10.3390/s18092955.

52.

Alazzawi

Wang

. Deep convolution neural network for damage identifications based on time-domain PZT impedance technique. J Mech Sci Technol 2021; 35: 1809–1819.

53.

Zhang

Ren

, et al. Deep residual learning for image recognition. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 2016; 2016-Decem: 770–778.

54.

Chu

Han

. Deep residual learning with demodulated time-frequency features for fault diagnosis of planetary gearbox under nonstationary running conditions. Mech Syst Signal Process 2019; 127: 190–201.

55.

Zhang

Ding

. Deep residual learning-based fault diagnosis method for rotating machinery. ISA Trans 2018; 95: 295–305.

56.

Zhao

Kang

Tang

, et al. Deep residual networks with dynamically weighted wavelet coefficients for fault diagnosis of planetary gearboxes. IEEE Trans Ind Electron 2018; 65: 4290–4300.

57.

Song

Xin

, et al. Real-time tunnel crack analysis system via deep learning. IEEE Access 2019; 7: 64186–64197.

58.

Yuan

Zhang

, et al. Unified vision-based methodology for simultaneous concrete defect detection and geolocalization. Comput Civ Infrastruct Eng 2018; 33: 527–544.

59.

Gao

Mosalam

, et al. Deep residual network with transfer learning for image-based structural damage recognition. In: 11th national conference on earthquake engineering, 2018, NCEE 2018, Los Angeles, USA, June 25–29. Integrating Science, Engineering, and Policy, 2018, 11, pp. 6971–6981.

60.

Guo

Wang

, et al. Damage detection in a novel deep-learning framework: a robust method for feature extraction. Struct Heal Monit 2020; 19: 424–442.

61.

Wang

Chencho

Jun

, et al. Deep residual network framework for structural health monitoring. Struct Heal Monit 2021; 20: 1443–1461.

62.

Shirley

. Report on the building structural health monitoring problem phase 1 analytical, 2011, https://datacenterhub.org/resources/2806.

63.

Dyke

Bernal

Beck

, et al. Experimental phase II of the structural health monitoring benchmark problem. Proc 16th ASCE Eng Mech Conf 2003: 1–7.

64.

Kim

Zhang

Chang

, et al. Ambient and vehicle-induced vibration data of a steel truss bridge subject to artificial damage. J Bridg Eng 2021; 26: 04721002.

65.

Zhou

Chen

Wang

. Convolutional deep networks for visual data classification. Neural Process Lett 2013; 38: 17–27.

66.

Günther

Pilarski

Helfrich

, et al. First steps towards an intelligent laser welding architecture using deep neural networks and reinforcement learning. Proced Technol 2014; 15: 474–483.

67.

Jarrett

Kavukcuoglu

Ranzato

, et al.

What is the best multi-stage architecture for object recognition?

Proc IEEE Int Conf Comput Vis 2009: 2146–2153.

68.

Ioffe

Szegedy

. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: 32nd international conference on machine learning, ICML 2015, Lille, France, July 7–9, 2015; 1: 448–456.

69.

Golik

Doetsch

Ney

. Cross-entropy vs. squared error training: a theoretical and experimental comparison. In: Proceedings of the annual conference of the international speech communication association, INTERSPEECH, Lyon, France, Aug 25–29, 2013; 1756–1760.

70.

Zhang

Ren

, et al. Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of 2015 IEEE international conference on computer vision (ICCV), Santiago, Chile, Dec 11–18, 2015: 1026–1034.

71.

Brochu

Cora

de Freitas

. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning, 2010, http://arxiv.org/abs/1012.2599.

72.

Zhang

Ding

. Deep residual learning-based fault diagnosis method for rotating machinery. ISA Trans 2019; 95: 295–305.

73.

Grover

. An introduction to Bayes’ theorem and Bayesian belief networks (BBN). In: Strategic Economic Decision-Making: Using Bayesian Belief Networks to Solve Complex Problems. New York, NY: Springer New York, 2013, pp. 1–9.

74.

Dyke

Louis

Bernal

, et al. An experimental benchmark problem in structural health monitoring. In: Third Int Work Struct Heal Monit Stanford, CA, Qingdao, China, 2001, pp. 1–10.

75.

van der

Hinton

. Visualizing data using t-SNE. J Mach Learn Res 2008; 9: 2579–2625.

76.

MATLAB R2019b (Version 9.7), 2019. https://www.mathworks.com/products/matlab/whatsne-w.html.