Cross working condition bearing fault diagnosis based on the combination of multimodal network and entropy conditional domain adversarial network

Abstract

In the realm of intelligent fault diagnosis, fault diagnosis methods based on deep learning have been widely used and have achieved tremendous success. However, traditional single-modal fault diagnosis methods face challenges in terms of accuracy and reliability under conditions such as noise interference. To address this issue, this paper proposes a cross working condition rolling bearing fault diagnosis method based on the combination of multimodal network and entropy conditional domain adversarial network (ECDAN). Firstly, the time-domain signal is transformed into a time–frequency matrix through continuous wavelet transform (CWT), and then a deep feature extraction network is designed. This network integrates convolutional neural network (CNN) and 2D-ResNet18 to extract features from both time-domain signals and time–frequency matrices, and fuses these features. In order to enhance the transferability of learning features, the adversarial strategy of ECDAN is utilized to ensure alignment of bearing sample data between the source and target domains. Experimental validation on bearing dataset from the comprehensive fault simulation test platform for machinery demonstrates the effectiveness of the proposed method, indicating its capability to handle complex and variable working conditions as well as noise interference.

Keywords

deep learning multimodal cross working conditions fault diagnosis entropy conditional domain adversarial networks

1. Introduction

For large mechanical equipment, rolling bearings play an irreplaceable role. Due to the need for rolling bearings and to operate under complex and variable conditions such as high temperature and speed, they are prone to malfunctions, affecting the normal operation of mechanical equipment and personnel safety (Qian et al., 2022). Consequently, implementing fault diagnosis tailored for rolling bearings is pivotal not only for safeguarding human life and assets but also for ensuring the unhindered operation of machinery (Liu et al., 2023). Machine learning is a subfield of artificial intelligence (AI) that is used to process complex data and perform various tasks (Anitescu et al., 2019; Samaniego et al., 2020). With the arrival of the big data era and the development of sensor technology, a large number of deep learning algorithms have been used to achieve intelligent diagnosis, such as convolutional neural networks (Chen et al., 2021), long short-term memory networks (Lia et al., 2023), autoencoders (Zhao et al., 2023), and generative adversarial networks (Wang et al., 2023). Tian et al. (Tian et al., 2023) solved the problem of the model not being dynamically adjusted by optimizing the CNN-LSTM model with mixed particle swarm optimization. Zhao et al. (Zhao et al., 2023) proposed a hybrid deep automatic encoder network that combines convolutional automatic encoders, novel fault detectors, and classifiers to achieve the detection of new faults and the classification of known faults. Pham et al. (Pham et al., 2022) proposed an improved fault diagnosis method for GAN, which transforms the transmitted monitoring signal into a two-dimensional spectrum through constant Q transformation to solve the problems of data imbalance and insufficient training samples. Gao et al. (Gao et al., 2022) used MCKD to filter and denoise rolling bearing fault signals and introduced the Adaptive Multi-Strategy Rhododendron Search Algorithm (MSACS) to optimize MCKD. Finally, the denoised signal was input into 1D CNN to achieve high-precision fault diagnosis. Wang et al. (Wang et al., 2021) proposed a new method for fusing multimodal sensor signals, using 1D CNN to extract features from raw vibration and acoustic signals, and perform feature fusion to achieve more accurate bearing fault diagnosis. Wang et al. (Wang et al., 2022) transformed multi-channel vibration signals into two-dimensional images using short time Fourier transform (STFT) and inputted them into 2D CNN, achieving good diagnostic results. Zhang et al. (Zhang et al., 2022) added a multi-scale information fusion layer to the 2D multi-scale cascaded CNN (2D MC-CNN) and used the time–frequency map obtained by converting the time-domain vibration signal using synchronous compression transformation as input, solving the problem of difficult extraction of important classification features under complex and variable equipment operating conditions. In order to more effectively express features, Ma et al. (Ma et al., 2022) proposed a model based on multimodal neural networks, which combines 2D and 1D convolutional structures for feature extraction and fully utilizes two-level information fusion to improve the efficiency and accuracy of fault diagnosis.

The above method requires that the training dataset and test dataset meet the same distribution, and sufficient fault labels are required for diagnosis (Zhao et al., 2021). However, operating conditions, equipment wear, and environmental noise can lead to changes in operating conditions, resulting in distribution differences between samples obtained under different conditions (Jiao et al., 2020). In practical industrial applications, there is very little fault data for certain equipment, making it difficult to collect sufficient samples from a single distribution, and there is very little fault data for certain equipment (Yao et al., 2021). Therefore, transfer learning is a good solution. Domain adaptation is the most commonly used transfer learning method, which can apply the knowledge learned from labeled data (source domain) to unlabeled data (target domain) to achieve cross domain fault diagnosis. Domain adaptation can be divided into domain adaptation based on distance measurement and domain adversarial adaptation. Domain adaptation based on distance measurement, such as MMD, maps both the source and target domains onto a reproducing kernel Hilbert space and then calculates the difference between the mean operations on the data of the source and target domains in this space, using this difference as the distance measurement between the two domains. Liu et al. (Liu et al., 2023) introduced MMD to minimize the distribution differences between the source and target domains, and combined vibration and acoustic data to achieve cross domain fault diagnosis. Li et al. (Li et al., 2021) proposed an optimal integrated deep transfer network (OEDTN) for fault diagnosis of rolling bearings. Multiple deep transfer networks (DTNs) were constructed using different kernel MMDs, and ensemble learning methods were used to combine these DTNs to obtain the final diagnostic results. For domain adversarial adaptation, such as domain adversarial neural networks (DANNs), domain adaptation is achieved by adding additional domain discriminant classifiers to the model. Liu et al. (Liu et al., 2021) proposed a deep adversarial domain adaptive model for rolling bearing fault diagnosis. This method combines a domain discriminator and two sub network models, and uses a stack autoencoder as a feature extraction method to solve the problem of uneven distribution between the source and target domains. However, there are still problems with adversarial domain adaptation methods: they cannot effectively align different domains of the inherent multimodal distribution in classification problems. Therefore, Long et al. (Long et al., 2018) proposed entropy conditional domain adversarial network (ECDAN). ECDAN can more accurately measure the uncertain predictions generated when processing samples that are difficult to migrate due to the use of maximum–minimum optimization methods in conditional domain discriminators by introducing entropy conditions, thereby improving diagnostic accuracy. Although the field adaptive fault diagnosis party has achieved satisfactory results, whether it uses the original vibration signal as the network input directly or converts the original vibration signal into a two-dimensional signal as the network input, it only considers the single mode of the signal. The information of bearing fault signals is rich and multi-directional, and the single mode feature extraction method cannot guarantee the effectiveness of the task of mining fault features.

This paper proposes a cross working condition rolling bearing fault diagnosis method based on the combination of multimodal networks and entropy conditional domain adversarial networks. The time-domain signal is converted into a time–frequency matrix through CWT, and the one-dimensional time-domain features of bearing faults are extracted using CNN. The 2D-ResNet18 network is used to extract the two-dimensional time–frequency matrix features and perform feature fusion. By combining it with the entropy conditional domain adversarial network and using adversarial thinking to align bearing sample data in both the source and target domains, rolling bearing fault diagnosis can be achieved under cross operating conditions. Its main contributions are as follows:

(1) By combining fault information from time-domain and time–frequency matrices, more comprehensive fault features can be obtained, thereby better distinguishing different types of faults.

(2) Introducing ECDAN into the CNN-ResNet multimodal model improves transferability by reweighting entropy perception weights, achieves cross working condition bearing fault diagnosis, and verifies its diagnostic performance in strong noise environments.

2. Theoretical background

2.1. ResNet

In response to the gradient vanishing problem caused by increasing the depth of convolutional neural networks, He et al. (He et al., 2016) proposed residual networks and introduced the concept of residual learning on the basis of convolutional neural networks. The schematic diagram of the residual block structure is shown in Figure 1.

The residual block contains two types of mappings: one is an identity mapping (in the right part of Figure 1) and the other type is residual mapping (left side of Figure 1). $x$ is the information input, and the output result is $H (X) = F (X) + X$ . When $H (X) = X$ , i.e., $F () = 0$ , the residual map is compressed to zero, leaving only the identity map. The network reaches its optimal state, and as the network depth increases, there will be no negative optimization in performance and it will remain in its optimal state.

2.2. Continuous wavelet transform

CWT is used to convert the original one-dimensional vibration information into a time–frequency distribution. Its adjustable time-frequency window enables CWT to have the adaptability to the signal, overcoming the disadvantage of short time Fourier transform (STFT) not being localized in both the time and frequency domains. For signal $x (t)$ , its continuous wavelet transform is

\begin{array}{l} T (a, b) = \int_{- \infty}^{+ \infty} x (t) ψ_{a, b} (t) d t \\ = \frac{1}{\sqrt{b}} \int_{- \infty}^{+ \infty} x (t) ψ (\frac{t - a}{b}) d t \end{array}

(1)

In the formula, b is the scale factor; a is the translation factor; and

ψ (t)

is the mother wavelet and its expression is

ψ_{a, b} (t) = \frac{1}{\sqrt{b}} ψ (\frac{t - a}{b})

(2)

Due to the similarity in shape between the Morlet wavelet and the impact signal generated during bearing failure (Liu et al., 2022), and the good balance between time and frequency localization, this article uses the Morlet wavelet and uses “cmor3-3” as the wavelet mother function.

2.3. Entropy conditional domain adversarial network

The conditional domain adversarial network (CDAN) draws inspiration from the conditional generative adversarial network (CGAN) by obtaining the cross covariance between the features extracted by the feature extractor and the classifier prediction, in order to enhance the model’s processing ability when the source and target domain data exhibit multimodal structures.

In order to mitigate the impact of uncertain prediction, the entropy condition $H (g) = - \sum_{c = 1}^{c} g_{c} \log g_{c}$ is used to quantify the uncertainty of bearing fault samples in prediction, where c is the total category of fault samples and $g_{c}$ represents the probability that the predicted fault samples are of class. The objective function of ECDAN is

\begin{array}{l} {\underset{G}{\min E}}_{(x_{i}^{s}, y_{i}^{s}) : D_{s}} L (G (x_{i}^{s}), y_{i}^{s}) \\ + λ (E_{x_{i}^{s} : D_{s}} ω (H (g_{i}^{s})) \log [D (T (h_{i}^{s}))] \\ + E_{x_{j}^{t} : D_{t}} ω (H (g_{j}^{i})) \log [1 - D (T (h_{j}^{t}))] \\ {\underset{D}{\max E}}_{x_{i}^{s} : D_{s}} ω (H (g_{i}^{s})) \log [D (T (h_{i}^{s}))] \\ + E_{x_{j}^{t} \sim :} ω (H (g_{j}^{t})) \log [1 - D (T (h_{j}^{t}))] \end{array}

(3)

In the formula,

D_{s}

is the labeled source domain bearing fault sample;

D_{t}

is the unlabeled bearing fault sample in the target domain;

x_{i}^{s}

and

y_{i}^{s}

represent the

i

-th sample of the bearing fault sample set in the source domain and its corresponding label;

x_{j}^{t}

represents the jth sample in the target domain bearing fault sample set;

λ

is the weight coefficient for weighing

G

and

D

; and

L

is the cross entropy loss.

ω (H (g))

is the weight of perceived entropy, and its expression is

ω (H (g)) = 1 + e^{- H (g)}

2.4. Leaky ReLU

In traditional CNN, ReLU is used as the activation function, and its mathematical expression is

f (x) = {\begin{cases} x, x \geq 0 \\ 0, x < 0 \end{cases}

(4)

From formula (4), it can be seen that when the input in the neural network is negative, its output is 0, which may discard some important information contained in these values, which is not conducive to extracting bearing fault features. The Leaky ReLU activation function reduces the oscillation during the learning process and improves the diagnostic accuracy of 1D CNN by introducing negative axis values (You et al., 2020). For this purpose, the CNN network in this article uses the Leaky ReLU activation function, and the mathematical expression of Leaky ReLU is

f (x) = {\begin{cases} x, x \geq 0 \\ α x, x < 0 \end{cases}

(5)

In the equation, when the input is negative, the output is controlled not to be 0. When it is 0, it is equal to the mathematical expression of ReLU.

3. Multimodal network model

3.1. Multimodal network structure

To extract feature information of various modes from the original vibration signal, using one-dimensional CNN has advantages in processing temporal data, which can capture local and global modes of the signal from a time-domain perspective. Similarly, by employing the 2D-ResNet18 for processing the time–frequency matrix features, it is possible to capture changes in signal time and frequency. Fusion of the extracted time-domain features and time–frequency matrix features is performed. Feature fusion can integrate different types of features and fully utilize their complementarity. The time-domain features extracted by one-dimensional CNN and the time–frequency features extracted by 2D-ResNet18 can provide fault information from different perspectives. Through fusion, the accuracy and robustness of fault diagnosis can be improved. The framework structure of CNN-ResNet is shown in Figure 2. For one-dimensional time-domain signals, a CNN composed of six convolutional layers and five pooling layers is used to extract features. The size of the six convolutional kernels gradually decreases. By gradually reducing the size of the convolutional kernels, CNN can extract features of different scales at different levels, while the number of convolutional kernels gradually increases, improving the network’s representation ability. For the time–frequency matrix obtained through CWT transformation, a 2D-ResNet18 network is selected to extract features. In the figure, “7 * 7, conv, 64, s2, p3” represents the convolutional kernel size of 7 * 7, the number of convolutional kernels is 64, the step size is 2, and the padding is 3.

3.2. Cross working condition bearing fault diagnosis process based on the combination of multimodal network and ECDAN

Aiming at the problems that different fault types may show different characteristics in different signal domains, that a single mode may not be able to fully capture the multi-dimensional characteristics of the signal, and that it is very sensitive to environmental noise and interference, a CNN-ResNet multimodal network for fault diagnosis of rolling bearings under Cross working condition is proposed and combined with ECDAN. The fault diagnosis flowchart is shown in Figure 3. The cross condition bearing fault diagnosis process based on the combination of multimodal network and entropy conditional domain adversarial network is as follows.

(1) Determine the source domain and target domain data based on the needs of cross task scenarios.

(2) Segregate the data from both the source and target domains into training and testing sets using an 8:2 proportion. The label sets of the source and target domains are consistent. Thereafter, standardize the samples utilizing Z-Score normalization (calculate the difference between each data point and the mean of the dataset, and divide it by the standard deviation of the dataset to convert the data into a distribution with a mean of 0 and a standard deviation of 1).

(3) Using the CNN-ResNet multimodal model to extract feature information and learn fault features, the extracted fault features are classified using a fault classifier and compared with real labels to obtain classification loss values.

(4) Using the CNN-ResNet multimodal model to extract the features of the source domain training set and the target domain training set samples, the domain discriminator is used to determine the domain of the features and calculate the discriminant loss value.

(5) When the number of iterations of the model reaches the set value, finalize the optimal CNN-ResNet multimodal model.

(6) Input the target domain test set into the model, test the performance of the trained model, output fault diagnosis results, and complete the cross working condition fault diagnosis process.

4. Experimentation

4.1. Experimental design and data acquisition

The test bench used for this experiment is Spectra Quest Co’s Mechanical Fault Simulator (MFS), as shown in Figure 4. The test bearing is installed on the side near the motor, and the piezoelectric acceleration sensor is installed on the bearing base closest to the test bearing to obtain the most accurate and reliable vibration signal possible. The bearing model used for the test is ER-12K, and the sampling frequency is 25.6 kHz. The bearing dataset for this experiment collected bearing vibration signals with rotation frequencies of 19.86 Hz, 29.86 Hz, and 39.86 Hz (represented by A, B, and C, respectively), for a total of three working conditions. Each working condition contains four different types of fault states (outer ring fault, inner ring fault, rolling element fault, and normal state). There are 327,680 sampling points for each type of faulty bearing, with a sample length of 1024. Each type of faulty bearing can be divided into 320 samples, resulting in a total of 1280 data samples for each working condition. In addition, to further verify the diagnostic applicability of the proposed method under different operating conditions, collect data under two operating conditions of 1HP and 3HP loads (represented by D and E, respectively). Each working condition contains three different types of fault states (outer ring fault, inner ring fault, and rolling element fault). Similarly, there are 327,680 sampling points for each type of faulty bearing, with a sample length of 1024. Each type of faulty bearing can be divided into 320 samples, resulting in a total of 960 data samples for each working condition.

4.2. Experimental parameter settings

In the cross working condition experimental task, each model trains 100 epochs, and the optimizer uses Adam. The initial learning rate is set to 0.001, which attenuates (multiplied by 0.1) in the 50th and 75th epochs (Zhao et al., 2021). The activation function of the classification layer adopts the Softmax classification function, with a training batch size of 64.

4.3. Comparison methods and fault diagnosis results

To verify the effectiveness and superiority of CNN-ResNet’s multimodal feature extraction method, it was compared with the other four methods on the same dataset.

Method 1: Using the original vibration signal of the bearing as the input, extracting time-domain features using WDCNN (Zhang et al., 2017), and using ECDAN for aligning the distributions of two domains.

Method 2: Using the original vibration signal of the bearing as the input, extracting time-domain features using AlexNet (Krizhevsky et al., 2017), and using ECDAN for aligning the distributions of two domains.

Method 3: Using the original vibration signal of the bearing as the input, extracting time-domain features using CNN (Zhao et al., 2021), and using ECDANs for aligning the distributions of two domains.

Method 4: Converting the original vibration signal of the bearing into a time–frequency matrix through CWT, extracting the time–frequency matrix features using 2D ResNet18 (He et al., 2016), and using ECDAN for aligning the distributions of two domains.

Proposed Method: Converting the original vibration signal of the bearing into a time–frequency matrix through CWT, extracting time-domain features using CNN and time–frequency matrix features using 2D ResNet18, performing feature fusion, and using ECDAN for aligning the distributions of two domains.

The experiment adopts 5-fold cross validation. Among the six cross working condition tasks with variable rotational frequency and two cross working condition tasks with variable load, the diagnostic results of the proposed method and other comparative methods are shown in Table 1 (Acc represents accuracy, Var represents variance), where the diagnostic accuracy is the average of 5-fold cross validation. From Table 1, it can be seen that the diagnostic performance of proposed method is better than other comparative methods in all cross working condition tasks. Method 3, Method 4, and the proposed method have achieved good results in the diagnosis of bearing faults across working conditions tasks of A→B, A→C, B→C, C→B, D→E, and E→D, with an average diagnostic accuracy of over 98%. However, Method 3 has a greater variance than Method 4 and the proposed method. The diagnostic accuracy of Method 1 and Method 2 in eight cross domain migration tasks is low and the variance is large, and they are not competent for cross working condition fault diagnosis tasks. To better demonstrate the recognition effect of the model on various faults in the test set, a confusion matrix is used to visualize the results of the test set. Figure 5 shows the confusion matrix of the migration task B→A using different feature extraction methods, with labels 0, 1, 2, and 3 corresponding to rolling element fault, inner circle fault, normal type, and outer circle fault. From Figure 5, it can be seen that in the migration task B→A, method 1, method 2, and method 3 show severe classification confusion, with poor classification performance for rolling element faults and outer ring faults. The diagnosis results of the outer ring fault are the worst, and both the proposed method and the four comparison methods in this article have diagnostic errors.

Table 1.

Diagnostic accuracy and variance of different feature extraction methods (100%).

		Method 1	Method 2	Method 3	Method 4	Proposed method
A→B	Acc	88.51	94.77	98.6	99.77	99.84
A→B	Var	7.99	8.07	1.26	0.12	0.05
A→C	Acc	86.17	85.7	98.13	99.22	99.84
A→C	Var	16.38	21.8	0.64	0.23	0.05
B→A	Acc	79.85	78.51	90	97.27	98.67
B→A	Var	16.07	23.41	6.53	0.3	0.2
B→C	Acc	94.38	94.92	99.38	100	100
B→C	Var	5.39	6.1	0.27	0	0
C→A	Acc	76.64	76.41	88.67	92.97	95.54
C→A	Var	7.51	5.39	1.65	2.59	0.35
C→B	Acc	93.75	92.73	98.6	99.45	99.53
C→B	Var	7.01	7.53	0.96	0.12	0.11
D→E	Acc	98.44	97.92	98.65	99.69	99.79
D→E	Var	1.49	3.4	1.03	0.08	0.08
E→D	Acc	98.23	97.19	98.75	99.79	99.9
E→D	Var	1.03	1.85	0.22	0.08	0.05

To verify the effectiveness and superiority of ECDANs, comparisons were made with three other deep transfer learning methods on the same bearing dataset. The comparative deep transfer learning methods include CORAL (Sun and Saenko, 2016), MK-MMD (Long et al., 2015), and DANN (Ganin et al., 2016). In the comparison of deep transfer learning methods, CNN-ResNet’s multimodal model is used as the backbone network. Table 2 presents the experimental results of four deep transfer learning fault diagnosis methods on six transferable tasks, and confusion matrix is shown in Figure 6.

Table 2.

Diagnostic accuracy and variance of different deep transfer learning methods (100%).

		CORAL	MK-MMD	DANN	Proposed method
A→B	Acc	78.52	99.53	99.45	99.84
A→B	Var	11.59	0.11	0.12	0.05
A→C	Acc	74.53	99.84	99.69	99.84
A→C	Var	3.76	0.05	0.11	0.05
B→A	Acc	70.08	98.13	98.28	98.67
B→A	Var	18.58	1.32	0.81	0.2
B→C	Acc	94.69	99.84	99.61	100
B→C	Var	5.08	0.05	0.46	0
C→A	Acc	54.06	95.55	93.52	95.54
C→A	Var	15.75	3.8	1.64	0.35
C→B	Acc	85.14	99.53	98.44	99.53
C→B	Var	13.92	0.11	0.23	0.11
D→E	Acc	96.04	99.9	99.69	99.79
D→E	Var	9.44	0.05	0.22	0.08
E→D	Acc	93.96	99.9	99.9	99.9
E→D	Var	21.8	0.05	0.05	0.05

From Table 2 and Figure 6, it can be seen that for the eight transfer tasks, except for CORAL, other deep transfer learning methods have achieved good diagnostic results. Compared with deep transfer learning methods such as CORAL, MK-MMD, and DANN, ECDAN has better comprehensive fault diagnosis performance in various transfer tasks.

4.4. Noise resistance test

The working environment of rotating machinery often contains a large amount of noise. In order to verify the cross working condition fault diagnosis performance of the model proposed in this article under strong noise conditions, a signal-to-noise ratio of −5 db was set. Similarly, compared with the four feature extraction methods mentioned above, the effectiveness and superiority of the multimodal feature extraction method of CNN-ResNet were verified. The experiment used 5-fold cross validation, and the results are shown in Figure 7.

Compared with deep transfer learning methods such as CORAL, MK-MMD, and DANN, the effectiveness and superiority of ECDAN in strong noise environments were verified. The experimental results are shown in and Figure 8.

From Figures 7 and 8, it can be seen that due to the influence of noise, the diagnostic accuracy of all methods for bearing datasets has decreased, but the proposed method still has good diagnostic accuracy. The diagnostic accuracy of the proposed method is lower than that of the DANN method in the A→C migration task, but in other migration tasks, this method is superior to other comparative methods. And from the upper and lower limits of the accuracy of 5-fold cross validation in Figures 7 and 8 and the variance of Tables 3 and 4, it can be seen that the diagnostic accuracy of the proposed method fluctuates relatively small.

Table 3.

Variance of different feature extraction methods in –5 dB noise environment.

	Method 1	Method 2	Method 3	Method 4	Proposed method
A→B	13.14	26.51	3.83	7.13	1.02
A→C	15.2	8.97	7.02	3.56	0.43
B→A	10.87	7.38	3.69	2.64	1.35
B→C	4.75	13.85	10.41	4.4	0.97
C→A	6.32	15.95	11.31	6.05	2.98
C→B	16.83	13.85	3.24	3.67	0.8
D→E	8.28	8.08	3.34	3.45	1.76
E→D	9.87	5.76	9.96	6.63	1.86

Table 4.

Variance of different deep transfer learning methods in –5 dB noise environment.

	CORAL	MK-MMD	DANN	Proposed method
A→B	25.37	3.11	8.05	1.02
A→C	17.88	5.83	11.9	0.43
B→A	17.61	19.38	2.33	1.35
B→C	26.98	4.32	1.52	0.97
C→A	13.47	3.39	13.07	2.98
C→B	29.17	6.51	3.68	0.8
D→E	18.89	3.45	2.43	1.76
E→D	21.35	3.72	7.78	1.86

5. Conclusion

Aiming at the problem that the distribution of bearing vibration data under different working conditions is inconsistent, a single mode cannot extract comprehensive fault features, and is vulnerable to environmental noise, this paper proposes a diagnosis method based on the combination of multimodal network and entropy condition domain countermeasure network. The time-domain signal is converted into a time–frequency matrix through CWT, and multimodal methods are used to process the time-domain signal and time–frequency matrix. The obtained features are fused to obtain more comprehensive fault information, improve fault diagnosis performance, and introduce ECDAN to achieve alignment of data multimodal distribution. Finally, the effectiveness and feasibility of the proposed method were verified through data from a mechanical comprehensive fault simulation testing platform.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Guangdong Provincial Key Laboratory of Intelligent Lithium Battery Manufacturing Equipment (2022B1212020003), AI Enabled Production Lifecycle Management for Flexible HMC (2022YFE0114100), Guangdong Provincial Education Science Planning Project (2019GXJK107), School Innovative and Enhance Engineering Project of Department of Education of Guangdong Province (E410918), and Innovation Project of Guangxi Graduate Education (YCBZ2023135).

ORCID iD

Yuanpeng Feng

References

Anitescu

Atroshchenko

Alajlan

, et al. (2019) Artificial neural network methods for the solution of second order boundary value problems. CMC-Computers, Materials & Continua 59(1): 345–359. DOI: 10.32604/cmc.2019.06641.

Chen

Liu

Yang

, et al. (2021) An improved fault diagnosis using 1d-convolutional neural network model. Electronics 10(1): 59. DOI: 10.3390/electronics1001005.

Ganin

Ustinova

Ajakan

, et al. (2016) Domain-adversarial training of neural networks. Journal of Machine Learning Research 17: 2030–2096. DOI: 10.5555/2946645.2946704.

Gao

Shi

Zhang

(2022) Rolling bearing compound fault diagnosis based on parameter optimization mckd and convolutional neural network. IEEE Transactions on Instrumentation and Measurement 71: 1–8. DOI: 10.1109/TIM.2022.3158379.

Zhang

Ren

et al. (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition. New Orleans: (CVPR). 77–778. DOI: 10.1109/CVPR.2016.90.

Jiao

Zhao

Lin

, et al. (2020) Classifier inconsistency-based domain adaptation network for partial transfer intelligent diagnosis. IEEE Transactions on Industrial Informatics 16(9): 5965–5974. DOI: 10.1109/TII.2019.2956294.

Krizhevsky

Sutskever

Hinton

(2017) Imagenet classification with deep convolutional neural networks. Communications of the ACM 60(6): 84–90. DOI: 10.1145/3065386.

Jiang

Wang

, et al. (2021) Rolling bearing fault diagnosis using optimal ensemble deep transfer network. Knowl-Based Syststem 213: 106695. DOI: 10.1016/j.knosys.2020.106695.

Lia

Sua

Hea

, et al. (2023) Research on fault diagnosis of highway bi-lstm based on attention mechanism. Eksploat Niezawodn 25(2): 110587. DOI: 10.17531/ein/162937.

10.

Liu

Wei

, et al. (2021) Deep adversarial domain adaptation model for bearing fault diagnosis. IEEE Transactions on Systems Man Cybernetics 51(7): 4217–4226. DOI: 10.1109/TSMC.2019.2932000.

11.

Liu

Zhang

Meng

, et al. (2022) Fault diagnosis of wind turbine bearings based on cnn and ssa - elm. Journal of Vibrational Engineering and Technologies 11: 1–17. DOI: 10.1007/s42417-022-00793-5.

12.

Liu

Xiang

Jiang

, et al. (2023) A domain adaption resnet model to detect faults in roller bearings using vibro-acoustic data. Sensors-Basel 23(6): 3068. DOI: 10.3390/s23063068.

13.

Long

Cao

Wang

et al. (2015) Learning transferable features with deep adaptation networks. International Conference on Machine Learning 37: 97–105.

14.

Long

Cao

Wang

et al. (2018) Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems 31: 1640–1650.

15.

Wen

Cheng

, et al. (2022) Multimodal convolutional neural network model with information fusion for intelligent fault diagnosis in rotating machinery. Measurement Science and Technology 33(12): 125109. DOI: 10.1088/1361-6501/ac7eb0.

16.

Pham

Kim

(2022) Rolling bearing fault diagnosis based on improved gan and 2-d representation of acoustic emission signals. IEEE Access 10: 78056–78069. DOI: 10.1109/ACCESS.2022.3193244.

17.

Qian

Jiang

Shen

, et al. (2022) An intelligent fault diagnosis method for rolling bearings based on feature transfer with improved densenet and joint distribution adaptation. Measurement Science and Technology 33(2): 25101. DOI: 10.1088/1361-6501/ac3b0b.

18.

Samaniego

Anitescu

Goswami

, et al. (2020) An energy approach to the solution of partial differential equations in computational mechanics via machine learning : concepts , implementation and applications. Computational Methods in Applied Mathematics 362: 112790. DOI: 10.1016/j.cma.2019.112790.

19.

Sun

Saenko

(2016) Deep coral : correlation alignment for deep domain adaptation. Computer Vision – ECCV 2016 9915: 443–450. DOI: 10.1007/978-3-319-49409-8_35.

20.

Tian

Fan

Feng

, et al. (2023) Fault diagnosis of rolling bearing based on hpso algorithm optimized cnn-lstm neural network. Sensors-Basel 23(14): 6508. DOI: 10.3390/s23146508.

21.

Wang

Mao

(2021) Bearing fault diagnosis based on vibro-acoustic data fusion and 1d-cnn network. Measurement. 173: 108518.

22.

Wang

Feng

Huo

, et al. (2022) A bearing fault diagnosis method based on spectrum map information fusion and convolutional neural network. Processes 10(7): 1426. DOI: 10.3390/pr10071426.

23.

Wang

Lang

, et al. (2023) Ftgan : a novel gan-based data augmentation method coupled time-frequency domain for imbalanced bearing fault diagnosis. IEEE Transactions on Instrumentation and Measurement 72: 1–14. DOI: 10.1109/TIM.2023.323409.

24.

Yao

Qin

Wang

, et al. (2021) Multiscale domain adaption models and their application in fault transfer diagnosis of planetary gearboxes. Engineering Applications of Artificial Intelligence 104: 104383. DOI: 10.1016/j.engappai.2021.104383.

25.

You

Shen

Wang

, et al. (2020) An intelligent deep feature learning method with improved activation functions for machine fault diagnosis. IEEE Access 8: 1975–1985. DOI: 10.1109/ACCESS.2019.2962734.

26.

Zhang

Peng

, et al. (2017) A new deep learning model for fault diagnosis with good anti-noise and domain adaptation ability on raw vibration signals. Sensors-Basel 17(2): 425. DOI: 10.3390/s17020425.

27.

Zhang

Huang

, et al. (2022) Bearing fault diagnosis under various operation conditions using synchrosqueezing transform and improved two-dimensional convolutional neural network. Measurement Science and Technology 33(8): 85002. DOI: 10.1088/1361-6501/ac69b1.

28.

Zhao

Zhang

, et al. (2021) Applications of unsupervised deep transfer learning to intelligent fault diagnosis : a survey and comparative study. IEEE Transactions on Instrumentation and Measurement 70: 1–28. DOI: 10.1109/TIM.2021.311630.

29.

Zhao

Hao

Chen

, et al. (2023) Novelty detection and fault diagnosis method for bearing faults based on the hybrid deep autoencoder network. Electronics 12(13): 2826. DOI: 10.3390/electronics12132826.