Improved group sparse decomposition and its application in gear fault diagnosis

Abstract

Group Sparse Mode Decomposition (GSMD) is an emerging signal decomposition technique that introduces the concept of group sparsity. It utilizes an ideal filter bank to decompose a signal into a series of orthogonal mode components. However, when decomposing complex vibration signals, GSMD is prone to over-decomposition, which compromises the integrity of fault information, rendering it unsuitable for fault diagnosis. To address this issue, this paper proposes an improved version of GSMD, called Improved Group Sparse Mode Decomposition (IGSMD). IGSMD first employs the group sparsity optimization optimizing strategy to decompose the signal into a series of group sparse sub-components. Further, IGSMD measures the fault intensity of these sub-components based on their periodic characteristics. Finally, the sub-components are fused according to their periodic strength to complete the signal decomposition. It is worth noting that IGSMD overcomes the over-decomposition issue inherent in GSMD when processing complex vibration signals. Additionally, IGSMD preserves the integrity of gear fault information, eliminates noise interference, and is well-suited for practical gear fault diagnosis. Simulation and real-world experimental results demonstrate that the proposed method exhibits superior performance in signal decomposition, enhancing the accuracy of gear fault diagnosis and showing considerable practical value.

Keywords

gear signal decomposition fault diagnosis periodic improve group sparse decomposition

1. Introduction

Gears are fundamental components in large-scale rotating machinery, playing a critical role in ensuring the operational performance and reliability of the entire mechanical system (Chang et al., 2024; Xie et al., 2023; Zheng et al., 2025). However, the vibration signals generated by gears are inherently complex and are often significantly contaminated by background noise during acquisition, complicating the extraction of meaningful fault-related information. To address this challenge, signal decomposition techniques have emerged as essential tools in vibration signal analysis, enabling the breakdown of complex signals into simpler components for enhanced fault diagnosis. Over the years, substantial research has been devoted to the development and refinement of signal decomposition methodologies (Chen et al., 2025; Cheng, et al. 2024, 2025; Zheng, et al. 2024, 2026).

Singular Value Decomposition (SVD) (Gerzen and Barthold, 2009; Sergio and Ulrych, 1988) is a widely utilized matrix-based signal decomposition technique. By constructing a Hankel matrix from the original signal, SVD decomposes the signal into characteristic components, allowing the removal of smaller singular values to mitigate noise. The singular values derived from this process encapsulate the intrinsic properties of the signal and effectively preserve critical discontinuities. Nevertheless, SVD lacks a well-established framework for determining the optimal rank order, limiting its robustness and generalizability.

In comparison, Singular Spectrum Analysis (SSA) (Elsner, 2002) provides a more structured approach to matrix decomposition and generally achieves superior results in signal processing. However, SSA is highly sensitive to parameter selection, particularly the embedding dimension, which can significantly influence noise reduction outcomes and lead to substantial variability in the decomposition results. Under high noise conditions, SSA often exhibits incomplete decomposition, with residual noise contaminating the extracted components.

Empirical Mode Decomposition (EMD) (Boudraa and Cexus, 2007) offers an adaptive approach by iteratively decomposing a signal into intrinsic mode functions (IMFs) based on its local characteristics. This makes EMD well-suited for analyzing nonlinear and non-stationary signals. However, EMD is hindered by the absence of a solid mathematical foundation and is highly sensitive to noise, which can lead to issues such as mode mixing and envelope distortion. To address these limitations, Wu and Huang introduced Ensemble Empirical Mode Decomposition (EEMD) (Zhou et al., 2025), an improved variant of EMD that employs noise-assisted decomposition to mitigate the effects of noise. Despite its advancements, EEMD is computationally intensive and struggles to completely eliminate the Gaussian white noise introduced during the process.

To improve the decomposition efficiency of EEMD, Yeh proposed the Complementary Ensemble Empirical Mode Decomposition (CEEMD) method (Zhou et al., 2025). Unlike EEMD, CEEMD introduces two opposing white noise signals into the original signal before performing EMD on each noise-perturbed version. This approach achieves comparable decomposition results to EEMD while significantly reducing the impact of white noise on the signal. Similar to EMD, Smith proposed Local Mean Decomposition (LMD), which adaptively decomposes a signal into several product functions (Li et al., 2014). The product function is obtained by multiplying an envelope signal with a pure frequency-modulated signal. As a result, the instantaneous amplitude and instantaneous frequency of the product function can be directly obtained from the envelope signal and the frequency-modulated signal, respectively.

Further enhancing CEEMD, the Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN) was developed (Li et al., 2019; Zhang et al., 2017). CEEMDAN adaptively injects controlled noise at each decomposition stage and computes unique residuals to derive each mode. This results in a more complete decomposition with minimal reconstruction error. Due to its superior noise robustness, lower reconstruction error, and fewer iterations, CEEMDAN has become a widely adopted tool for nonlinear and nonstationary signal analysis.

To overcome the theoretical constraints of EMD, Variational Mode Decomposition (VMD) was developed as a modern alternative (Li et al., 2020; Zhang et al., 2018). VMD formulates the decomposition problem as a constrained variational problem, optimizing the extraction of multiple IMFs using Wiener filter models. While VMD offers a more robust mathematical foundation, it requires manual configuration of parameters, such as the number of modes and initial center frequencies, and lacks standardized evaluation criteria for the decomposition results. Additionally, the IMFs produced by VMD are not inherently orthogonal, which can compromise the accuracy of signal reconstruction. Since Variational Mode Decomposition (VMD) requires manual parameter selection, Li et al. introduced an adaptive energy-constrained VMD algorithm based on spectral segmentation (Li et al., 2021). This method automatically determines optimal VMD parameters using Fourier spectral segmentation and the Gini index, ensuring sharper energy concentration in each extracted mode.

Feature Mode Decomposition (FMD) is a sophisticated decomposition technique that leverages the design of adaptive finite impulse response (FIR) filters (Miao et al., 2022). The core principle of FMD involves the utilization of adaptive FIR filters to perform mode decomposition, wherein it simultaneously accounts for both the impulsive and periodic characteristics inherent in fault signals. By cyclically eliminating redundant and periodic modes, FMD facilitates the precise extraction of fault features, while maintaining a certain degree of robustness against noise and interference.

Moreover, the application of stochastic resonance systems and blind source deconvolution methods for the analysis of vibration signals has gained considerable traction, particularly within the domain of machinery fault diagnosis. For instance, CHEN (Chen et al., 2015) integrated EEMD with an adaptive stochastic resonance method to enhance the extraction of early-stage, subtle fault features in planetary gears. This approach has proven effective in amplifying fault signatures. Additionally, methods such as minimum entropy deconvolution (McDonald and Zhao, 2017), maximum correlation kurtosis deconvolution (McDonald et al., 2012), and their improved variants have found extensive application in mechanical vibration systems, further advancing fault detection and diagnostic capabilities.

Fourier Decomposition Method (FDM) (Singh et al., 2017) is a nonlinear and non-stationary signal analysis technique rooted in Fourier theory. It decomposes a signal into a set of Fourier Intrinsic Band Functions (FIBFs), ensuring a complete and orthogonal representation. Expanding on this framework, Singh et al. introduced the Perfect Reconstruction Filter Banks (PRFB) approach (Singh et al., 2023), which employs diverse smoothing filters to further decompose signals. However, a key limitation of both FDM and PRFB is the potential generation of numerous physically meaningless FIBF modes, reducing their practical utility.

In recent years, spectrum segmentation-based decomposition methods have gained significant attention. A prominent example is the Empirical Wavelet Transform (EWT) (Gilles, 2013), which partitions the signal spectrum into contiguous adaptive intervals and reconstructs the signal using compactly supported wavelet filters to extract amplitude- and frequency-modulated components. Despite its effectiveness, EWT suffers from mode mixing when processing complex signals with closely spaced or overlapping modes.

To overcome this limitation, Empirical Fourier Decomposition (EFD) (Zhou et al., 2022) was developed as an alternative spectrum segmentation-based approach. EFD integrates enhanced spectral segmentation techniques with zero-phase rectangular filters, enabling effective separation of complex signals into distinct modes by prespecifying the decomposition number. While EFD demonstrates strong performance in handling intricate signals, it relies on the lowest minima criterion for spectrum segmentation and requires manual selection of the mode number that can lead to over-segmentation or under-segmentation, limiting its adaptability.

To further enhance the capabilities of signal decomposition, Group Sparse Mode Decomposition (GSMD) (Mourad, 2022, 2023) was proposed by Mourad as an innovative approach. GSMD incorporates group sparsity principles and employs an ideal filter bank in the Discrete Cosine Transform domain (DCT domain) to decompose signals into orthogonal IMFs. It is computationally efficient and can autonomously determine parameters, such as the number of modes, based on signal characteristics. However, GSMD’s reliance on frequency-domain decomposition introduces potential estimation errors in the ideal filters, particularly under high noise conditions. These errors may lead to the fragmentation of fault-related components into multiple low-energy subcomponents, resulting in over-decomposition and compromising the integrity of mechanical fault features.

Through deeper analysis, it has been found that periodic components are important indicators of gear faults, whereas noise typically generates random and sparse strong impact disturbances. Based on this understanding, this paper proposes an Improved Group Sparse Mode Decomposition (IGSMD) method. First, IGSMD proposes the group sparsity optimization optimizing strategy, which utilizes the frequency-domain group sparsity characteristic of fault features to construct a series of ideal filter banks that decompose the signal into group sparse sub-components. Next, IGSMD calculates the periodic features of group sparse sub-components. Finally, based on the periodic features, the sub-components are merged to recover the fault information. It is noteworthy that IGSMD effectively preserves the integrity of gear fault features, suppressing noise interference effectively. IGSMD exhibits several advantages:

1. IGSMD proposed the group sparsity optimization optimizing strategy, to decompose the signal into a series of group sparse sub-components, significantly reducing noise interference and demonstrating robustness to noise.

2. IGSMD proposed the periodic merge strategy, which protects the integrity of mechanical fault features and avoid over-decomposition.

IGSMD can protect the integrity of fault information while weakening noise interference, making it highly suitable for gear fault diagnosis. The remaining structure of this paper is organized as follows: Section 2 presents the theory of IGSMD, Section 3 discusses the analysis of simulated signals, Section 4 covers the analysis of experimental signals, and Section 5 concludes the paper.

2. Improved Group Sparse Mode Decomposition

2.1. Related work: Group Sparse Mode Decomposition

Group Sparse Mode Decomposition is a recently emerging signal decomposition method that incorporates the concept of group sparsity into signal decomposition. This method decomposes signals based on their frequency-domain characteristics.

GSMD assumes that each signal consists of a series of Intrinsic Mode Functions (IMFs):

y (t) = x (t) + n (t) = \sum_{j = 1}^{J} h_{j} (t) + n (t), t = 1, . . ., N

(1)

where

y (t)

represents the original signal,

x (t)

denotes the useful component,

n (t)

represents noise, and

h_{j} (t)

is the j-th IMF.

The group sparse signal decomposition theory makes the following assumptions about the intrinsic mode functions:

1. In the DCT domain, each IMF has compact support, and due to the bandwidth limitations of the modes, all elements within each support set are adjacent.

2. The support sets of different components are non-adjacent and have certain intervals between them.

Given these assumptions, equation (1) can be rewritten as follows:

c_{y} = c_{x} + c_{v} = \sum_{j = 1}^{J} c_{hj} + c_{v}

(2)

where

c_{y}

c_{x}

c_{v}

, and

c_{hj}

represent the DCT coefficients of

y (t)

x (t)

n (t)

, and

h_{j} (t)

, respectively.

The corresponding group sparse filter can be obtained by solving the optimization formula shown in equation (3).

\hat{f} = argmin P (f) : = {‖ c_{y} - c_{x} ‖}_{2}^{2} = {‖ c_{y} - c_{y} \circ f ‖}_{2}^{2}

(3)

where

\hat{f}

is the ideal filter,

{‖ \cdot ‖}_{2}

is the Euclidean norm,

\circ

is the Hadamard product, and

\hat{f}

is the ideal filter.

2.2. IGSMD

While GSMD is capable of decomposing signals into a series of orthogonal mode components, it is prone to errors in estimating the ideal filter in noisy environments. In such cases, some fault-related components may be split into multiple lower-energy sub-components, which compromises the mechanical fault feature information. To address this issue, this paper proposes an Improved Group Sparse Mode Decomposition (IGSMD) method.

IGSMD utilizes group sparsity to construct an ideal filter bank that divides the signal into a series of sub-modes. It then reconstructs the mode components by employing a periodic feature fusion strategy, extracting fault characteristics hidden within the noise. Below, we detail the algorithmic process of IGSMD.

Let y be the initial time series and $c_{y}$ is DCT coefficients. We establish the following Group Sparse Optimization Model:

\hat{f} = argmin J (f) : = {‖ c_{y} - c_{y} \circ f ‖}_{2}^{2} + λ {‖ f ‖}_{0, w}

(4)

where

λ

is the regularization parameter and

\hat{f}

represents the ideal filter.

The DCT transformation is a linear transformation, so equation (4) is separable. The i-th element of the ideal filter can be represented by the following optimization:

{\hat{f}}_{i} = argmin J (f) : = {(c_{y} [i])}^{2} (1 - f) + λ w_{i} I_{{f}}, i = 1, . . ., T

(5)

where

w_{i}

denotes the filter coefficients and

{\hat{f}}_{i}

represents the sub-matrix corresponding to the i-th component.

By solving this optimization, we obtain the optimal ideal filter vector $\hat{f}$ with a group sparse structure. The ideal filter vector $\hat{f}$ can be expressed as a sum of J sub-filters. Each sub-filter can extract specific sub-components of the signal in the DCT domain. We extract the sub-filter vectors from the ideal filter $\hat{f}$ as follows:

{f_{1}, f_{2}, \dots, f_{T}}

(6)

The time-domain representation of each component is obtained by applying the inverse DCT (IDCT), yielding the decomposed sub-components, denoted as GSC:

G S C_{i} = T^{- 1} (f_{i} \circ c_{y})

(7)

Using the filter bank represented by equation (7), the original signal is forced into a series of individual components. However, due to strong noise interference, some characteristic components may be split into multiple modes. As a result, the components are not mutually independent. Therefore, it is necessary to reconstruct independent components based on the intrinsic characteristics of the initial sub-components.

To achieve this, for $d$ initial effective sub-components, we first calculate their periodicity. Next, we select the first initial sub-component $Y_{1}$ and compare it periodically with the remaining effective sub-components. If the error is within an acceptable range, we recombine the sub-components to obtain the first Intrinsic Mode Functions $I M F_{1}$ . Then, we remove the $I M F_{1}$ from the set and sum the remaining sub-components, denoted as $g_{1}$ . Finally, the first remaining effective sub-component is compared with the others for periodicity, and this process continues iteratively to obtain $I M F_{2}$ .

Through $h$ iterations, the $h$ th Intrinsic Mode Functions $I M F_{h}$ are obtained, and the sum of the remaining initial sub-components is denoted as $g_{h}$ . The stopping condition for the decomposition is as follows: when the normalized mean square error (NMSE) of the residual signal $g_{h}$ is smaller than a preset threshold, the decomposition process ends; otherwise, iterative decomposition continues. The stopping condition is given by the following equation:

N M S E^{h} = \frac{\sum_{e = 1}^{n} g^{h} (e)}{\sum_{e = 1}^{n} x (e)}

(8)

where

g^{h} (e)

and

x (e)

is the normalized mean square error of

g^{h}

and

x

, respectively.

The flowchart of fault diagnosis based on IGSMD is shown as Figure 1.

Figure 1.

The flowchart of fault diagnosis based on IGSMD.

3. Numerical simulation experiment

In this section, the gear signal simulation experiment is carried out to confirm the efficiency of IGSMD in gear fault diagnosis. The simulation signal expression is shown as follows:

x (t) = [1 - 0.2 \cos (2 π f_{s r} t)] \times [1 + A \cos (2 π f_{g} t)] \times \cos [2 π f_{m} t + B \sin (2 π f_{g} t) + ϕ]

(9)

where rotation frequency of driving wheel

f_{s r} = 10 Hz

, fault characteristic frequency of driving wheel

f_{g} = 30 Hz

, meshing frequency

f_{m} = 200 Hz

, amplitude modulation coefficient

A = 2.3

, frequency modulation coefficient

B = 1.6

, and initial phase

ϕ = 0

We add −10 dB Gaussian white noise in the numerical simulation experiment. The waveforms and envelope spectrum of the noise-containing signals are depicted in Figure 2.

Figure 2.

The waveform and envelope spectrum of simulation signal: (a) waveform; (b) envelope spectrum.

It can be observed that the amplitude of the fault characteristic frequency is overwhelmed by the noise frequency, making it difficult to accurately assess the gear’s condition. Therefore, six signal decomposition methods—IGSMD, GSMD, EEMD, FMD, VMD, and SSA—are employed to decompose the vibration signal, aiming to reduce the noise components in the gear vibration signal.

The settings for the comparison methods are as follows: The noise value added in EEMD is 0.2, with an average frequency set to 120. SSA determines the window function size using the PSD. For GSMD, the maximum window size is set to 20, while IGSMD is configured with a window size of 20 and a period fusion threshold of 0.2.

We use the correlation kurtosis (Wang et al., 2025) as the metric for identifying the optimal mode. The mode with the highest correlation kurtosis is considered the fault mode. Specifically, IGSMD decomposes 3 modes, with the second mode identified as the fault mode. GSMD decomposes 8 modes, with the third mode identified as the fault mode. EEMD decomposes 12 modes, with the third mode identified as the fault mode, and SSA decomposes 5 modes, with the fourth mode identified as the fault mode. VMD decomposes 5 modes, with the 3rd mode identified as the fault mode. FMD decomposes 5 modes, with the 4th mode identified as the fault mode. The time-domain and envelope spectrum plots of the fault modes obtained from the six decomposition methods are shown below.

Figures 3 –8 show the time-domain waveforms and corresponding envelope spectra for the six methods. Upon observation, it is evident that the fault mode obtained using GSMD has the poorest decomposition effect, with the time-domain waveform lacking clear modulation characteristics, making it impossible to determine whether the gear has a fault.

Figure 3.

The optimal mode from GSMD: (a) waveform; (b) envelope spectrum.

Figure 4.

The optimal mode from EEMD: (a) waveform; (b) envelope spectrum.

Figure 5.

The optimal mode from SSA: (a) waveform; (b) envelope spectrum.

Figure 6.

The optimal mode from VMD: (a) waveform; (b) envelope spectrum.

Figure 7.

The optimal mode from FMD: (a) waveform; (b) envelope spectrum.

Figure 8.

The optimal mode from IGSMD: (a) waveform; (b) envelope spectrum.

In contrast, the fault modes obtained using IGSMD, SSA, VMD, FMD, and EEMD exhibit distinct fault characteristic information in the time-domain waveforms, and their corresponding envelope spectra show a very clear amplitude at the fault characteristic frequency. However, IGSMD shows very prominent amplitude at the fault characteristic frequency and its harmonics, with no noise interference surrounding the amplitude. Overall, its performance is superior to that of SSA and EEMD.

It is noteworthy that both IGSMD and GSMD adopt the concept of group sparsity for signal decomposition. However, GSMD fails to consider the periodic characteristics of the fault information, leading to the fault mode being mis-decomposed into several meaningless modes, making gear fault diagnosis difficult. On the other hand, IGSMD employs a periodic fusion strategy, which reconstructs the fault information of the signal while eliminating noise interference, making it more suitable for fault diagnosis.

4. Bevel gear box experiment

To evaluate the performance of the proposed IGSMD method in signal decomposition in gear vibration signals under practical working conditions, we intend to conduct tests using experimental data from real-world bevel gear fault scenarios. The experimental setup features a bevel gear system where the driving gear has 12 teeth and the driven gear has 24 teeth. Gear vibration signals are captured using LMS SCM09 as the signal acquisition system, equipped with PCB 356A25 acceleration sensors. The sampling frequency for all recorded fault signals is set at 4096 Hz. The physical diagram and fault bevel gears are shown in Figure 9. By applying the IGSMD method to these complex experimental datasets, we aim to validate its capability in extracting fault features accurately under realistic noise conditions.

Figure 9.

The physical diagram and fault bevel gears.

4.1. Experiment 1

A pitting fault experiment on the bevel gear is set up to verify the effectiveness of IGSMD in fault diagnosis. The fault characteristic frequency of the bevel gear pitting fault signal is 25 Hz. In our experiment, the driving gear has 12 teeth and the driven gear has 24 teeth. The rotational speed is set to 1500 rpm, which corresponds to a rotational frequency of 25 Hz for the driving gear. The gear mesh frequency can therefore be calculated as the number of teeth on the driving gear multiplied by its rotational frequency, or equivalently, the number of teeth on the driven gear multiplied by its rotational frequency (Wang et al., 2025). The calculation of gear mesh frequency is as follows:

f_{m} = f_{g 1} \times Z_{1} = 300 H z

(10)

where

f_{g 1}

is the rotational frequency of driving gear and

Z_{1}

is the number of teeth on driving gear.

Since the test rig is a single-stage gear system and the fault is located on the driving gear, the fault characteristic frequency is equal to the rotational frequency of the driving gear.

Figure 10 shows the time-domain waveform and envelope spectrum of the original signal. It can be observed that the amplitude of the fault characteristic frequency is overwhelmed by the noise frequency, making it difficult to accurately assess the gear’s condition. Therefore, six signal decomposition methods—IGSMD, GSMD, EEMD, FMD, VMD, and SSA—are applied to decompose the vibration signal and reduce the noise components in the gear vibration signal.

Figure 10.

The waveform and envelope spectrum of gear pitting signal: (a) waveform; (b) envelope spectrum.

The parameters for the comparison methods in this experiment were selected based on empirical observations, with each parameter representing the optimal empirical value. The specific parameter settings are as follows: for EEMD, the added noise value is 0.2, and the average frequency is set to 120; SSA determines the window function size using PSD; GSMD sets the maximum window size to 20; and IGSMD configures the window size to 20 with a periodic fusion threshold of 0.2.

IGSMD decomposed the signal into 8 modes, with the 6th mode identified as the fault mode. GSMD produced 30 modes, with the 9th mode identified as the fault mode. EEMD yielded 12 modes, with the 1st mode identified as the fault mode. SSA decomposed the signal into 5 modes, with the 4th mode identified as the fault mode. VMD decomposes 5 modes, with the 4th mode identified as the fault mode. FMD decomposes 5 modes, with the 2nd mode identified as the fault mode.

The time-domain waveforms and envelope spectra of the fault modes obtained from the six decomposition methods are shown below.

Figures 11 –16 show the time-domain waveforms and corresponding envelope spectra obtained using the six methods. Upon observation, it is evident that EEMD delivers the poorest fault mode decomposition, with its time-domain waveform lacking distinct modulation characteristics, making it impossible to determine whether the gear is faulty. GSMD exhibits severe over-decomposition, producing significant amplitudes at half the fault characteristic frequency. While VMD and SSA show a clear amplitude at the fault characteristic frequency, it is accompanied by substantial noise interference around the fault characteristic frequency, making it difficult to assess the gear’s condition.

Figure 11.

The optimal mode from GSMD: (a) waveform; (b) envelope spectrum.

Figure 12.

The optimal mode from EEMD: (a) waveform; (b) envelope spectrum.

Figure 13.

The optimal mode from SSA: (a) waveform; (b) envelope spectrum.

Figure 14.

The optimal mode from VMD: (a) waveform; (b) envelope spectrum.

Figure 15.

The optimal mode from FMD: (a) waveform; (b) envelope spectrum.

Figure 16.

The optimal mode from IGSMD: (a) waveform; (b) envelope spectrum.

In contrast, both FMD and IGSMD extract fault mode components with clearly distinct amplitudes at the fault characteristic frequency and its harmonics in the envelope spectrum, with minimal noise interference around these frequencies. Based on this analysis, it is clear that IGSMD outperforms the other methods in signal decomposition and is more suitable for gear fault diagnosis. Additionally, IGSMD shows less noise interference compared to FMD, resulting in superior overall performance.

4.2. Experiment 2

To further validate the effectiveness of IGSMD in gear fault diagnosis, a gear crack fault experiment is planned for additional verification. The sampling frequency of the bevel gear tooth breakage fault signal is 4096 Hz, with a motor speed of 1500 rpm and a fault characteristic frequency of 25 Hz.

Figure 17 presents the time-domain waveform and envelope spectrum of the original signal, respectively. It can be observed that the amplitude at the fault characteristic frequency is overwhelmed by noise frequencies, making it difficult to accurately assess the gear’s condition.

Figure 17.

The waveform and envelope spectrum of crack signal: (a) waveform; (b) envelope spectrum.

To further diagnose the fault, the vibration signal was decomposed using six signal decomposition methods: IGSMD, GSMD, EEMD, SSA, FMD, and VMD, aiming to suppress the noise components within the gear vibration signal. IGSMD produced 4 modes, with the 4th mode identified as the fault mode. GSMD decomposed the signal into 38 modes, with the 19th mode being the fault mode. EEMD generated 12 modes, with the 1st mode determined as the fault mode, while SSA yielded 5 modes, with the 1st mode identified as the fault mode. VMD and FMD decomposed the signal into 5 modes, with the 3rd mode being the fault mode.

The time-domain waveforms and envelope spectra of the fault modes obtained from these six decomposition methods are presented below.

Figures 18 –23 illustrate the time-domain waveforms and corresponding envelope spectra obtained using the six methods. Upon observation, the time-domain waveforms of the fault modes derived from SSA, GSMD, FMD, VMD, and IGSMD exhibit strong modulation characteristics, while the fault mode waveform from EEMD appears relatively noisy.

Figure 18.

The optimal mode from GSMD: (a) waveform; (b) envelope spectrum.

Figure 19.

The optimal mode from EEMD: (a) waveform; (b) envelope spectrum.

Figure 20.

The optimal mode from SSA: (a) waveform; (b) envelope spectrum.

Figure 21.

The optimal mode from VMD: (a) waveform; (b) envelope spectrum.

Figure 22.

The optimal mode from FMD: (a) waveform; (b) envelope spectrum.

Figure 23.

The optimal mode from IGSMD: (a) waveform; (b) envelope spectrum.

Analyzing the envelope spectra of the fault modes reveals that EEMD provides the poorest fault mode decomposition, with highly noisy envelope spectra that makes it impossible to determine whether the gear is faulty. Although GSMD, FMD, VMD, and SSA show distinct amplitudes at the fault characteristic frequency, they are accompanied by significant noise interference around the characteristic frequency, making it difficult to assess the gear’s health condition. In contrast, the envelope spectrum of the fault mode obtained by IGSMD shows clear and prominent amplitudes at the fault characteristic frequency and its harmonics, with no noise interference around the amplitudes.

It is worth noting that both IGSMD and GSMD are based on group sparsity for decomposition. However, GSMD suffers from severe over-decomposition, splitting the fault mode into a series of meaningless components. IGSMD, on the other hand, leverages the periodic characteristics of the mode components for fusion and reconstruction, effectively removing non-periodic noise components while preserving the integrity of fault information.

From the above analysis, when dealing with complex signal noise, IGSMD excels at highlighting the periodic modulation information in the signal, thereby enhancing the fault characteristics of gear teeth. Given its superior performance in gear fault diagnosis, the proposed method demonstrates significant practical application value.

5. Conclusion

To address the limitations of GSMD in fault diagnosis, we propose an improved Group Sparse Mode Decomposition method (IGSMD). IGSMD first applies a group sparsity optimization strategy to decompose the signal into a series of group-sparse subcomponents. It then evaluates the fault intensity of these subcomponents based on their periodic characteristics. Finally, the subcomponents are fused according to their periodic strength to complete the signal decomposition. Notably, IGSMD overcomes the over-decomposition problem encountered in GSMD when analyzing complex vibration signals, while preserving the integrity of gear fault information and effectively suppressing noise interference. This makes it well-suited for practical applications in gear fault diagnosis. Both simulation and real-world experimental results validate the effectiveness of IGSMD in early-stage fault detection, demonstrating superior noise reduction performance compared to EEMD, SSA, FMD, VMD, and GSMD.

Although IGSMD performs remarkably well in gear fault diagnosis, it currently requires manual setting of parameters for periodic fusion and group sparse filtering, which limits its adaptability. In future work, we will focus on developing adaptive parameter optimization methods to enhance its robustness and usability.

Footnotes

ORCID iD

Xianbin Zheng

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is supported by the National Natural Science Foundation of China (52275103).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Boudraa

A-O

Cexus

J-C

(2007) EMD-based signal filtering. IEEE Transactions on Instrumentation and Measurement 56(6): 2196–2202. https://doi.org/10.1109/tim.2007.907967

Chang

Jia

Han

, et al. (2024) Towards more reliable photovoltaic energy conversion systems: a weakly-supervised learning perspective on anomaly detection. Energy Conversion and Management 316(1): 118845. https://doi.org/10.1016/j.enconman.2024.118845

Chen

Cheng

Shan

, et al. (2015) Research of weak fault feature information extraction of planetary gear based on ensemble empirical mode decomposition and adaptive stochastic resonance. Measurement 73: 55–67. https://doi.org/10.1016/j.measurement.2015.05.007

Chen

Liu

Rao

, et al. (2025) Explicit speed-integrated LSTM network for non-stationary gearbox vibration representation and fault detection under varying speed conditions. Reliability Engineering & System Safety 254: 110596. https://doi.org/10.1016/j.ress.2024.110596

Cheng

Pan

Zheng

(2024) Maximum ramanujan spectrum signal-to-noise ratio deconvolution method: algorithm and applications. IEEE Transactions on Industrial Informatics 20(10): 11977–11986. https://doi.org/10.1109/tii.2024.3413347

Cheng

Pan

Zheng

, et al. (2025) Global optimal ramanujan spectrum: a feature extraction method without pseudo-monotonicity. Expert Systems with Applications 260(4): 125425. https://doi.org/10.1016/j.eswa.2024.125425

Elsner

(2002) Analysis of time series structure: SSA and related techniques. Journal of the American Statistical Association 97(460): 1207–1208. https://doi.org/10.1198/jasa.2002.s239

Gerzen

Barthold

F-J

(2009) Applications of singular value decomposition in structural optimisation. Proceedings in Applied Mathematics and Mechanics 9(1): 579–580. https://doi.org/10.1002/pamm.200910261

Gilles

(2013) Empirical wavelet transform. IEEE Transactions on Signal Processing 61(16): 3999–4010. https://doi.org/10.1109/tsp.2013.2265222

10.

Wang

Tang

, et al. (2014) An improved method based on CEEMD for fault diagnosis of rolling bearing. Advances in Mechanical Engineering 6: 676205. https://doi.org/10.1155/2014/676205

11.

Liu

, et al. (2019) Review of local mean decomposition and its application in fault diagnosis of rotating machinery. Journal of Systems Engineering and Electronics 30(4): 799–814. Available at: https://doi.org/10.21629/JSEE.2019.04.17

12.

Liu

, et al. (2020) An optimized VMD method and its applications in bearing fault diagnosis. Measurement 166: 108185. https://doi.org/10.1016/j.measurement.2020.108185

13.

Cheng

, et al. (2021) Adaptive energy-constrained variational mode decomposition based on spectrum segmentation and its application in fault detection of rolling bearing. Signal Processing 183: 108025. https://doi.org/10.1016/j.sigpro.2021.108025

14.

McDonald

Zhao

(2017) Multipoint optimal minimum entropy deconvolution and convolution fix: application to vibration fault detection. Mechanical Systems and Signal Processing 82: 461–477. https://doi.org/10.1016/j.ymssp.2016.05.036

15.

McDonald

Zhao

Zuo

(2012) Maximum correlated Kurtosis deconvolution and application on gear tooth chip fault detection. Mechanical Systems and Signal Processing 33: 237–255. https://doi.org/10.1016/j.ymssp.2012.06.010

16.

Miao

Zhang

, et al. (2022) Feature mode decomposition: new decomposition theory for rotating machinery fault diagnosis. IEEE Transactions on Industrial Electronics 70(2): 1949–1960. https://doi.org/10.1109/TIE.2022.3156156

17.

Mourad

(2022) Group-sparse mode decomposition: a signal decomposition algorithm based on group-sparsity in the frequency domain. Digital Signal Processing 122: 103375. https://doi.org/10.1016/j.dsp.2021.103375

18.

Mourad

(2023) Multivariate group-sparse mode decomposition. Digital Signal Processing 137: 104024. https://doi.org/10.1016/j.dsp.2023.104024

19.

Sergio Ulrych

(1988) Application of singular value decomposition to vertical seismic profiling. Geophysics 53(6): 778–785. https://doi.org/10.1190/1.1442513

20.

Singh

Joshi

Patney

, et al. (2017) The fourier decomposition method for nonlinear and non-stationary time series analysis. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 473(2199): 20160871. https://doi.org/10.1098/rspa.2016.0871

21.

Singh

Singhal

Fatimah

, et al. (2023) A novel PRFB decomposition for non-stationary time-series and image analysis. Signal Processing 207: 108961. https://doi.org/10.1016/j.sigpro.2023.108961

22.

Wang

Guo

Kang

, et al. (2025) A novel fault feature extraction method for planet-bearing based on generalized total variation model. IEEE Sensors Journal 25(11): 20870–20879. https://doi.org/10.1109/JSEN.2025.3558822

23.

Xie

Han

Pei

, et al. (2023) A unified out-of-distribution detection framework for trustworthy prognostics and health management in renewable energy systems. Engineering Applications of Artificial Intelligence 125: 106707. https://doi.org/10.1016/j.engappai.2023.106707

24.

Zhang

, et al. (2017) A combined model based on CEEMDAN and modified flower pollination algorithm for wind speed forecasting. Energy Conversion and Management 136: 439–451. https://doi.org/10.1016/j.enconman.2017.01.022

25.

Zhang

Miao

Zhang

, et al. (2018) A parameter-adaptive VMD method based on grasshopper optimization algorithm to analyze vibration signals from rotating machinery. Mechanical Systems and Signal Processing 108: 58–72. https://doi.org/10.1016/j.ymssp.2017.11.029

26.

Zheng

Yang

, et al. (2024) A novel empirical reconstruction Gauss decomposition method and its application in gear fault diagnosis. Mechanical Systems and Signal Processing 210: 111174. https://doi.org/10.1016/j.ymssp.2024.111174

27.

Zheng

Cheng

, et al. (2025) Blaschke learning machine: a novel and efficient continual learning classifier toward intelligent fault diagnosis. Mechanical Systems and Signal Processing 239: 113350. https://doi.org/10.1016/j.ymssp.2025.113350

28.

Zheng

Liu

Jiang

, et al. (2026) Geometric algebra singular spectrum analysis: a novel multivariate signal decomposition method. Mechanical Systems and Signal Processing 252: 114267. https://doi.org/10.1016/j.ymssp.2026.114267

29.

Zhou

Feng

, et al. (2022) Empirical fourier decomposition: an accurate signal decomposition method for nonlinear and non-stationary time series analysis. Mechanical Systems and Signal Processing 163: 108155. https://doi.org/10.1016/j.ymssp.2021.108155

30.

Zhou

Peng

Shao

, et al. (2025) Empirical fourier-bessel heuristic denoising and its application to gear fault diagnosis. ISA Transactions 167: 1906–1924. https://doi.org/10.1016/j.isatra.2025.10.003