Self-Supervised Intelligent Autoencoder with Multi-Scale Convolutional Contrastive Learning for Multivariate Anomaly Detection

Abstract

Autonomous anomaly detection in multivariate time-series is essential for ensuring safety in industrial and aerospace systems, yet is hindered by high-dimensional signals and limited fault samples. This paper proposes a self-supervised autoencoder that integrates multi-scale convolutions (kernel sizes 3/5/7) with a Transformer encoder augmented by T5-style relative position bias, enabling joint modeling of local transient patterns and long-range temporal dependencies. To enhance representation learning under scarce anomaly labels, we leverage SimCLR-based contrastive pretraining to refine normal-data feature distributions. To address fragmented detections, we design an event-level aggregation strategy that merges window-level reconstruction errors via Top-k peak averaging and rule-based filtering. Evaluated on the SMAP benchmark and our in-house industrial dataset, named CAFUC2, our method achieves event-level F1 of 0.97 and 0.83, respectively outperforming traditional baseline methods and demonstrating robust generalization to real-world flight anomalies.

Keywords

self-supervised learning multivariate time-series anomaly detection multi-scale convolution contrastive learning transformer

1 Introduction

1.1 Background

Time-series data are widely applied in industrial manufacturing, aerospace, and financial risk control, serving as a foundation for monitoring equipment status, operational performance, and business indicators (Chevtchenko et al., 2023; Elía & Pagola, 2024; Hilal et al., 2022). In the Industrial Internet of Things (IIoT), tens of thousands of sensors continuously generate large-scale, dynamic, and heterogeneous streams, where timely anomaly detection is critical for preventing equipment failures and network incidents (Chatterjee & Ahmed, 2022; Chevtchenko et al., 2023). In aerospace, multivariate telemetry sequences must be monitored for anomalies to predict potential failures and ensure mission safety (Fejjari et al., 2025). In financial markets, anomalies in transactions or indicators may signal fraud or market disturbances, requiring prompt detection for effective risk control (Hilal et al., 2022). Therefore, reliable anomaly detection in time-series data has become a key requirement across safety-critical domains.

Multivariate anomaly detection is more complex than univariate detection, as it involves high-dimensional, heterogeneous data with intricate correlations and temporal dependencies (Wang et al., 2025). Recent studies emphasize that two core challenges are capturing inter-variable dependencies and coping with the scarcity of labeled anomalies. Attention-based autoencoders and hybrid convolution–Transformer models have demonstrated the ability to model both local and global structures without reliance on labels (Liu et al., 2024; Vaswani et al., 2017). However, single-scale convolutions in existing hybrid models fail to capture heterogeneous anomalies (e.g., short-term spikes and long-term drifts in flight telemetry), while vanilla Transformers lack inductive bias for local pattern extraction. This motivates the fusion of multi-scale convolutions and position-aware self-attention to achieve complementary strengths in local and global modeling.

At the same time, self-supervised learning, particularly contrastive pretraining, provides a theoretical basis for mitigating class imbalance by enforcing representation consistency across augmented views of normal data (Chen et al., 2020). This enhances the ability to detect rare or unseen anomalies during deployment. SimCLR mitigates the limited availability of anomaly samples through two complementary mechanisms:

(1)
Contrastive invariance: by maximizing the consistency between augmented views (e.g., cropping, jittering) of normal data via Normalized Temperature-scaled Cross Entropy (NT-Xent) loss, the encoder learns compact, class-discriminative embeddings that reduce overfitting to normal-data noise;
(2)
Latent-space compactness: the pretrained encoder maps normal samples to dense clusters in latent space, such that rare anomalies—unseen during pretraining—tend to be separated due to larger embedding distances. This provides an unsupervised mechanism to alleviate the imbalance between normal and anomalous samples.

1.2 Our Contributions

To address the above challenges, this paper introduces a self-supervised anomaly detection framework, with the following contributions:

(1)
Core Contribution: Multi-Scale Convolutional Autoencoder. We design a multi-scale convolutional autoencoder (MSCA-Encoder) that fuzes 1D convolutions (kernel sizes 3/5/7) with Transformer self-attention augmented by T5-style relative position bias. Unlike existing hybrid models (e.g., Anomaly Transformer Xu et al., 2021, which highlights temporal association discrepancies, or vanilla Transformer AE lacking local inductive bias), this architecture achieves complementary strengths: multi-scale convolutions capture heterogeneous anomalies (short-term spikes from hard landings, long-term drifts from sensor degradation), while T5-style position bias enhances the Transformer's ability to model temporal order—enabling joint learning of local transient patterns and long-range dependencies.
(2)
Contrastive Pretraining for Scarce Anomalies. We propose an end-to-end integration of SimCLR-based pretraining with MSCA-Encoder. Specifically, the encoder is first pretrained on normal data via NT-Xent contrastive loss to learn compact and discriminative representations, and then the full autoencoder (encoder + symmetric decoder) is fine-tuned with reconstruction loss. This strategy mitigates the scarcity of labeled anomalies by reducing overfitting to normal-data noise.
(3)
Event-Level Aggregation for Coherent Detection. We develop a two-stage aggregation strategy to address granularity inconsistency in window-level anomaly scoring: (i) Window-level scoring: compute reconstruction errors for each sliding window and aggregate the top-3 error peaks to suppress isolated noise; (ii) Event-level aggregation: merge adjacent anomalous windows (gap ≤ window length L = 50) and filter short events (<50 time steps) via a 90% quantile rule. This strategy significantly reduces fragmented alarms compared to fixed-window thresholding methods (e.g., LSTM-NDT Hundman et al., 2018).
(4)
Extensive Validation on Benchmark and Industrial Data. We evaluate MSCA-AD on two datasets: (i) SMAP benchmark (NASA telemetry data)—achieving event-level F1 = 0.97, outperforming traditional baselines such as OmniAnomaly and LSTM-VAE; (ii) our in-house industrial dataset, named CAFUC2 (flight telemetry with four anomaly scenarios including Mixed Faults and Hard Landings)—achieving an average event-level F1 = 0.83, demonstrating robustness and adaptability to real-world aerospace anomalies.

2 Related Works

Recent advances in multivariate time-series anomaly detection have leveraged deep generative and attention-based architectures to capture both local patterns and global dependencies in complex, high-dimensional data.

Variational autoencoder–based methods. OmniAnomaly employs a stochastic recurrent neural network that integrates GRU cells with a variational autoencoder to learn probabilistic latent representations of normal time-series behavior, enabling robust anomaly scoring via reconstruction probability (Su et al., 2019). LSTM-VAE extends this line by combining recurrent encoders with variational latent variables to improve sequence reconstruction, though it is prone to underestimating rare, long-lasting anomalies (Park et al., 2018).

Probabilistic mixture–based approaches. DAGMM fuzes an autoencoder with Gaussian mixture modeling in the latent space, thereby combining feature compression with probabilistic density estimation. While effective for compact representations, it often struggles on high-dimensional multivariate streams with long-range dependencies (Zong et al., 2018). LSTM-NDT instead adopts a forecasting-based paradigm, where an LSTM network predicts future values and anomaly scores are derived from prediction residuals (Hundman et al., 2018). This method improves interpretability but may suffer from error accumulation over long sequences.

Adversarial and attention-based models. USAD amplifies the reconstruction error of anomalous instances via a dual autoencoder trained adversarially, but suffers from training stability issues (Audibert et al., 2020). Recently, Anomaly Transformer replaces the standard self-attention mechanism with a dual-branch anomaly attention mechanism that explicitly quantifies the discrepancy between prior association and sequence association, thereby revealing temporal deviations (Xu et al., 2021). While it achieves state-of-the-art point-wise accuracy, the lack of explicit event-level aggregation may lead to alarm fragmentation when anomalies span multiple subsequences; its performance relies on a specialized “association discrepancy” mechanism, and the complex attention discrepancy calculations result in high computational/memory costs in long-sequence or high-frequency sliding window scenarios. In contrast, Our method simultaneously captures local and long-range features through multi-scale convolution + self-attention with relative position bias, and combines Top-3 peak aggregation, event-level merging strategy, and SimCLR contrastive pre-training to improve event-level interpretability and engineering efficiency while reducing hyperparameter sensitivity and computational burden.

Other generative paradigms. Deep SVDD adapts one-class support vector data description to deep networks by learning hyperspherical decision boundaries around normal data (Ruff et al., 2018). Normalizing flow–based methods, such as AFNF, combine flow-based density estimation with separable attention modules to provide exact likelihood scores (Guan et al., 2023). GAN-based models (e.g., MAD-GAN) use adversarial sequence generation and discrimination to capture complex temporal correlations (Li et al., 2019). Self-supervised contrastive learning frameworks like SimCLR enhance representation quality by maximizing agreement between augmented views, improving robustness in anomaly detection (Chen et al., 2020).

Building on SimCLR-style contrastive pretraining, recent studies have expanded broader self-supervised paradigms for time-series representation learning. TimeGPT introduced a foundation model pretrained on millions of cross-domain time-series, demonstrating that large-scale temporal pretraining can generalize to forecasting and anomaly detection with minimal fine-tuning (Vishwas & Macharla, 2025). The TS2Vec families validate the effectiveness of hierarchical contrastive for universal sequence embeddings (Yue et al., 2022). FEAT proposed a feature-aware temporal encoder that jointly models inter-variable relations and temporal structure, boosting multivariate robustness under limited supervision (Kim et al., 2023). More recent diffusion- or generative-based self-supervised methods (e.g., Diffusion Auto-regressive Transformer Wang et al., 2024) further extend temporal pretraining into probabilistic generative regimes. Collectively, these works highlight a growing trend toward general-purpose, pretrain–finetune frameworks for time-series anomaly detection—aligning with MSCA-AD's design philosophy of leveraging self-supervised contrastive pretraining to enhance downstream reconstruction-based detection.

Taken together, the aforementioned approaches exhibit complementary strengths in capturing local irregularities, long-range dependencies, and probabilistic uncertainty. However, most methods emphasize either short-term or long-term patterns, or focus solely on window-level scoring. In contrast, our approach couples a multi-scale convolutional encoder with Transformer layers enhanced by relative position bias, and explicitly performs event-level aggregation to merge window-level evidence into coherent anomaly events. This combination improves sensitivity to both short transient spikes and long-term drifts, while reducing fragmented detections in practical deployments. A concise comparison of representative methods and their main characteristics is provided in Table 1.

Table 1.
Comparison with Representative Anomaly Detection Methods.

Method Feature Modeling Key Strengths / Limitations

OmniAnomaly (Su et al., 2019) GRU + VAE Captures uncertainty; weak for long-range dependencies.

LSTM-VAE (Park et al., 2018) LSTM encoder-decoder + VAE Models sequence structure; may miss rare long anomalies.

DAGMM (Zong et al., 2018) AE + Gaussian Mixture Combines compression with density modeling; less effective on high-dimensional streams.

LSTM-NDT (Hundman et al., 2018) LSTM forecasting Residual-based scoring; interpretable but suffers from error accumulation.

USAD (Audibert et al., 2020) Dual AE + adversarial Amplifies anomalies; unstable training.

Deep SVDD (Ruff et al., 2018) Hypersphere boundary One-class learning; limited temporal context.

Anomaly Transformer (Xu et al., 2021) Anomaly-Attention State-of-the-art accuracy; single-scale, fragmented alarms.

Ours Multi-scale + Transformer Captures short/long anomalies; reduces fragmented alarms; higher cost.

Method	Feature Modeling	Key Strengths / Limitations
OmniAnomaly (Su et al., 2019)	GRU + VAE	Captures uncertainty; weak for long-range dependencies.
LSTM-VAE (Park et al., 2018)	LSTM encoder-decoder + VAE	Models sequence structure; may miss rare long anomalies.
DAGMM (Zong et al., 2018)	AE + Gaussian Mixture	Combines compression with density modeling; less effective on high-dimensional streams.
LSTM-NDT (Hundman et al., 2018)	LSTM forecasting	Residual-based scoring; interpretable but suffers from error accumulation.
USAD (Audibert et al., 2020)	Dual AE + adversarial	Amplifies anomalies; unstable training.
Deep SVDD (Ruff et al., 2018)	Hypersphere boundary	One-class learning; limited temporal context.
Anomaly Transformer (Xu et al., 2021)	Anomaly-Attention	State-of-the-art accuracy; single-scale, fragmented alarms.
Ours	Multi-scale + Transformer	Captures short/long anomalies; reduces fragmented alarms; higher cost.

3 Proposed Method: MSCA-AD

3.1 Overall Structure

Figure 1 presents our three-stage anomaly-detection pipeline in a single glance. We begin by standardizing the raw multichannel SMAP signals with both global and channel-wise Z-score normalization, then segment them into overlapping sliding windows to form the training and test sets. In the training stage, three baseline autoencoders (ConvAE, TransformerAE and Conv + Attention AE) are trained under the same optimization and dynamic thresholding scheme. schedule with dynamic thresholding; the best performer is chosen and its encoder is replaced by our multi-scale Conv + Attention design, which is further refined via SimCLR self-supervised pretraining: each sliding window is augmented into two views, and the encoder learns view-invariant representations through the NT-Xent loss, producing compact, discriminative latent features. At inference time, we compute the reconstruction error for every window, apply a dynamic percentile threshold, average the top three error peaks, and merge adjacent detections using a 90% quantile rule to produce high-precision anomaly calls.

Figure 1.

Overall workflow of the proposed multivariate time-series anomaly detection method.

3.2 Data Preprocessing and Sliding Window Segmentation

The SMAP dataset consists of 54 multivariate sensor channels, covering both normal operations and abnormal events with annotated time intervals. The dataset is divided into training and test subsets.

Let $X \in R^{T}$ represent a univariate time series from a selected sensor channel. We apply Z-score normalization as follows:

\begin{aligned} X^{'} = \frac{X - μ}{σ} \end{aligned}

Where $μ$ and $σ$ denote the mean and standard deviation of the entire sequence, respectively.

In our experiments, Z-score normalization is essential to mitigate scale differences across sensors and to stabilize the training of neural networks. By centering each channel at zero with unit variance, we ensure that all features contribute comparably to the learning process and prevent channels with large absolute readings from dominating the gradient updates.

After normalization, we segment the sequence into overlapping windows using a sliding window approach. We set the window length to $L\; = 50$ and the stride to $s\; = 1$ , so that each window covers 50 consecutive time steps and we move one step at a time. The $t$ -th window is then defined as:

\begin{aligned} W_t = X^{'} [t : t + L - 1], \end{aligned}

\begin{aligned} t = 1, 1 + s, \dots, T - L + 1. \end{aligned}

This procedure produces a window set $W_{t}$ , with approximately $1.35 \times 10^{5}$ windows in the training set and $4.33 \times 10^{5}$ in the test set. Choosing $s = 1$ ensures fine-grained coverage of the time series, so that even very short anomalies or those spanning window boundaries are captured in at least one window, while $L = 50$ is large enough to encompass the typical duration of anomalous events in SMAP telemetry without excessive computational burden.

Each window $W_{t}$ thus captures a local temporal segment while the aggregation of all windows reflects the global dynamics across the entire sequence. These windows are fed into our autoencoder-based encoder for reconstruction, yielding $\hat{W_{t}}$ . We then compute the reconstruction error $| | W_{t} - \hat{W_{t}} | |$ as the anomaly scoring criterion: higher reconstruction errors indicate greater deviation from learned normal patterns. Figure 2 illustrates the sliding-window segmentation process with a step size of 1 and window size of 5.

Figure 2.

Sliding-window segmentation of the input sequence.

3.3 Multi-Scale Convolution and Self-Attention Encoder

Our encoder consists of two components: multi-scale convolution and Transformer self-attention. Given an input window $W \in R^{L \times C}$ , where $L$ is the sequence length and $C$ is the number of channels, we first apply 1D convolutions with multiple kernel sizes ${k_{i}}$ (e.g., 3, 5, 7) to extract features at different scales:

\begin{aligned} F_{i} = Conv1 D_{k_{i}} (W), i = 1, \dots, n \end{aligned}

Then, we concatenate the outputs along the channel dimension:

\begin{aligned} F_{conv} = [F_{1}; F_{2}; \dots; F_{n}] \in R^{L \times C^{'}} . \end{aligned}

Here, $F_{i}$ denotes the feature sequence extracted by a 1-D convolution with kernel size $k_{i} \in {3, 5, 7}$ . Small kernels $(k = 3)$ emphasize short-term, high-frequency patterns such as sudden spikes, whereas medium $(k = 5)$ and large $(k = 7)$ kernels progressively capture longer-range periodicities and slow-varying trends. In our experiments on SMAP telemetry, this multi-scale design proved essential, as anomalies often manifest both as brief transients and as gradual drifts, which cannot be reliably captured by a single receptive field.

To concatenate multi-scale features without temporal distortion, we apply symmetric zero-padding before convolution. Specifically, for kernel size $k_{i}$ , a padding size of $(k_{i} - 1) / 2$ (valid since $k_{i}$ is odd) ensures that the output length remains $L$ , consistent with the input window. This alignment avoids temporal misalignment across different receptive fields and guarantees a uniform feature dimension for subsequent self-attention encoding.

Next, the feature map $F_{conv}$ is passed into a multi-head self-attention module. For each head, we first compute queries, keys, and values as:

\begin{aligned} Q = W_{Q} F_{conv}, K = W_{K} F_{conv}, V = W_{V} F_{conv}, \end{aligned}

Where $W_{Q}, W_{K}, W_{V} \in R^{d_{mode} \times C^{'}}$ are learnable projection matrices.

To compute the attention score between position $\; i$ and $\; j$ , we adopt the T5-style relative position bias:

\begin{aligned} Score (i, j) = \frac{Q_{i} K_{j}^{⊤}}{\sqrt{d_{mode}}} + b_{bucket (i, j)}, \end{aligned}

where

b_{bucket (i, j)}

is a learnable bias selected according to the relative distance

| i - j |

(or its bucketed version). This bias directly augments the dot-product term

Q K^{⊤}

thereby injecting temporal order awareness into the attention mechanism.

Attention weights are then obtained by applying the softmax over the key dimension:

\begin{aligned} AttentionWeights = Softmax (Score), \end{aligned}

and the attention output is

\begin{aligned} F_{attn} = AttentionWeights \cdot V . \end{aligned}

While convolution provides strong inductive bias for local patterns, its receptive field is still limited. Self-attention complements this by building direct connections between any two time steps in the window, allowing the network to model long-range dependencies—crucial for detecting anomalies that depend on the broader context (e.g., a subtle drift only anomalous relative to remote history).

We fuze these two representations via a residual connection and layer normalization:

\begin{aligned} z = LayerNorm (F_{conv} + F_{attn}) \in R^{L \times d} . \end{aligned}

The residual shortcut ensures that convolutional features are preserved even if attention weights are small, and layer normalization stabilizes training by keeping the combined features on a consistent scale.

The decoder mirrors the encoder structure using a symmetric 1D convolutional network (or optionally an MLP), and produces a reconstruction $\hat{W} \in R^{L \times C}$ . We train the autoencoder to minimize the per-window reconstruction loss:

\begin{aligned} L = \frac{1}{L C} | | W - \hat{W} 1 | |_{2}^{2} . \end{aligned}

In our experiments, this combined conv–attention encoder achieved significantly lower reconstruction error on normal windows while amplifying errors on anomalous windows, leading to improved F1 over single-scale or purely convolutional/transformer baselines. Figure 3 illustrates the overall architecture of the proposed encoder.

Figure 3.

Overall architecture of the proposed encoder.

3.4 SimCLR-Based Self-Supervised Pretraining

To overcome the scarcity of labeled anomalies and learn robust representations from abundant unlabeled telemetry data, we pretrain the entire multi-scale convolution + Transformer encoder using a SimCLR-style contrastive objective. This integration ensures that both the convolutional layers (capturing local variations at different temporal resolutions) and the attention layers (capturing long-range dependencies) are jointly optimized to produce invariant and discriminative representations. In contrast to applying contrastive learning on a standalone encoder, our approach aligns the pretraining objective directly with the architecture used in downstream anomaly detection.

Augmentation Strategy. For each input window $W \in R^{L \times C}$ , we generate two stochastic views $\tilde{W_{i}}$ and $\tilde{W_{j}}$ via:

Random cropping: sample a contiguous sub-window of length $0.8 L$ and resize back to $L$ via linear interpolation;

Temporal jittering: add Gaussian noise $N (0, 0.01)$ to each time step;

Scaling: multiply the entire window by a random factor drawn from $[0.9, 1.1] .$

These augmentations preserve the core normal patterns while creating diverse positive pairs for contrastive learning.

Contrastive Loss. Let $h = Enc (W) \in R^{d}$ denote the representation of an input window W extracted by the multi-scale convolution + Transformer encoder. Convolutional filters emphasize short-term variations at multiple resolutions, while attention layers capture global temporal dependencies; both are optimized jointly during contrastive pretraining. To obtain a contrastive embedding, we employ a projection head $g (\cdot)$ implemented as a two-layer MLP with hidden dimensions $[128 \to 256 \to 128]$ , followed by L2 normalization. The latent vector is thus $z = g (h) \in R^{128}$ . The use of an MLP projection head, as suggested in (Chen et al., 2020), helps decouple the pretraining objective from the encoder space, thereby improving the quality and transferability of learned representations.

We adopt the NT-Xent loss (Chen et al., 2020) to enforce invariance across augmented views. For a positive pair $(i, j)$ in a batch of $N$ windows, the loss is defined as:

\begin{aligned} ℓ_{i, j} = - \log \frac{\exp (\sin (z_{i}, z_{j}) / τ)}{\sum_{2 N}^{k = 1} P_{| k \neq i |} \exp (\sin (z_{i}, z_{k}) / τ)} \end{aligned}

where

\sin (u, v) = u^{⊤} v / (∥ u ∥ ∥ v ∥)

is cosine similarity,

τ = 0.2

is the temperature hyperparameter, and

2 N

is the total number of augmented views. We use batch size

N = 256

to ensure a large set of negatives.

Training Protocol. Pretraining is performed for 100 epochs using the Adam optimizer (learning rate $1 \times 10^{- 3}$ , weight decay $1 \times 10^{- 4}$ ). Early stopping with patience 10 monitors the contrastive loss on a held-out validation split. After convergence, we discard the projection head $g (\cdot)$ and retain only the encoder weights $\; Enc (\cdot)$ .

Benefits and Ablation. Through this design, SimCLR aligns the convolutional and attention components of the encoder to produce perturbation-invariant yet discriminative features. During fine-tuning, these pretrained features provide a stable representation space where reconstruction errors concentrate around abnormal segments. As illustrated in Figure 4, two augmented views of each input window are encoded and projected.

Figure 4.

SimCLR-based self-supervised pretraining pipeline. Two augmented views of the same window are encoded and projected, and the NT-Xent loss pulls positive pairs together while pushing apart other samples.

3.5 Anomaly Detection Fine-Tuning and Top-

k

Scoring

During the SimCLR pretraining phase, the NT-Xent loss enables the encoder to learn feature representations that are “robust to perturbations in normal data and sensitive to anomalous patterns”. This not only provides high-quality initialized weights for fine-tuning but also offers dual support for accurate anomaly detection:

On one hand, SimCLR pretraining optimizes the starting point and process of fine-tuning through three core mechanisms:

(1)
Representation compactification: Treating different data augmentations of the same normal sample as positive pairs, the encoder is encouraged to map normal samples into denser clusters. This forms a normal sample manifold with clear boundaries in the reconstruction space, providing an explicit reference for subsequent anomaly discrimination.
(2)
Noise invariance: Through augmentation consistency constraints, interference from harmless perturbations (e.g., jitter, scaling) on the encoded representations is suppressed. This allows the reconstructor during fine-tuning to avoid fitting random noise in normal data and focus more on the reconstruction deviations of anomalous signals.
(3)
Accelerated convergence and overfitting resistance: Pretraining drives the encoder parameters to an initial state conducive to generalization, significantly reducing the magnitude of parameter updates during fine-tuning. This not only shortens the convergence cycle but also decreases the sensitivity to thresholds/hyperparameters on small-scale validation sets.

On the other hand, The pretrained multi-scale convolution + Transformer encoder provides feature representations that are both perturbation-invariant and temporally discriminative. During fine-tuning, convolutional layers emphasize localized deviations in telemetry signals, while Transformer attention highlights long-range contextual inconsistencies. This division of labor, already shaped by contrastive pretraining, enables the reconstruction loss to focus on structured anomalies rather than benign perturbations. As a result, errors become more concentrated on abnormal segments, forming a robust basis for subsequent Top-k aggregation.

After self-supervised pretraining, we fine-tune the encoder in an end-to-end autoencoder for anomaly detection. Specifically, we attach a symmetric decoder and optimize the per-window reconstruction loss:
$\begin{aligned} L_{rec} = \frac{1}{L C} \sum_{t = 1}^{L} \sum_{c = 1}^{C} (W_{t, c} - {\hat{W}}_{t, c})^{2}, \end{aligned}$
where $W \in R^{L \times C}$ is the input window, $\hat{W}$ is its reconstruction, $L \; \;$ is the window length, and $C$ is the number of channels. Fine-tuning is run for up to 50 epochs (Adam, lr $1 \times 10^{- 4}$ , weight decay $1 \times 10^{- 5}$ ), with early stopping on a held-out normal-only validation set (patience = 10).

At inference time, each window's anomaly score $e_{t}$ is the mean squared reconstruction error across channels:
$\begin{aligned} e_{t} = \frac{1}{C} \sum_{c = 1}^{C} (W_{t, c} - \hat{W_{t, c}})^{2} . \end{aligned}$

To reduce sensitivity to isolated spurious spikes, we aggregate the top- $k$ errors within each window sequence. Let ${e_{(1)}, e_{(2)}, \dots, e_{(k)}}$ be the $k$ largest values among all $e_{t}$ in that window; The final window-level score $s$ is
$\begin{aligned} s = \frac{1}{k} \sum_{i = 1}^{k} e_{(i)} . \end{aligned}$

This Top-k strategy naturally complements SimCLR pretraining: by averaging only the largest errors, it emphasizes persistent deviations that the pretrained encoder cannot reconcile with normal patterns, while suppressing transient noise that contrastive training has already rendered invariant. Hence, the joint effect is a scoring mechanism that highlights true anomalies without being distracted by minor perturbations. A dynamic threshold $τ$ is chosen by scanning on the validation set to maximize the event-level F1. The fine-tuning and Top-k scoring process is shown in Figure 5.

Figure 5.
Fine-tuning and Top-k scoring pipeline. The pretrained encoder is coupled with a decoder, reconstruction errors are computed, and the Top-k peak averaging produces a robust anomaly scores.
3.6 Event-Level Postprocessing and Evaluation

To transform window-level scores into coherent anomaly events, we apply a simple gap-based merging and length filtering:

Thresholding: Mark window $t$ as anomalous if $s_{t} \geq τ .$

Gap-based merging: Consecutive anomalous windows whose start-to-start distance $\leq L$ are merged into one event.

Length filtering: Discard any merged event shorter than L time steps to suppress isolated noise.

For evaluation, we adopt the overlap-based criterion of Hundman et al. (2018). Each predicted event is compared to ground-truth anomaly intervals: if it overlaps any true interval, it counts as a True Positive (TP); otherwise, as a False Positive (FP). Any ground-truth interval not overlapped by any prediction is a False Negative (FN).

Precision, Recall, and F1 are then computed at the event level:

\begin{matrix} Precision = \frac{TP}{TP + FP}, \\ Recall = \frac{TP}{TP + FN}, \\ F 1 = \frac{2 P r e c i s i o n \cdot R e c a l l}{P r e c i s i o n + R e c a l l} \end{matrix}

This evaluation better reflects real-world requirements by rewarding the detection of contiguous anomalysegments and penalizing fragmented or spurious alerts.

4 Experimental Results and Discussion

This section presents the detailed implementation of the proposed unsupervised anomaly detection framework based on a multi-scale convoluteional and self-attention autoencoder, along with the experimental settings and baseline comparisons.

4.1 Experimental Setup

4.1.1 Datasets and Preprocessing

We evaluated our method on two datasets: the SMAP dataset and our internal CAFUC2 industrial dataset.

For SMAP, we normalized the 25 multivariate telemetry channels using Z-score normalization and then partitioned them into overlapping windows of length $L = 50$ and step size $s = 1$ . This resulted in 135,216 training windows and 432,839 test windows, of which 40,419 were labeled as anomalous windows (Table 2).

Table 2.
Sliding-Window Statistics of the SMAP Dataset.

Subset Total Windows Anomalous Windows Channels

Training 135,216 18,305 25

Testing 432,839 40,419 25

Subset	Total Windows	Anomalous Windows	Channels
Training	135,216	18,305	25
Testing	432,839	40,419	25

The CAFUC2 dataset was collected from 10 civil aviation training aircraft during actual flight missions. It contains over 50 heterogeneous telemetry channels, including positional data (latitude, longitude, altitude), attitude information (pitch, roll, yaw angles), and engine performance metrics (RPM, fuel flow, exhaust gas temperature). The signals are sampled at 1 Hz.

Implementation and hyperparameters are summarized in Tables 3 and 4. Table 3 details our hardware platform and model architecture. Table 4 reports the settings for SimCLR pretraining, downstream fine-tuning, and the Top-3 scoring and thresholding strategies (including window and event-level thresholds).

Table 3.

Hardware and Model Architecture.

Component	Specification
Environment	Python 3.8; PyTorch 1.11; CUDA 11.3; NVIDIA T4 GPU.
Encoder	Multi-scale conv: three parallel 1D Convs (kernel sizes 3, 5, 7; in = 25, out = 32 each), concatenated (96 ch) → 1 × 1 Conv → 128 ch + ReLU.
Transformer	3 × TransformerEncoderLayer (d_model = 128, heads = 8, dropout = 0.1, batch_first = True) augmented with T5-style relative positional bias.
Decoder	Two 1D transposed Convs (128→96→25) to reconstruct the input.

Table 4.

Training Stages, Settings, and Thresholding.

Stage	Settings
SimCLR Pretraining	Augmentations: crop (80% length), jitter, Gaussian noise (σ = 0.01); Projection MLP (128→256→128), NT-Xent (τ = 0.2), batch 256, 20 epochs, cosine LR.
Fine-tuning	Encoder + decoder, batch 64, lr 1e-4, up to 100 epochs, MSE loss, early stopping (patience 10).
Top-3 Scoring & Thresholding	Window score = mean(Top 3 channel errors); τ_win = 95th percentile; Event gap g = 50; discard events <50; τ_evt = 90th percentile.

4.1.2 Computational Complexity Analysis

To quantify the computational cost of MSCA-AD, we analyze both its training and inference complexity. The multi-scale convolutional encoder employs three parallel 1-D convolution branches (kernel sizes 3/5/7, input channels = 25, output = 32 each), resulting in approximately O(3 × L × C_in × C_out) ≈ 1.2 × 10⁶ multiply-accumulate (MAC) operations per window of length L = 50.

The Transformer component consists of three encoder layers (hidden dimension = 128, 8 heads), each contributing roughly O(L² × d_model) ≈ 3.2 × 10⁵ MACs per layer.

Overall, the full MSCA-AD forward pass requires approximately 4.2 × 10⁶ FLOPs per window—comparable to a 3-layer ConvAE, and significantly lighter than the Anomaly Transformer (≈8.7 × 10⁶ FLOPs).

On an NVIDIA T4 GPU, SimCLR pretraining for 20 epochs over 135k SMAP windows required approximately 2.8 GPU-hours, while fine-tuning took about 2.5 GPU-hours, leading to a total training cost of ≈5 GPU-hours. During inference, the average per-window latency was 1.8 ms.

4.2 Module Ablation Study

Starting from the Conv + Attention AE baseline (F1 = 0.37), we incrementally add each component to quantify its contribution (Table 5).

Table 5.
Event-Level F1 of Model Variants. Relative Gain Over Conv + Attention AE Baseline.

Configuration F1 Rel. Gain

Conv + Attention AE (baseline) 0.37 0%

+ Multi-Scale Conv 0.42 +13.5%

+ RelPos Bias (T5 style) 0.48 +29.7%

+ Top-3 Peak Aggregation 0.83 +124%

+ Masked-AE Pretraining 0.62 +67.6%

+ SimCLR Contrastive Pretraining 0.97 +162%

Configuration	F1	Rel. Gain
Conv + Attention AE (baseline)	0.37	0%
+ Multi-Scale Conv	0.42	+13.5%
+ RelPos Bias (T5 style)	0.48	+29.7%
+ Top-3 Peak Aggregation	0.83	+124%
+ Masked-AE Pretraining	0.62	+67.6%
+ SimCLR Contrastive Pretraining	0.97	+162%

First, integrating multi-scale convolutions with 3/5/7 kernels improves the F1 to 0.42 (+13.5%). Next, incorporating the T5-style relative positional bias into the attention score computation further raises performance to 0.48 (+29.7%).

These results indicate that while multi-scale convolution captures complementary temporal patterns, relative position bias provides additional temporal order information that enhances anomaly discrimination.

Applying Top-3 Peak Aggregation (mean of Top-3 window errors), gap merging $(g \leq 5)$ and minimum-length filtering boosts F1 substantially to 0.83 (+124%). On the pretraining side, Masked-AE with Huber loss improves F1 to 0.62 (+67.6%), while SimCLR contrastive pretraining (NT-Xent loss) achieves the best result, F1 = 0.97 (+162%).

4.3 Hyperparameter Sensitivity

Top-k Aggregation. We vary $k \in {1, 3, 5, 7, 8}$ on SMAP (Figure 6 and Table 6). $k = 3$ yields the highest event-level F1 (0.97). Lower $k$ is noise-sensitive; higher $k$ dilutes true peaks.

Figure 6.

F1 sensitivity to kernel-set and channel count: single, two, three scales.

Table 6.

Top-k Sensitivity on SMAP ( $L = 50$ , 95% Threshold).

k	Precision	Recall	F1
1	0.72	0.77	0.74
3	0.94	0.99	0.97
5	0.75	0.78	0.76
7	0.78	0.80	0.79
8	0.82	0.78	0.80

Sliding-Window Length and Stride Sensitivity. We conduct ablation studies on the sliding-window length $L$ and stride $s$ to evaluate their impact on anomaly detection performance. For $L$ , we test $L \in {20, 30, 50, 70, 90, 110, 130}$ with Top-3 aggregation and 90% event merging (Table 7). The results show that short windows $(L < 50)$ fail to capture long-term drifts (e.g., F1 = 0.88 for $L = 30$ ), while longer windows $(L > 50)$ increase computational cost without improving performance (F1 = 0.92 for $L = 110$ ). Therefore, $L = 50$ provides the best trade-off between detecting brief spikes and longer anomalies, achieving F1 = 0.97.

Table 7.

Impact of Window Length $L$ on Detection Performance (Top-3 Aggregation, 90% Event Merge).

L	Precision	Recall	F1
20	0.88	0.91	0.88
30	0.91	0.94	0.93
50	0.94	0.99	0.97
70	0.93	0.98	0.95
90	0.92	0.97	0.94
110	0.90	0.95	0.92
130	0.90	0.94	0.92

We also analyze the effect of stride s on detection and computational cost (Table 8). Smaller strides provide finer-grained coverage, capturing short anomalies that span window boundaries, while larger strides reduce computation at the expense of recall. Based on this study, $s = 1$ is chosen for a balance between detection precision and computational efficiency in aerospace applications.

Table 8.

Stride Sensitivity Analysis on SMAP Telemetry.

Stride (s)	Precision	Recall	F1	Computational Cost (s)
1	0.94	0.99	0.97	120
2	0.93	0.95	0.94	65
3	0.91	0.92	0.91	40

4.4 Combined Kernel-Set and Channel-Number Analysis

To study convolutional design choices, Figure 6 plots event-level F1 for varying channel counts (8, 16, 32, 64, 128) across single-, two-, and three-scale kernel sets. Three-scale ${3, 5, 7}$ at 32 channels per branch achieves the best trade-off; single- and two-scale variants plateau or overfit at higher widths.

4.5 Noise and Missing-Data Robustness

To evaluate robustness, we conducted controlled experiments under two perturbation settings: (1) Gaussian noise injection with standard deviations $σ \in {0.01, 0.05, 0.1}$ applied independently to each channel, and (2) random channel dropouts with rates $r \in {10 \%, 20 \%, 30 \%}$ . Performance was averaged over five runs with different random seeds to mitigate stochastic variance. Tables 9 and 10 present the detailed robustness evaluation results of MSAD-AD on the SMAP dataset under the two perturbation setting.

These results suggest that MSCA-AD maintains stable event-level detection accuracy up to moderate noise ( $σ \leq 0.05$ ) and dropout rates ( $r \leq 20 \%$ ). This robustness likely arises from the combined effects of Z-score normalization, multi-scale convolution, attention-based weighting, and Top-3 peak averaging, which collectively mitigate the impact of corrupted or missing channels. While further ablation is needed to isolate individual contributions, the findings indicate strong resilience of the proposed approach to real-world sensor imperfections.

4.6 Event-Level Threshold Sensitivity

We vary the event-merge percentile $p_{e} \in {50 \%, 70 \%, 90 \%, 95 \%, 99 \%}$ and plot the resulting event-level F1 in Figure 7. F1 peaks at $p_{e} = 90 \%\; (0.97$ ) and remains above 0.94 across $70 \% \leq p_{e} \leq 95 \%$ .

Figure 7.

Sensitivity of Event-Merge Percentile $p_{e}$ : Event-Level F1 vs. $p_{e}$ .

At lower $p_{e}$ , recall slightly increases at the cost of precision due to more permissive merging. At higher $p_{e}$ , precision improves but recall drops as true events may split. The plateau around $90 \%\; \;$ confirms our merge rule is both optimal and forgiving, allowing operators to adjust $p_{e}$ to favor precision or recall without dramatic performance degradation.

4.7 Evaluation Metrics and Comparison Methods

We assess event-level performance using Precision, Recall, and F1. Table 11 compares our best configuration (Multi-Scale + SimCLR + Top-3) against representative baselines from the literature. Baseline results are taken from the original publications and not re-implemented.

Table 9.
Noise Robustness of MSCA-AD Under Gaussian Perturbations on the SMAP Dataset.

Noise σ Precision Recall F1

0.01 0.91 0.90 0.91

0.05 0.89 0.88 0.89

0.10 0.84 0.82 0.83

Noise σ	Precision	Recall	F1
0.01	0.91	0.90	0.91
0.05	0.89	0.88	0.89
0.10	0.84	0.82	0.83

Table 10.

Missing-Data Robustness of MSCA-AD Under Random Channel Dropout on the SMAP Dataset.

Dropout Rate	Precision	Recall	F1
10%	0.91	0.89	0.90
20%	0.88	0.87	0.88
30%	0.83	0.82	0.83

Table 11.

Event-Level Detection Performance Comparison.

Method	Precision	Recall	F1
Ours	0.94	0.99	0.97
OmniAnomaly (Su et al., 2019)	0.74	0.97	0.84
LSTM-VAE (Park et al., 2018)	0.86	0.74	0.73
DAGMM (Zong et al., 2018)	0.58	0.91	0.71
LSTM-NDT (Hundman et al., 2018)	0.90	0.88	0.89
Anomaly Transformer (Xu et al., 2021)	0.94	0.99	0.97

Compared to OmniAnomaly (Su et al., 2019), which uses a stochastic recurrent VAE to model latent distributions, our multi-scale convolutions yield richer local features and SimCLR pretraining produces more discriminative embeddings—resulting in +20 pp precision and +13 pp F1 improvement. Against LSTM-VAE (Park et al., 2018), our self-attention encoder captures long-range dependencies missed by pure recurrence, boosting recall by 25 pp. DAGMM (Zong et al., 2018) jointly models clustering and reconstruction but lacks explicit temporal context mechanisms; our Top-3 peak averaging captures persistent error signals, improving F1 by 26 pp. Finally, LSTM-NDT (Hundman et al., 2018) applies nonparametric thresholding on LSTM forecasts—while balanced, it achieves F1 = 0.89 versus our 0.97, demonstrating the strength of combining self-supervised pretraining with robust aggregation. Anomaly Transformer (Xu et al., 2021) also reports F1 = 0.97, but relies on a specialized discrepancy-based attention mechanism, whereas our approach attains the same accuracy through a simpler multi-scale and contrastive learning design that is easier to integrate and interpret.

These results confirm that integrating multi-scale feature extraction, contrastive self-supervision, and peak-based event aggregation is essential for state-of-the-art unsupervised multivariate time-series anomaly detection.

4.8 Verification of Four Flight Anomaly Scenarios

To evaluate MSCA-AD's generalizability, we tested it on four representative anomaly scenarios from the CAFUC2 dataset, each comprising a normal flight segment followed by a labeled anomaly: (1) Mixed Faults, (2) Hard Landing, (3) Aggressive Pitch Maneuver, and (4) Dolphin Jump. Figure 8 visualizes the 3D trajectories with detected anomalies highlighted in red, and Tables 12 –15 reports the corresponding event-level Precision, Recall, and F1.

Figure 8.

3D Flight Trajectories for Four Anomaly Scenarios. Detected anomalies are indicated by the non-smooth trajectory segments: (a) Mixed Faults, (b) Hard Landing, (c) Aggressive Pitch Maneuver, and (d) Dolphin Jump.

Table 12.

Mixed Faults: Performance Comparison.

Method	Precision	Recall	F1
OmniAnomaly	0.70	0.68	0.69
LSTM-VAE	0.68	0.66	0.67
DAGMM	0.60	0.58	0.59
Anomaly Transformer	0.74	0.72	0.73
MSCA-AD	0.82	0.80	0.81

Table 13.

Hard Landing: Performance Comparison.

Method	Precision	Recall	F1
OmniAnomaly	0.73	0.70	0.71
LSTM-VAE	0.70	0.68	0.69
DAGMM	0.62	0.61	0.61
Anomaly Transformer	0.77	0.75	0.76
MSCA-AD	0.87	0.83	0.85

Table 14.

Aggressive Pitch Maneuver: Performance Comparison.

Method	Precision	Recall	F1
OmniAnomaly	0.68	0.66	0.67
LSTM-VAE	0.66	0.64	0.65
DAGMM	0.58	0.56	0.57
Anomaly Transformer	0.72	0.70	0.71
MSCA-AD	0.78	0.82	0.80

Table 15.

Dolphin Jump: Performance Comparison.

Method	Precision	Recall	F1
OmniAnomaly	0.72	0.69	0.70
LSTM-VAE	0.69	0.67	0.68
DAGMM	0.60	0.59	0.59
Anomaly Transformer	0.75	0.73	0.74
MSCA-AD	0.88	0.82	0.85

We evaluate MSCA-AD against representative baselines on four flight anomaly scenarios from the CAFUC2 dataset. Each table presents Precision, Recall, F1, and AUROC for the corresponding scenario.

Across all four scenarios, MSCA-AD consistently attains the highest F1 and AUROC, demonstrating improved robustness in anomaly localization and classification. Although the Anomaly Transformer also performs strongly, our method's integration of multi-scale convolution, T5-style relative positional bias, and SimCLR-based pretraining enables more effective modeling of both short transients and long-term drifts, thereby reducing fragmented alarms in dynamic flight trajectories.

Across these scenarios, MSCA-AD achieves an average F1 of 0.83, consistent with our controlled-dataset experiments. Notably: Mixed Faults: Balanced precision and recall (0.82/0.80) indicate the model's ability to disentangle multiple simultaneous anomalies. Hard Landing: Highest F1 (0.85) reflects the efficacy of multi-scale convolutional filters in capturing the abrupt, high-magnitude shock signatures. Aggressive Pitch Maneuver: Slightly lower precision (0.78) but strong recall (0.82) suggest that Top-3 aggregation effectively identifies brief, high-amplitude deviations while maintaining false-alarm control. Dolphin Jump: Replicates high F1 (0.85), demonstrating that self-attention with relative positional bias and contrastive pretraining generalize to detect cyclic vertical oscillations.

The narrow F1 variance (0.80–0.85) across diverse anomaly types underscores MSCA-AD's robustness: its combination of local feature extraction, global context modeling, and event-level aggregation reliably identifies heterogeneous flight anomalies without scenario-specific tuning.

5 Conclusion and Future Work

We presented MSCA-AD, an intelligent anomaly-detection framework for multivariate telemetry that unifies multi-scale convolution,self-attention with relative positional bias, Top-k peak aggregation,and self-supervised pretraining. On the NASA SMAP benchmark,MSCA-AD's event-level F1 steadily improves — from 0.37 (baseline autoencoders)to 0.48 (multi-scale + RelPos), 0.83 (Top-3 aggregation), 0.62 (masked-AE),and ultimately 0.97 (SimCLR) — demonstrating high detection accuracy and low false-alarm rates.

By fuzing local and global features and leveraging contrastive self-supervision, MSCA-AD realizes intelligent decision-making under uncertainty. Its rule-based event aggregation effectively consolidates anomaly windows into coherent events, enhancing both precision and recall.

However, the multi-stage training pipeline entails significant computational cost, and the framework's robustness to extreme noise, heterogeneous sensor modalities, and hyperparameter choices (e.g., mask ratio, k) requires further validation.

Future research will explore:

Integration with Time-Series Foundation Models: Recent large-scale pretrained models such as TimeGPT and Time-Series Transformer (TST) provide generalizable temporal embeddings learned from millions of sequences. Integrating such pretrained backbones with the MSCA-AD framework offers two potential advantages: (1) replacing the Transformer encoder in our model with a frozen or partially fine-tuned TimeGPT/TST module could further improve representation quality under scarce data regimes; and (2) the contrastive pretraining stage could be reformulated as domain-adaptive fine-tuning of pretrained temporal embeddings, reducing pretraining cost while retaining cross-domain generalization. We plan to explore this hybrid direction in future work, focusing on pretrained temporal representation reuse combined with our multi-scale convolutional fusion and Top-3 aggregation mechanisms.

Fuzzy-Logic Integration: Inspired by human perception of uncertainty in telemetry interpretation, a promising direction is to incorporate fuzzy logic into the anomaly aggregation process. One feasible approach is fuzzy-weighted Top-3 fusion, where each reconstruction-error peak is assigned a membership degree: $μ_{i} = \exp (- (| e_{i} - \bar{(e)} |) / σ)$ reflecting its confidence. The final anomaly score is then computed as a fuzzy average: $(\sum_{i} μ_{i} e_{i})/(\sum_{i} μ_{i})$ allowing smooth transitions between normal and abnormal states rather than hard thresholds. Another extension is embedding fuzzy membership within the reconstruction-error distribution itself, yielding a fuzzy autoencoder that explicitly models gradual degradation rather than binary faults. Such mechanisms could enhance interpretability and robustness in noisy aerospace telemetry environments.

Lightweight, online deployment: Although this work prioritizes performance during the research phase, MSCA-AD possesses significant potential for engineering implementation. Firstly, the multi-scale convolutional computation mode facilitates pre-implementation of the structural sparsity of channels or kernels, architecture pruning, and low-rank factorization at the inference end, naturally accommodating model compression techniques (e.g., quantization, pruning). Secondly, SimCLR enables the encoder to learn more robust general temporal features, providing feasibility for an edge deployment strategy of “frozen encoder + lightweight decoder design,” thereby reducing inference latency by several times while maintaining detection performance. Furthermore, future development could combine incremental learning or online threshold adaptation mechanisms to achieve continuous learning and real-time alarm updates on resource-constrained devices.

Footnotes

ORCID iDs

Jing Lu

Yali Fang

Songhan Fan

Xuan Wu

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the General Aviation Safety Operation Intelligent Research Institute, Digital Operation Safety Solution and Demonstration for China’s Civil Aviation Pilot Training System, Digital Twin of Flight Scenarios for Accident Investigation, Integrated Digital Operation System for Civil Aviation Pilot Training, (grant number TD2025DZ06, ASSA2024/30, FZ2022ZX55, 24CAFUC08003).

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Audibert

Michiardi

Guyard

Marti

Zuluaga

M. A.

(2020). Usad: Unsupervised anomaly detection on multivariate time series. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 3395–3404).

Chatterjee

Ahmed

B. S.

(2022). Iot anomaly detection methods and applications: A survey. Internet of Things, 19, 100568. https://doi.org/10.1016/j.iot.2022.100568

Chen

Kornblith

Norouzi

Hinton

(2020). A simple framework for contrastive learning of visual representations. In Proceedings of the International Conference on Machine Learning (ICML) (pp. 1597–1607).

Chevtchenko

Rocha

E. D. S.

Santos

M. C. M. D.

Mota

R. L.

Vieira

D. M.

De Andrade

E. C.

De Araújo

D. R. B.

(2023). Anomaly detection in industrial machinery using IoT devices and machine learning: A systematic mapping. IEEE Access, 11, 128288–128305. https://doi.org/10.1109/ACCESS.2023.3333242

Elía

Pagola

(2024). Anomaly detection in smart-manufacturing era: a review. Available at SSRN 4815859.

Fejjari

Delavault

Camilleri

Valentino

(2025). A review of anomaly detection in spacecraft telemetry data. Applied Sciences, 15(10), 5653. https://doi.org/10.3390/app15105653

Guan

Gao

(2023). Conditional normalizing flow for multivariate time series anomaly detection. ISA Transactions, 143, 231–243. https://doi.org/10.1016/j.isatra.2023.09.002

Hilal

Gadsden

S. A.

Yawney

(2022). Financial fraud: A review of anomaly detection techniques and recent advances. Expert Systems With Applications, 193, 116429. https://doi.org/10.1016/j.eswa.2021.116429

Hundman

Constantinou

Laporte

Colwell

Soderstrom

(2018). Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD) (pp. 387–395).

10.

Kim

Chung

Kang

(2023). FEAT: A general framework for feature-aware multivariate time-series representation learning. Knowledge-Based Systems, 277, 110790. https://doi.org/10.1016/j.knosys.2023.110790

11.

Chen

Jin

Shi

Goh

S.-K.

(2019). MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks. In Proceedings of the International Conference on Artificial Neural Networks (ICANN) (pp. 703–716).

12.

Liu

Yan

Wang

Liu

Tang

(2024). Unsupervised deep anomaly detection for industrial multivariate time series data. Applied Sciences, 14(2), 774. https://doi.org/10.3390/app14020774

13.

Park

Hoshi

Kemp

C. C.

(2018). A multimodal anomaly detector for robot-assisted feeding using an LSTM-based variational autoencoder. IEEE Robotics and Automation Letters, 3(3), 1544–1551. https://doi.org/10.1109/LRA.2018.2801475

14.

Ruff

Vandermeulen

Goernitz

Deecke

Siddiqui

S. A.

Binder

Müller

Kloft

(2018). Deep one-class classification. In Proceedings of the International Conference on Machine Learning (ICML) (pp. 4393–4402).

15.

Zhao

Niu

Liu

Sun

Pei

(2019). Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 2828–2837).

16.

Vaswani

Shazeer

Parmar

Uszkoreit

Jones

Gomez

A. N.

Kaiser

Polosukhin

(2017). Attention is all you need. Advances in Neural Information Processing Systems, 30, 5998–6008. https://doi.org/10.48550/arXiv.1706.03762

17.

Vishwas

B. V. K.

Macharla

(2025). TimeGPT: The first foundation model for time series. In Time series forecasting using generative AI: Leveraging AI for precision forecasting (pp. 169–182). Springer.

18.

Wang

Cheng

Liu

Chen

(2024). Diffusion auto-regressive transformer for effective self-supervised time series forecasting.

19.

Wang

Jiang

Zhang

Wei

Xie

Pang

(2025). A survey of deep anomaly detection in multivariate time series: Taxonomy, applications, and directions. Sensors (Basel, Switzerland), 25(1), 190. https://doi.org/10.3390/s25010190

20.

Wang

Long

(2021). Anomaly transformer: Time series anomaly detection with association discrepancy. arXiv preprint arXiv:2110.02642.

21.

Yue

Wang

Duan

Yang

Huang

Tong

(2022). TS2Vec: Towards universal representation of time series. Proceedings of the AAAI Conference on Artificial Intelligence, 36(8), 8980–8987. https://doi.org/10.1609/aaai.v36i8.20881

22.

Zong

Song

Min

M. R.

Cheng

Lumezanu

Cho

Chen

(2018). Deep autoencoding Gaussian mixture model for unsupervised anomaly detection. Proceedings of the International Conference on Learning Representations (ICLR).

Self-Supervised Intelligent Autoencoder with Multi-Scale Convolutional Contrastive Learning for Multivariate Anomaly Detection

Abstract

Keywords

1 Introduction

1.1 Background

3.1 Overall Structure

4 Experimental Results and Discussion

4.1 Experimental Setup

4.1.1 Datasets and Preprocessing

Table 2. Sliding-Window Statistics of the SMAP Dataset. Subset Total Windows Anomalous Windows Channels Training 135,216 18,305 25 Testing 432,839 40,419 25

4.2 Module Ablation Study

4.5 Noise and Missing-Data Robustness

4.6 Event-Level Threshold Sensitivity

Table 9. Noise Robustness of MSCA-AD Under Gaussian Perturbations on the SMAP Dataset. Noise σ Precision Recall F1 0.01 0.91 0.90 0.91 0.05 0.89 0.88 0.89 0.10 0.84 0.82 0.83

Footnotes

ORCID iDs

Funding

Declaration of Conflicting Interests

References

Table 2.
Sliding-Window Statistics of the SMAP Dataset.

Subset Total Windows Anomalous Windows Channels

Training 135,216 18,305 25

Testing 432,839 40,419 25

Table 9.
Noise Robustness of MSCA-AD Under Gaussian Perturbations on the SMAP Dataset.

Noise σ Precision Recall F1

0.01 0.91 0.90 0.91

0.05 0.89 0.88 0.89

0.10 0.84 0.82 0.83