Cross-Subject Event-Related Potential Classification via Multi-View Based Contrastive Learning

ERP signal analysis

Extensive research on ERP detection has primarily focused on developing effective recognition algorithms (Lotte et al., 2007). Traditional approaches, such as support vector machines (Müller et al., 2018) and linear discriminant analysis (Lotte et al., 2007; Krusienski et al., 2008), have been widely applied in ERP analysis. These methods classify samples by constructing high-dimensional feature spaces, showing particularly strong performance on small datasets. However, as dataset size increases and user diversity grows, these traditional methods often suffer from performance degradation, especially in scenarios characterized by pronounced individual differences. To overcome these limitations, researchers have developed advanced algorithms. For instance, xDAWN-based methods enhance ERP features to improve the signal-to-noise ratio, thereby increasing classification accuracy (Barachant et al., 2012; Rivet et al., 2009). In addition, Riemannian geometry–based ERP classifiers exploit geometric representations to capture the intrinsic structure of the data, demonstrating promising results on complex datasets (Barachant et al., 2012; Congedo et al., 2017).

With the rapid advancement of deep learning, significant progress has been achieved in pattern recognition and feature engineering (LeCun et al., 2015). Deep neural networks, particularly convolutional neural networks, have been extensively used to extract temporal and spatial information from EEG signals, yielding remarkable results (Craik et al., 2019; Roy et al., 2019). Classical architectures such as Deep ConvNets employ deep spatiotemporal convolutional layers for EEG feature extraction (Schirrmeister et al., 2017). Building on this foundation, EEGNet introduced separable convolutions to design a compact yet efficient network, which has achieved excellent results across multiple experiments and has become a benchmark model in EEG processing (Lawhern et al., 2018). Subsequently, EEG-TCNet reduced the number of trainable parameters, enabling greater efficiency on resource-constrained devices (Ingolfsson et al., 2020). Eduardo et al. further proposed EEG-Inception, which integrates the Inception module from computer vision and achieved significant improvements in ERP recognition (Santamaría-Vázquez et al., 2020).

Despite these advances, unresolved challenges remain in ERP research, which continue to limit the performance of recognition tasks using such data. We focus on two key challenges: 1.

Compared with conventional EEG signals, ERP signals offer a higher signal-to-noise ratio but a substantially shorter effective duration, which requires efficient extraction of discriminative features within a limited temporal window. However, existing methods have not sufficiently leveraged the rich phase information, specific waveforms, and other attributes embedded in ERP signals.

Multi-view learning

Multi-view learning (MVL) aims to exploit complementary information provided by different views of the same phenomenon. In MVL, data are typically represented as multiple views that integrate different modalities or feature sets (Zhang et al., 2020a). MVL was initially applied to multimedia data, which commonly originate from heterogeneous sources such as text, video, and audio or are characterized by multiple representations (e.g., time-domain and frequency-domain features) (Zhang et al., 2020b; Li et al., 2022; Tang et al., 2020). Existing studies indicate that fusing diverse views to leverage complementary information can improve model accuracy (Tao et al., 2020; Tang et al., 2023). This benefit has been demonstrated across tasks. For example, in text classification, the Multimodal Bitransformer model fuses textual and visual views to leverage image features associated with accompanying text (Kiela et al., 2020). In the image domain, Tian et al. proposed a multi-view contrastive learning framework called Contrastive Multiview Coding, which learns compact representations for raw images and optical flow to maximize the mutual information between different views of the same scene (Tian et al., 2020).

Recently, several studies have extended MVL to EEG signal analysis. Ye et al. constructed temporal and spectral views of EEG signals and exploited their complementarity to mine additional positive pairs, thereby improving contrastive learning performance (Ye et al., 2022). An et al. proposed an amplitude-temporal dual-view fusion method for temporal-feature learning in automatic sleep staging (An et al., 2024a). Zhao et al. investigated the relationship between temporal and time–frequency views and developed a self-supervised multi-view representation learning framework for sleep-stage classification (Zhao et al., 2024). However, existing multi-view approaches in the EEG domain have not been extended to ERP recognition. A robust view-generation and fusion strategy tailored to ERP signals is still lacking.

Domain generalization

Owing to the pronounced inter-subject variability in EEG signals, numerous studies have applied domain generalization (DG) approaches to cross-subject classification tasks. These approaches aim to learn domain-invariant representations from source domains and exploit neural networks to capture shared inter-subject features, thereby improving classification performance (Santamaría-Vázquez et al., 2019). Within existing DG research, Pan et al. employed prototype learning to extract domain-level feature prototypes, which were then used to predict outcomes for new subjects, achieving promising results in zero-shot learning (Pan et al., 2023; An et al., 2024b). Domain-adversarial neural networks (DANNs), which learn domain-invariant representations through adversarial training between a discriminator and a generator, have also demonstrated substantial utility in DG tasks. Li et al. introduced the bi-hemisphere DANN, which incorporates one global and two local domain discriminators to extract emotion-discriminative features specific to each hemisphere (Li et al., 2021b). More recently, contrastive learning has been applied to DG tasks. Shen et al. developed a CNN-based contrastive learning framework that outperformed existing DG methods in cross-subject emotion recognition, thereby demonstrating the effectiveness of contrastive learning strategies in DG, particularly in cross-subject scenarios (Shen et al., 2023).

As an unsupervised learning paradigm, contrastive learning seeks to derive more discriminative feature representations by maximizing similarity between positive pairs while simultaneously minimizing alignment with negative pairs. Contrastive learning has demonstrated strong performance across multiple domains, including computer vision (Chen et al., 2020), natural language processing (NLP; Devlin et al., 2019), and bioinformatics (Li et al., 2021a; Liu et al., 2021). Initial applications of contrastive learning in the EEG domain typically adopted frameworks originally designed for computer vision or NLP. A representative example is the work of Mohsenvand et al., who adopted the Simple Framework for Contrastive Learning of Visual Representations (SimCLR) framework (Chen et al., 2020) to model similarities among multiple augmented views of the same raw EEG data (Mohsenvand et al., 2020), achieving favorable performance across several downstream tasks, including sleep stage classification, clinical anomaly detection, and emotion recognition.

In the context of cross-subject EEG recognition, Shen et al., inspired by inter-subject correlation studies (Hasson et al., 2004; Dmochowski et al., 2012), proposed a CNN-based contrastive learning framework designed to extract domain-invariant features. Without relying on extensive external data, their method achieved state-of-the-art performance in cross-subject emotion recognition (Shen et al., 2023). Deng et al. proposed the multi-source contrastive learning transfer model, which leverages contrastive learning to capture interindividual variability and incorporates domain adaptation and multi-domain feature learning to enhance model generalization across subjects (Deng et al., 2024). Zhi et al. introduced a supervised contrastive learning-based domain generalization network for cross-subject motor imagery and motor execution decoding, which extracts both domain-invariant and class-relevant discriminative representations from multiple EEG frequency bands (Zhi et al., 2025). Collectively, these studies further underscore the potential of contrastive learning in cross-subject EEG signal recognition.

Signal transformation

ERP exhibits phase-locking, that is, under specific stimulus conditions, participants’ EEG responses show a high degree of phase consistency (Helfrich and Knight, 2019). Given the importance of phase information for ERP recognition, we exploit phase as one of the bases for constructing multi-view representations. When using phase information, reliance on phase-synchronization indices (e.g., phase-locking value and phase-lag index) can overly simplify the information and may yield suboptimal performance (Moon et al., 2018; Wang et al., 2019). Therefore, we use both raw EEG and HT-EEG as two complementary input views to the network so as to exploit phase information more fully.

The Hilbert transform is a widely used method for extracting phase information from raw EEG. A two-view architecture enables us to retain amplitude information from the raw EEG while simultaneously deriving phase information from the HT-EEG view (Kim and Im, 2024). Specifically, the Hilbert transform is a mathematical operation that maps a real-valued signal to a signal that is orthogonal to the original. For a real-valued signal x(t), the Hilbert transform H(x(t)) is defined as:

H ( x ( t ) ) = 1 π p . v . ∫ − ∞ + ∞ u ( τ ) t- τ d τ

(1)

H ( x ( t ) ) = 1 π t *x ( t )

(2)

H(x(t)) is a complex-valued signal whose real part corresponds to the original signal x(t), while the imaginary part corresponds to the Hilbert-transformed signal (denoted here as x’(t)). The magnitude and argument (phase) of the complex signal H(x(t)) correspond to the instantaneous amplitude and phase, respectively, thus both amplitude and phase information of the original EEG signal are preserved. Unlike other approaches (e.g., wavelet transforms) that require a predefined time window, the Hilbert transform is data driven and computationally efficient, rendering it suitable for the real-time constraints of online BCI systems. By employing the Hilbert transform, our CNN is expected to capture both amplitude and phase features from EEG data, thereby improving the model’s capacity to extract class-discriminative features.

Multi-view contrastive learning domain generalization

Domain-generalization module

To improve cross-subject performance, our model incorporates a domain-generalization module that learns class-related, subject-invariant features from ERP signals, thereby enhancing robustness to individual variability. The domain-generalization module comprises two components: a domain-alignment component and a contrastive-learning component, both intended to address distributional discrepancies between domains. The domain-alignment component minimizes distributional differences among source subjects, while the contrastive-learning component facilitates learning of domain-invariant representations.

The domain-alignment component is specifically designed to mitigate intersubject distributional variability. Unlike domain-adaptation approaches, which align source and target distributions using target data, domain generalization aims to achieve robustness without access to target-domain samples. Consequently, we perform alignment across pairs (or groups) of source subjects: the rationale is that representations that are invariant to source-domain perturbations are more likely to generalize to unseen target subjects (Zhou et al., 2023). To realize domain alignment, we integrate the Deep Correlation Alignment (CORAL) loss into our framework to align second-order statistics across distributions. Deep CORAL promotes domain-invariant feature extraction by aligning the covariance matrices of the learned feature representations. Let f ( x ) ∈ R d denote the feature representation produced by the network for input x, where d is the feature dimensionality. For two source domains s1 and s2, let C s 1 and C s 2 denote the empirical covariance matrices of their feature sets. The CORAL loss between s1 and s2 is defined as:

L CORAL = 1 4 N 2 | | C s 1 − C s 2 | | F 2

(3)

where N denotes number of source domains and | | · | | F denotes the Frobenius norm. By minimizing L CORAL jointly with the other objectives, the network aligns second-order statistics across domains. With N source domains, we perform global alignment by averaging pairwise CORAL losses:

L D A = 1 N × ( N − 1 ) / 2 ∑ i = 1 N − 1 ∑ j = i + 1 N L CORAL ( C s i , C s j )

(4)

Contrastive learning aims to obtain domain-invariant representations by pulling together representations of samples from the same class (positive pairs) while pushing apart representations from different classes (negative pairs) in the latent space. To this end, a feature projection head is inserted between the base encoder and the contrastive loss to better shape representations for downstream prediction. The projection head architecture popularized by SimCLR (a nonlinear Multi-Layer Perceptron (MLP)) is widely used because it helps the base encoder learn improved representations. However, because ERP datasets are relatively small, MLP-based projection heads tend to be over-parameterized and prone to overfitting. Motivated by modular architectural design, we implement the projection head using convolutional units analogous to those in the base encoder. The projection head comprises two stages: (1) multi-scale temporal convolutions with kernels (1,T/8), (1,T/16), and (1,T/32) and (2) a depthwise (channel-wise) convolution stage with kernel size (2D,1) to model relationships across channels. This design reduces parameter count while preserving spatiotemporal feature extraction capacity. Each convolutional stage is followed by batch normalization (for feature standardization), an ELU activation (for nonlinearity), and appropriate regularization (e.g., dropout) to mitigate overfitting. After projection, features are vectorized so that inner products in the projection space can be used to measure similarity. The projection head is discarded after training; at inference time, we use only the feature encoder and the classifier. In deployment, only the feature encoder and classifier C are used for ERP recognition.

Contrastive losses

The MVCLDG framework defines loss functions for the raw EEG view, the HT-EEG view, and the fused view. For each view, three objectives are jointly optimized: the classification loss L C , the domain-alignment loss L D A , and the contrastive-learning loss L C C .

The classification loss is the cross-entropy loss, which minimizes the discrepancy between predicted and ground-truth labels. For a batch of N samples, the cross-entropy loss is defined as:

L C = − 1 N ∑ i = 1 N y i l o g ( y i ′ )

(5)

Here y i denotes the ground-truth label indicator for input X i , y i ′ denotes the model’s predicted probability, and N is the batch size.

To construct effective batches for contrastive learning, we employ a stratified sampling strategy. Let the source domain consist of n subjects. For each subject, k samples are randomly and uniformly selected from each of the C experimental classes, resulting in a total batch size of N = n × C × k. This design ensures strict class balance within each batch and avoids sample duplication. The proposed strategy mitigates potential bias caused by class imbalance and encourages each batch to encompass diverse intersubject variations, thereby facilitating the learning of domain-invariant representations.

Following the batch construction strategy described above, positive and negative sample pairs for contrastive learning are formally defined. A positive pair is formed by two projection vectors p si + and p sj + that belong to the same class but originate from different subjects si and sj. Samples from different classes are treated as negatives. In this way, each positive pair is contrasted against a set of negative examples. Features extracted from the raw EEG view (denoted F raw ) are mapped through a nonlinear projection head to obtain projected representations P raw , which are used for the contrastive objective. We employ the NT-Xent loss (as in SimCLR) to define the contrastive loss:

l ( s i + , s j + ) = − log exp ( sim ( p si + , p sj + ) / τ ) ∑ k=1 2N 1 k ≠ i exp ( sim ( p si , p sk ) / τ )

(6)

L CL = ∑ i = 1 N ∑ Ci = Cj & j ≠ i N [ l ( i , j ) +l ( j , i ) ]

(7)

In the above formula, τ is the temperature hyperparameter and 1 k ≠ i ∈ { 0 , 1 } is an indicator that excludes the anchor itself from the denominator. sim ( u , v ) = u T v ||u||||v|| is the cosine similarity function. The total loss L raw of the original EEG view is as follows:

L raw = L C − raw + λ 1 L D A − raw + λ 2 L C L − raw

(8)

The weight coefficients λ 1 and λ 2 are used to balance the contributions of each loss. Analogous computations are performed for the HT-EEG view and for the fused-view contrastive objective. The overall loss of the MVCLDG framework is as follows:

L total =L raw +L HT +L fusion

(9)

By jointly optimizing these objectives, the model converges toward representations that are both class-discriminative and domain-invariant.

Datasets

Dataset II (Chinese semantic–syntactic violation ERP)

The Chinese semantic–syntactic violation ERP (CSSV-ERP) dataset was collected in-house. EEG signals were recorded from 12 healthy participants using 64 electrodes placed according to the 10–20 system, at a sampling rate of 500 Hz. As illustrated in Figure 2, Chinese sentences were presented to participants one word at a time. Each word was displayed for 400 ms (stimulus), followed by a 100 ms blank screen (fixation). Participants completed the task under three conditions (Zhu et al., 2022): (1) correct sentences (CORR), that is, well-formed subject–verb–object sentences; (2) local syntactic violation (SYN-P), in which a degree adverb that normally modifies an adjective is erroneously inserted before the object noun, rendering the phrase syntactically ill-formed; (3) semantic violation (SEM), in which the object conflicts with the verb’s selectional restrictions. This dataset aims to probe participants’ cognitive processing of grammatical and syntactic violations by analyzing EEG signals recorded after participants’ button presses. In BCIs that select characters by detecting users’ cognitive responses, semantic processing (N400) can serve either as an additional control signal or as a monitoring indicator of the appropriateness of system instructions. Each participant completed a total of 192 trials, with 64 trials per stimulus category. The final sample consisted of 10 males and 2 females aged 22–26 years, all of whom held a bachelor’s degree. Thirteen participants were initially recruited for this study. One participant was excluded due to excessive EEG artifacts, accounting for more than 25% of the total trials. Before analysis, EEG recordings were band-pass filtered between 0.1 and 60 Hz using an FIR filter in EEGLAB. Independent component analysis was applied to remove ocular and muscular artifacts, and the data were re-referenced to TP9 and TP10. After preprocessing, a median of 68.5 correct trials (range: 54–63), 61.5 semantic violation trials (range: 48–64), and 61.5 syntactic violation trials (range: 47–63) were retained per subject. EEG segments corresponding to semantic and syntactic violations were extracted from the [−0.15, 1] s time window relative to the critical word onset (first object noun position) and used as features for a multi-class classification task, with baseline correction applied using the −150 to 0 ms interval. Detailed information about the CSSV-ERP dataset can be found in the dataset card (Table 1).

OPEN IN VIEWER

FIG. 2.

Example stimuli from the semantic–syntactic violation paradigm. (a) Words highlighted in the corresponding colors indicate the keywords analyzed. Correct sentences (CORR, red); local syntactic violations (SYN-P, orange); semantic violations (SEM, blue). (b) Black bars indicate word presentation duration (400 ms); purple bars denote fixation duration (100 ms).

OPEN IN VIEWER

Table 1.

Dataset Card for Chinese Language Event-Related Potential Dataset

Item	Description
Purpose	To investigate the neural mechanisms underlying Chinese semantic (N400) and syntactic (P600) violations, providing benchmark data for developing language cognition–based brain–computer interfaces.
Subjects	12 healthy right-handed subjects (10 male, 2 female), aged 22–26 years, all holding bachelor’s degrees.
Paradigm and conditions	Single-word sequence presentation. Three conditions: correct sentences, local syntactic violation sentences, and semantic violation sentences, with 64 trials per condition per subject.
Acquisition parameters	• Device: Brain Products actiCHamp Plus • EEG cap model: Brain Products actiCAP slim/snap • Electrode montage: 64 channels, International 10–20 system • Sampling rate: 500 Hz • Key markers: S1/S2/S4 (stimulus presentation for correct/semantic violation/syntactic violation conditions), S7/S8 (correct/error response)
Preprocessing pipeline	1. 0.1–60 Hz band-pass filtering 2. ICA for removal of ocular and muscular artifacts 3. Re-referencing to TP9 and TP10 4. Epoch extraction [−0.15, 1] s, baseline correction based on [−0.15, 0] s interval
Recommended data split	Leave-one-subject-out cross-validation, to strictly evaluate cross-subject generalization capability.
Access	DOI: https://doi.org/10.5281/zenodo.18362005

EEG, electroencephalography; ICA, independent component analysis.

Experiment details

Results

We adopted the LOSO scheme, in which data from one subject were used for testing, while data from all other subjects were used for training the classifier, in order to evaluate the proposed method. The LOSO procedure resembles real-world cross-subject applications, where the model is trained on available data and validated on unseen test subjects. Since the number of target samples is much smaller than that of nontarget samples, the area under the Receiver Operator Characteristic (ROC) curve (AUC) was selected as the evaluation metric for binary classification. The AUC results on Dataset I and Dataset II are presented in Tables 2 and 3, respectively. For the three-class classification task on Dataset II, the one-vs-rest strategy is used to compute the multi-class AUC.

OPEN IN VIEWER

Table 2.

Classification Performance (Area Under the ROC Curve) on Dataset I (ERN)

Method	Deep learning				Domain generalization					Contrastive learning
	EEGNet	EEG-Inception	EEG-Conformer	HiReNet	Mixup	MMD	IRM	MLDG	DANN	SelfReg	Ours
sub1	0.5623	0.6752	0.6097	0.6891	0.5847	0.6612	0.6516	0.5847	0.6730	0.6959	0.6304
sub2	0.5397	0.5065	0.7383	0.5563	0.5189	0.4753	0.5239	0.5326	0.5125	0.519	0.6973
sub3	0.7082	0.703	0.7699	0.7294	0.7343	0.6775	0.6786	0.7428	0.7687	0.7131	0.8261
sub4	0.6872	0.725	0.705	0.6974	0.7018	0.6813	0.691	0.7191	0.6855	0.7307	0.754
sub5	0.5686	0.6056	0.5982	0.5918	0.5631	0.5882	0.5879	0.5889	0.6040	0.6099	0.5386
sub6	0.7585	0.7917	0.6645	0.7701	0.7748	0.7679	0.7705	0.7691	0.8189	0.7831	0.7986
sub7	0.5996	0.5641	0.6727	0.6179	0.6211	0.5295	0.5255	0.5818	0.6420	0.5752	0.6524
sub8	0.6917	0.6798	0.6598	0.6583	0.6892	0.6637	0.6639	0.6895	0.6450	0.687	0.7062
sub9	0.8068	0.7978	0.712	0.8329	0.7999	0.8074	0.8082	0.807	0.7958	0.8261	0.8394
sub10	0.6745	0.6621	0.6547	0.6219	0.6649	0.6663	0.6656	0.655	0.6467	0.6369	0.6658
sub11	0.6848	0.6825	0.6085	0.6567	0.6636	0.6314	0.6282	0.6579	0.6774	0.6802	0.7311
sub12	0.6717	0.7593	0.7798	0.8332	0.8253	0.7051	0.7226	0.8543	0.8342	0.8016	0.8611
sub13	0.8302	0.8403	0.8188	0.7901	0.8562	0.8197	0.8643	0.7755	0.8004	0.8504	0.8414
sub14	0.7596	0.7152	0.6271	0.693	0.7732	0.7371	0.7157	0.7759	0.7497	0.7422	0.7871
sub15	0.681	0.6945	0.5835	0.6366	0.6987	0.638	0.6395	0.7096	0.6602	0.7048	0.6862
sub16	0.5268	0.5334	0.5216	0.5367	0.5511	0.5898	0.591	0.5421	0.5990	0.543	0.5779
sub17	0.7030	0.6761	0.763	0.7336	0.6771	0.7062	0.6979	0.6667	0.7194	0.6887	0.7676
sub18	0.7526	0.8387	0.5832	0.7905	0.8001	0.7559	0.7043	0.7800	0.6998	0.8330	0.7586
sub19	0.8534	0.9067	0.9211	0.8615	0.8838	0.8575	0.8666	0.8950	0.9048	0.9117	0.9084
sub20	0.5158	0.5076	0.5247	0.5284	0.5445	0.5182	0.5095	0.5393	0.5697	0.5810	0.6112
sub21	0.7226	0.789	0.749	0.7747	0.8248	0.7329	0.6947	0.7979	0.8492	0.7732	0.7659
sub22	0.8120	0.7245	0.8918	0.7585	0.8124	0.8116	0.7569	0.8104	0.8280	0.7214	0.8739
sub23	0.8889	0.865	0.888	0.8957	0.9003	0.8899	0.903	0.9041	0.9052	0.8831	0.895
sub24	0.8766	0.8503	0.8377	0.8498	0.8412	0.8742	0.8072	0.8412	0.8813	0.8520	0.9016
sub25	0.7381	0.7463	0.6213	0.6709	0.7294	0.7356	0.6967	0.7288	0.5924	0.7595	0.7223
sub26	0.6872	0.7066	0.6549	0.7151	0.6976	0.6858	0.7018	0.6859	0.6907	0.7264	0.7078
Mean	0.7039	0.7133	0.6984	0.7112	0.7205	0.7003	0.6949	0.7167	0.7213	0.7242	0.7502
SD	0.1060	0.1082	0.1112	0.1023	0.1104	0.1081	0.1027	0.1093	0.1092	0.1044	0.1019

Values in bold indicate the best performance in each row.

DANN, domain-adversarial neural network; EEG, electroencephalography; IRM, invariant risk minimization; MLDG, meta-learning for domain generalization; MMD, maximum mean discrepancy; SD, standard deviation.

OPEN IN VIEWER

Table 3.

Classification Performance (Multi-Class Area Under the ROC Curve; Macro-F1) on Dataset II (Chinese Semantic–Syntactic Violation Event-Related Potential)

Method	Deep learning				Domain generalization					Contrastive learning
	EEGNet	EEG-Inception	EEG-Conformer	HiReNet	Mixup	MMD	IRM	MLDG	DANN	SelfReg	Ours
sub1	0.5398	0.5633	0.5177	0.5327	0.5464	0.5139	0.5441	0.5344	0.5985	0.5855	0.5277
sub2	0.5243	0.5213	0.5499	0.5007	0.5766	0.5563	0.5783	0.5592	0.5829	0.5818	0.6051
sub3	0.6919	0.7429	0.6441	0.6548	0.7152	0.6942	0.7303	0.7104	0.7430	0.6747	0.7219
sub4	0.5752	0.6619	0.5486	0.6748	0.5555	0.659	0.6559	0.5644	0.7162	0.7247	0.6908
sub5	0.5235	0.576	0.5011	0.5946	0.5513	0.5412	0.5427	0.5426	0.5956	0.6587	0.671
sub6	0.5713	0.5615	0.5506	0.5552	0.5563	0.5481	0.5539	0.5594	0.5919	0.5571	0.5822
sub7	0.5452	0.5116	0.5402	0.5929	0.6002	0.6082	0.6021	0.5906	0.5886	0.5794	0.622
sub8	0.6464	0.6298	0.6217	0.6121	0.5913	0.5908	0.5962	0.6103	0.6304	0.6623	0.6725
sub9	0.6149	0.6307	0.6076	0.6177	0.5838	0.6229	0.6162	0.6005	0.6652	0.6468	0.7006
sub10	0.6547	0.6975	0.6211	0.6264	0.6473	0.6349	0.6279	0.6819	0.6966	0.6994	0.7335
sub11	0.5862	0.58	0.5713	0.6241	0.6257	0.639	0.6484	0.6348	0.6614	0.6354	0.6951
sub12	0.5029	0.5232	0.5187	0.5326	0.547	0.5491	0.5356	0.5503	0.6086	0.5446	0.6045
Mean	0.5814	0.6000	0.5661	0.5932	0.5914	0.5965	0.6026	0.5949	0.6399	0.6292	0.6522
SD	0.0595	0.0734	0.0469	0.0529	0.0505	0.0553	0.0577	0.0561	0.0554	0.0583	0.0630
Macro-F1 a	0.3893	0.3814	0.3493	0.3757	0.3416	0.3553	0.3635	0.3600	0.3925	0.3833	0.4412

Values in bold indicate the best performance in each row.

a

The macro-F1 value represents the average macro-F1 score of the model across all subjects.

DANN, domain-adversarial neural network; EEG, electroencephalography; IRM, invariant risk minimization; MLDG, meta-learning for domain generalization; MMD, maximum mean discrepancy; SD, standard deviation.

In Dataset I, the AUC values of the deep learning methods EEGInception and EEGNet were 0.7133 ± 0.1082 and 0.7039 ± 0.1060, respectively, demonstrating their ability to capture the spatial and temporal representations of ERP signals. By exploiting phase information derived from the Hilbert transform, HiReNet achieves an AUC of 0.7112 ± 0.1023. EEG-Conformer, which is based on a Transformer architecture and is optimized for modeling long-range dependencies, shows limited effectiveness on short-duration ERP signals, yielding an AUC of 0.6984 ± 0.1112. The Mixup and MLDG methods achieved AUCs of 0.7205 ± 0.1104 and 0.7167 ± 0.1093, respectively, outperforming the standard deep learning baselines and indicating that they can better learn domain-invariant features. MMD and IRM tend to perform poorly on small-sample datasets. Among the domain generalization methods, DANN achieves the highest performance, with an AUC of 0.7213 ± 0.1092. MMD enforces alignment of feature distributions across domains; under limited-sample conditions, this can over-regularize the learned representations and collapse class-specific structure, thereby blurring class boundaries. IRM is particularly vulnerable when sample sizes are small: high variance in domain-wise risk estimates can destabilize optimization and cause the model to converge to poor local minima. The contrastive learning–based approaches, SelfReg and MVCLDG, further improved the results, yielding AUCs of 0.7242 ± 0.1044 and 0.7502 ± 0.1019, respectively. Notably, MVCLDG achieved the highest average AUC of 0.7502 ± 0.1019, and paired-sample t-tests demonstrated a statistically significant performance improvement over existing methods (p < 0.05). This suggests that our method can extract essential domain-invariant representations more effectively through contrastive learning. Moreover, the reduced variance observed in contrastive learning methods indicates lower sensitivity to specific domain shifts. Overall, these experiments confirm the superiority of the proposed algorithm on public ERP datasets.

On our self-collected Dataset II, standard deep learning models remained suboptimal: Although EEGInception (AUC = 0.6000 ± 0.0734) outperformed EEGNet (AUC = 0.5814 ± 0.0595), both were surpassed by specialized domain generalization algorithms and our proposed approach. Owing to the smaller size of this dataset, the performance of EEG-Conformer degrades markedly. In domain generalization algorithms, the aforementioned drawbacks of MMD and IRM on small-sample datasets are once again manifested. MVCLDG achieved the highest average AUC of 0.6522 ± 0.0630 and the highest average macro-F1 score of 0.4412 ± 0.0644, representing an improvement over the strongest baseline method, DANN (AUC = 0.6399 ± 0.0554), and an even larger advantage compared with standard deep learning models. Importantly, this performance gain was achieved while maintaining relatively high stability.

To rigorously assess the performance gains of MVCLDG, paired-sample t-tests were conducted to compare it against each of the 10 baseline models. To account for errors arising from multiple comparisons, all p values were adjusted using the Holm–Bonferroni correction, and statistical significance reported throughout the article is based on the corrected values. In addition, we report the mean AUC differences ( Δ AUC) together with their 95% confidence intervals, estimated via bootstrapping. The results are summarized in Tables 4 and 5.

OPEN IN VIEWER

Table 4.

Pairwise Statistical Comparisons Between Multi-View Contrastive Learning Domain Generalization and Baseline Models on Dataset I

Comparison	Mean difference (ΔAUC)	95% CI	t	Adjusted p
MVCLDG versus IRM	0.0554	[0.0337, 0.0778]	4.85	<0.001***a
MVCLDG versus EEG-Conformer	0.0518	[0.0286, 0.0754]	4.26	0.002**
MVCLDG versus mmd	0.0500	[0.0278, 0.0736]	4.13	0.002**
MVCLDG versus EEGNet	0.0463	[0.0280, 0.0665]	4.68	<0.001***
MVCLDG versus HiReNet	0.0391	[0.0210, 0.0573]	4.16	0.002**
MVCLDG versus EEG-Inception	0.0369	[0.0132, 0.0610]	2.89	0.024*
MVCLDG versus MLDG	0.0335	[0.0165, 0.0518]	3.64	0.006**
MVCLDG versus Mixup	0.0298	[0.0124, 0.0484]	3.12	0.018*
MVCLDG versus DANN	0.0289	[0.0083, 0.0497]	2.69	0.025*
MVCLDG versus SelfReg	0.0260	[0.0025, 0.0493]	2.16	0.041*

a

Note: *p < 0.05; **p < 0.01; ***p < 0.001.

AUC, area under the ROC curve; CI, confidence interval; DANN, domain-adversarial neural network; EEG, electroencephalography; IRM, invariant risk minimization; MLDG, meta-learning for domain generalization; MMD, maximum mean discrepancy.

OPEN IN VIEWER

Table 5.

Pairwise Statistical Comparisons Between Multi-View Contrastive Learning Domain Generalization and Baseline Models on the Dataset II

Comparison	Mean difference (ΔAUC)	95% CI	t	Adjusted p
MVCLDG versus EEG-Conformer	0.0862	[0.0616, 0.1116]	6.48	<0.001***a
MVCLDG versus EEGNet	0.0709	[0.0451, 0.0961]	5.17	0.002**
MVCLDG versus Mixup	0.0609	[0.0350, 0.0866]	4.31	0.006**
MVCLDG versus HiReNet	0.0590	[0.0384, 0.0777]	5.84	0.001***
MVCLDG versus MLDG	0.0573	[0.0349, 0.0807]	4.67	0.004**
MVCLDG versus MMD	0.0558	[0.0372, 0.0762]	5.44	0.002**
MVCLDG versus EEG-Inception	0.0523	[0.0255, 0.0783]	3.70	0.010*
MVCLDG versus IRM	0.0496	[0.0259, 0.0739]	3.89	0.010*
MVCLDG versus SelfReg	0.0230	[0.0014, 0.0408]	2.18	0.104
MVCLDG versus DANN	0.0123	[−0.0097, 0.0332]	1.08	0.303

a

Note: *p < 0.05; **p < 0.01; ***p < 0.001.

AUC, area under the ROC curve; CI, confidence interval; DANN, domain-adversarial neural network; EEG, electroencephalography; IRM, invariant risk minimization; MLDG, meta-learning for domain generalization; MMD, maximum mean discrepancy.

The results indicate that on the Dataset I, MVCLDG significantly outperforms all 10 baseline methods (corrected p < 0.05), with mean AUC improvements ranging from 0.026 to 0.055. On Dataset II, MVCLDG achieves statistically significant improvements over 8 of the 10 baselines. Notably, when compared with the strongest competitors, DANN and SelfReg, the average differences remain positive; however, their 95% confidence intervals include zero, and the corrected p values do not meet the significance criterion. This outcome highlights the inherent difficulty of obtaining further gains on this more challenging task.

To evaluate the performance of MVCLDG on the CSSV-ERP test set, we aggregated the LOSO test results across 12 subjects to construct the aggregated confusion matrix shown in Figure 3.

OPEN IN VIEWER

FIG. 3.

Aggregated confusion matrices of the MVCLDG model on the CSSV-ERP test set across 12 subjects under the leave-one-subject-out (LOSO) evaluation protocol. The left matrix shows the raw classification counts, while the right matrix presents recall rates normalized by the true classes for the three-class classification task. CSSV-ERP, Chinese semantic–syntactic violation event-related potential; MVCLDG, multi-view contrastive learning domain generalization.

Although MVCLDG achieved the best relative performance among all baseline methods, the confusion matrix in Figure 3 still reveals the inherent challenges of this classification task. The model demonstrates a measurable ability to discriminate between semantic and syntactic violations, suggesting that it may capture neural features specific to different violation types. Among the three classes, semantic violations are recognized most reliably, achieving a recall rate of 60.6%. Meanwhile, the model exhibits complex multidirectional error patterns, with confusion between correct trials and syntactic violations being the dominant error type. This phenomenon may arise because the recognition of syntactic violations depends more strongly on contextual and individual factors, rendering their neural patterns more difficult to distinguish from those of correct sentences at the single-trial level. In contrast, direct confusion between semantic and syntactic violations remains relatively low, suggesting that the model may have learned neural representations that differentiate between distinct types of anomalies.

Overall, these findings are consistent with those reported in Santamaría-Vázquez et al. (2020). Since single-trial decisions often fail to achieve the required accuracy, repeated trials are necessary to ensure reliable classification. Although improvements were observed compared with existing algorithms, the current state of research remains insufficient to achieve satisfactory discriminative capability in cross-subject single-trial settings. Furthermore, substantial intersubject variability resulted in relatively high algorithm variance.

Ablation study

An ablation study was conducted within our model framework to assess the contribution of each component to overall performance. To ensure a fair comparison, all hyperparameters were kept identical, while specific components or architectural modules were removed or replaced, with the remaining parts of the model adjusted accordingly. Six ablated variants were examined: (a) without the multi-scale temporal extraction module; (b) without raw EEG input; (c) without HT-EEG input; (d) without the contrastive learning loss module; (e) replacing the projection head with an MLP; and (f) using a single-path contrastive loss.

The results in Table 6 highlight several key findings.

OPEN IN VIEWER

Table 6.

Result of Ablation Study on Dataset II

Method	Without multi-scale temporal convolution module	Only raw EEG input	Only HT EEG input	Without contrastive loss	With MLP projection	Only fusion-view loss	Ours
sub1	0.5514	0.5454	0.5318	0.5505	0.5127	0.5139	0.5277
sub2	0.5282	0.5511	0.533	0.539	0.5487	0.5674	0.6051
sub3	0.6822	0.6707	0.6958	0.6764	0.7321	0.7042	0.7219
sub4	0.6368	0.6483	0.6413	0.6685	0.6946	0.6937	0.6908
sub5	0.6727	0.6994	0.6445	0.6701	0.6372	0.6976	0.671
sub6	0.569	0.6348	0.6151	0.5792	0.5648	0.5845	0.5822
sub7	0.5419	0.6209	0.671	0.5674	0.6087	0.6099	0.622
sub8	0.5845	0.6189	0.6149	0.6711	0.6305	0.6317	0.6725
sub9	0.6382	0.6286	0.6495	0.6477	0.6665	0.6755	0.7006
sub10	0.6629	0.6859	0.6726	0.6925	0.7045	0.6971	0.7335
sub11	0.6284	0.6775	0.6472	0.6185	0.6504	0.6324	0.6951
sub12	0.5716	0.5567	0.5006	0.5456	0.5616	0.583	0.6045
Mean	0.6057	0.6282	0.6181	0.6189	0.6260	0.6326	0.6522
SD	0.0541	0.0532	0.0628	0.0588	0.0684	0.0623	0.0630

EEG, electroencephalography; HT, Hilbert-transformed; SD, standard deviation.

Overall, our framework integrates modules that enhance feature extraction breadth, improve domain-invariant representation learning, and mitigate overfitting. The ablation experiments collectively demonstrate the contribution of each component and underscore their potential in advancing cross-subject ERP recognition tasks.

Feature distribution visualization

To illustrate the effect of contrastive learning, we present two t-SNE (t-distributed stochastic neighbor embedding) (Com and Hinton, 2008) visualizations of features extracted from subjects by different models. These visualizations help elucidate how the features are separated and clustered in the embedding space. Figure 4 compares the t-SNE projections of the baseline model and our model on the Dataset I training set. With the baseline model, features extracted from EEG signals disperse around distinct cluster centers that largely align with individual subjects (Fig. 4a). Clusters from different subjects that correspond to the same task class lie far apart in the embedding space; for example, features for the same task from Subject 1 and Subject 2 are distant, yet they can be closer to features from different tasks of the same subject.

OPEN IN VIEWER

FIG. 4.

t-SNE visualization of features for (a) Baseline and (b) MVCLDG on Dataset I. Dots denote projected features from the training set (source domain). Marker color and shape indicate class and subject, respectively. Arrows highlight the true subject/class associated with each cluster. MVCLDG, multi-view contrastive learning domain generalization; t-SNE, t-distributed stochastic neighbor embedding.

By contrast, MVCLDG yields features with clearer separation across stimulus classes (Fig. 4b). We attribute this to our domain-alignment mechanism together with the contrastive loss, which enlarges interclass differences while reducing intra-class variance. These results indicate that the model effectively learns domain-invariant representations, mitigates intersubject variability without sacrificing class separability, and thereby facilitates cross-subject ERP recognition. Quantitative evidence further substantiates the enhanced feature separability: Our model achieves a separation ratio (defined as the ratio of interclass to intra-class feature distances) of 1.8031, corresponding to a 9.79% improvement over the baseline model (1.6423).

Furthermore, Figure 5 contrasts the t-SNE projections of the baseline and our model on the Dataset I test set. The baseline produces test-sample features that are noticeably more dispersed and lack explicit cross-subject alignment; in contrast, MVCLDG forms tighter within-class clusters, and the concentric arrangement suggests that contrastive learning induces a hyperspherical feature space. Taken together, these t-SNE visualizations confirm that our approach learns features that are both domain-invariant and class-discriminative, and that this generalization extends to unseen test samples.

OPEN IN VIEWER

FIG. 5.

t-SNE visualization of features for (a) Baseline and (b) MVCLDG on Dataset I. Dots denote projected features from the test set (target domain). Different colors indicate different classes. MVCLDG, multi-view contrastive learning domain generalization; t-SNE, t-distributed stochastic neighbor embedding.

Interpretability

We employed class activation maps (CAM) (Chattopadhay et al., 2018; Selvaraju et al., 2017) to visualize the spatiotemporal activation patterns generated by MVCLDG and validated them against established neurophysiological findings. Among existing CAM methods, Eigen-CAM (Muhammad and Yeasin, 2020) visualizes the principal components of learned features without altering the network architecture or requiring gradient backpropagation, thereby providing a robust and reliable localization of discriminative regions. It has also been shown to outperform alternative approaches in identifying critical areas (Muhammad and Yeasin, 2020). Therefore, we adopted Eigen-CAM as our visualization tool to accurately identify the key features captured by MVCLDG.

We conducted interpretability analysis of MVCLDG on ERN (Dataset I) and semantic/syntactic violation (Dataset II) using the following procedure: high-confidence correctly classified trials were first selected as input, and Eigen-CAM was applied to obtain CAMs S i j and T i j from the spatial and temporal convolutional layers, respectively. The maps were then averaged across trials, and temporal maps were further averaged across channels, yielding the mean temporal features T ij ′ and mean spatial features S ij ′ for each subject–class combination. Subsequently, all T ij ′ were aggregated and normalized to generate temporal heatmaps. At the peak time points t j of these heatmaps, S i j were sampled and normalized to produce category-specific topographical maps. For comparison with neurophysiological evidence, topographies at peak moments were also extracted from the raw EEG signals.

The ERN is an electrophysiological signal closely associated with error feedback. As illustrated in Figure 6, panels A1 and A2 display temporal heatmaps of spatial convolutional features for correct and incorrect feedback trials, with time zero indicating feedback onset. The black dashed line marks the moment of maximum Eigen-CAM response, denoted as the “most salient moment.” The salient moments for correct and incorrect feedback were 0.335 and 0.390 s, respectively. The highlighted regions correspond precisely to two well-established ERN components (Margaux et al., 2012): the pos-ErrP and neg-ErrP. These findings indicate that the discriminative features extracted by MVCLDG align with prior neuroscientific knowledge and effectively capture class-related information.

OPEN IN VIEWER

FIG. 6.

Temporal heatmaps and scalp topographies visualized with Eigen-CAM on Dataset I. (A1, A2) Temporal heatmaps of the spatial convolutional layer in the MVCLDG model for correct and error feedback trials. The black dashed lines indicate the most salient moment (correct: 0.335 s; error: 0.390 s), which corresponds to the positions of the known pos-ErrP and neg-ErrP components. (B1, B2) Original EEG topographic maps corresponding to the most salient time points. (C1, C2) and (D1, D2) Topographic visualizations of the temporal convolutional layer outputs at the same time points for the baseline model and the MVCLDG model, respectively. CAM, class activation maps; EEG, electroencephalography; MVCLDG, multi-view contrastive learning domain generalization.

While learning class-relevant representations, MVCLDG simultaneously reduces its focus on task-irrelevant features. Panels B1 and B2 illustrate the scalp topographies of raw EEG data at the salient moments for correct and incorrect feedback trials. Panels C1 and C2, together with D1 and D2, present the temporal convolutional features of correct and incorrect feedback trials visualized from the baseline (MLDG) and MVCLDG models, respectively. Panels B1 and B2 suggest that, at salient time points, the topographies of raw EEG signals offer limited interpretability and fail to directly reveal channel relevance, often necessitating comparisons between target and nontarget conditions to identify informative electrodes. In contrast, CAM-based temporal feature visualizations explicitly highlight the relative importance of individual channels. Panels D1 and D2 display MVCLDG’s spatial feature visualizations at the salient moments. Panel D1 shows that MVCLDG assigns strong importance to channel P7 for correct feedback, whereas panel D2 indicates that Cz plays a dominant role in distinguishing erroneous feedback. The channels highlighted in the visualization exhibit scalp-surface topographic distributions that are consistent with those of brain regions well established in ERN research, thereby supporting the validity of MVCLDG’s spatial feature extraction. Compared with the baseline results in panels C1 and C2, MVCLDG places less emphasis on task-irrelevant regions and greater emphasis on task-relevant ones, underscoring its stronger capability for extracting domain-invariant features.

A semantic/syntactic violation refers to the occurrence in sentence processing when a word or structure violates semantic or grammatical rules. Such violations trigger additional neural resources to process the linguistic anomaly, producing characteristic electrophysiological responses. As illustrated in Figure 7, panels A1 and A2 present temporal heatmaps of spatial convolutional features for semantic and syntactic violation trials visualized by Eigen-CAM, with time zero marking the critical word onset. Because MVCLDG captured a greater number of salient moments in the more complex neural dynamics of Dataset II, the black dashed lines indicate overlaps between well-known ERP components (N400 and P600) and Eigen-CAM highlighted regions, which we refer to as “prior salient moments.” Semantic and syntactic violations were associated with salient moments at 0.374 and 0.595 s, respectively, again capturing features consistent with prior neuroscientific knowledge.

OPEN IN VIEWER

FIG. 7.

Temporal heatmaps and scalp topographies visualized with Eigen-CAM on Dataset II. (A1, A2) Temporal heatmaps of the spatial convolutional layer in the MVCLDG model for semantic violation and syntactic violation trials. The prior salient time points captured by the model (semantic: 0.374 s; syntactic: 0.595 s) correspond to the time windows of the classical event-related potential components N400 and P600 associated with language processing. (B1, B2) Original EEG topographic maps corresponding to the prior salient time points. (C1, C2) Spatial feature visualizations of the MVCLDG model at the same time points. CAM, class activation maps; EEG, electroencephalography; MVCLDG, multi-view contrastive learning domain generalization.

Panels C1 and C2 show MVCLDG’s spatial feature visualizations at the salient moments. Compared with the raw EEG signals shown in panels B1 and B2, panel C1 demonstrates that MVCLDG assigns strong importance to channels Fz and F1 during semantic violations, whereas panel C2 indicates heightened attention to channel CP1 during syntactic violations. The scalp-topographic emphasis on fronto-central (e.g., F1/Fz) and centro-parietal (e.g., CP1) electrodes for semantic and syntactic violations, respectively, is broadly consistent with the canonical distributions of the N400 and P600 components. This indirectly supports that our model learns features whose inferred neural generators align with regions traditionally associated with each process, confirming its ability to capture domain-invariant neurophysiological patterns.

BCI practicality

The number of trainable parameters during training and inference for the proposed method on Dataset I is summarized in Table 7. As a canonical baseline, EEGNet is the most parameter-efficient model among all compared methods. Conventional domain generalization methods do not modify the network architecture; therefore, their parameter counts remain identical to that of the baseline EEG-Inception model. Notably, DANN and contrastive learning–based methods exhibit distinct behaviors between training and inference: DANN removes the adversarial discriminator at inference time, whereas contrastive learning approaches discard the projection head.

OPEN IN VIEWER

Table 7.

Comparison of Model Complexity Across Different Methods

Method	Training parameters	Inference parameters
EEGNet	2,746	2,746
EEG-Inception	42,548	42,548
EEG-Conformer	211,306	211,306
HiReNet	14,298	14,298
Mixup	42,548	42,548
MMD	42,548	42,548
IRM	42,548	42,548
MLDG	42,548	42,548
DANN	46,467	42,548
SelfReg	87,438	42,254
Ours	103,940	28,492

DANN, domain-adversarial neural network; EEG, electroencephalography; IRM, invariant risk minimization; MLDG, meta-learning for domain generalization; MMD, maximum mean discrepancy.

Built upon multi-view contrastive learning, our method achieves a distinctive balance between model capacity and deployment efficiency. During training, an additional projection module termed Incep2Incep is introduced, comprising 57,012 parameters. This module mirrors the encoder architecture while incorporating additional nonlinearity, thereby enhancing feature learning and improving cross-subject generalization. At inference, the projection module is entirely removed, leaving only the encoder and classifier, resulting in a total of 28,492 inference parameters. This design reflects the fact that the projection module primarily serves to structure the feature space during training; once optimization is completed, this role is effectively absorbed by the encoder parameters.

From a practical BCI perspective, this design offers two key advantages. First, the lightweight inference model enables faster response times in real-world closed-loop systems. Second, despite the reduced parameter count during inference, the higher model capacity during training ensures superior classification performance compared with all baseline approaches.

Limitations and future work

This study still has considerable scope for further improvement. First, MVCLDG currently exploits only the raw signal and the Hilbert-transformed signal, without fully leveraging the potential advantages of incorporating multiple views. Future work may focus on exploring additional representative views, such as time–frequency views and amplitude views, to facilitate the extraction of domain-invariant features. Second, MVCLDG is primarily designed for recognizing different stimuli in cross-subject scenarios within a single dataset. However, its effectiveness across datasets and, more broadly, across tasks has yet to be systematically explored and validated. In future work, we plan to directly apply the model trained on the dataset in this study to publicly available ERP datasets with similar paradigms (e.g., applying the model trained on Dataset I to a P300 speller dataset) for cross-dataset validation.

Rehabilitation medicine increasingly acknowledges the value of EEG analysis in supporting patient recovery. Nevertheless, the neurophysiological significance of MVCLDG remains limited. Further investigations leveraging multimodal techniques, such as simultaneous EEG–fMRI acquisition, to more precisely characterize task-related and violation-related brain activity may provide valuable opportunities for real-time monitoring and assessment of rehabilitation progress.

Advancing the clinical translation of the proposed approach represents an important direction for future research. For example, the MVCLDG framework could be integrated into portable EEG systems to facilitate the assessment of post-stroke cognitive function. Patients would perform standardized semantic tasks (e.g., the Dataset II paradigm), while the system analyzes their ERP components and compares them against normative baselines derived from healthy cohorts, thereby providing clinicians with quantitative indicators of cognitive impairment and recovery. A key strength of MVCLDG lies in its cross-subject generalization capability, which allows models pre-trained on large-scale healthy datasets to be directly deployed for new patients, enabling reliable detection without the need for individual calibration. This property is of particular clinical relevance for patient populations with language or motor impairments, for whom active behavioral feedback is difficult or infeasible.