Abstract
Anomalous survey responses, including random, careless, extreme, acquiescent, straightline, and alternating responding, threaten the validity of survey-based research. Machine learning (ML) algorithms offer flexible, model-agnostic alternatives to traditional detection methods, yet their relative effectiveness across anomaly types remains poorly understood. This study evaluated 11 unsupervised anomaly detection algorithms spanning four paradigms (distance-based, density-based, reconstruction-based, and tree/boundary-based) against six simulated anomaly types embedded in a realistic survey dataset (N = 3,000). Results revealed pronounced differential sensitivity: globally deviant patterns (random, extreme, alternating) were universally detectable, whereas careless and acquiescent responding required reconstruction- or boundary-based methods, and straightline responding resisted detection by all algorithms (maximum area under the receiver operating characteristic curve [AUC-ROC] < .70). No single algorithm dominated across all types. These findings argue for multimethod approaches combining ML algorithms with traditional response quality indicators, and provide a framework for selecting detection methods based on anticipated anomaly types.
The integrity of survey data is fundamental to the validity of research in education, psychology, and the social sciences more broadly. From the perspective of survey response theory (Tourangeau et al., 2000), anomalous response behaviors, such as careless, random, alternating, acquiescent, straightlining, or extreme responding, pose serious threats to the factor structure, reliability, and validity of psychometric instruments (Huang & Cornell, 2015; Meade & Craig, 2012). Even relatively small proportions of careless respondents (5%–10%) can meaningfully distort results: attenuating factor loadings, producing improper or misleading factor structures, inflating Type I or Type II error rates, and degrading model fit in confirmatory analyses (Arias et al., 2020; Woods, 2006). The issue is especially pronounced in web-based studies, where the lack of a controlled testing environment increases the likelihood of low-quality responses (Bowling & Huang, 2018). As large-scale online data collection becomes increasingly common, the development of systematic and robust methods for identifying anomalous survey responses has become an urgent methodological priority.
Traditional approaches to detecting careless responding fall into three broad categories: direct-detection methods, response-pattern indices, and statistical outlier techniques. Direct methods include bogus and instructed-response items designed to flag inattentive participants. Response-pattern indices, such as the longstring index and psychometric synonym/antonym correlations, examine internal response consistency. For example, the longstring index captures response invariability by identifying the longest sequence of identical answers, whereas within-person correlations between synonym or antonym item pairs reveal inconsistency in conceptually related responses (Ulitzsch et al., 2024). Statistical outlier methods, such as the Mahalanobis distance, identify multivariate response patterns that are unusual. Although these approaches are relatively easy to implement, they often struggle to detect diverse and behaviorally nuanced forms of anomalous responding. Moreover, many are scale-specific and show limited generalizability across instruments or datasets (Schroeders et al., 2021; Ulitzsch et al., 2023).
Item response theory (IRT) offers a powerful framework for detecting anomalous survey responses by assessing how well an individual’s observed response pattern aligns with the pattern expected given estimated item parameters. This is typically accomplished through person-fit statistics that quantify probabilistic consistency between observed and model-implied responses (Braeken & van Laar, 2025; Ulitzsch et al., 2023, 2024). Person-fit indices span both nonparametric measures based on response pattern features (e.g., Guttman errors) and parametric, likelihood-based statistics (e.g., standardized log-likelihood indicators; for example, Lz) (Drasgow et al., 1985; Rupp, 2013). More specialized approaches further enhance sensitivity by using likelihood-ratio tests to contrast models of typical versus aberrant responding. To address limitations of traditional person-fit methods, particularly their vulnerability to population heterogeneity, recent mixture-based extensions integrate typical and aberrant response processes within a single model, reducing calibration bias and yielding posterior probabilities that directly quantify an individual’s likelihood of aberrant responding (Braeken & van Laar, 2025). Despite their strong theoretical grounding and clear probabilistic interpretation, these IRT-based approaches remain inherently model-dependent. Violations of model assumptions can undermine detection accuracy, underscoring the value of complementary or more flexible anomaly-detection strategies, such as model-agnostic approaches.
This has sparked interest in machine learning (ML) algorithms for anomaly detection, which offer greater flexibility and the potential to capture complex, multivariate patterns of aberrant responding without strong distributional assumptions or the need to define specific models (model-agnostic). We call this approach machine-learning-based or computational-based. It should be noted that, on one hand, the anomaly detection community has developed dozens of algorithms and conducted extensive benchmarking studies (Campos et al., 2016; Goldstein & Uchida, 2016), but these evaluations are overwhelmingly conducted on network intrusion, fraud detection, and industrial process data.
Recent work has begun to apply these computational approaches directly to survey and assessment response data. Ding (2026) demonstrated that autoencoder neural networks can effectively detect aberrant response patterns in survey instruments, evaluating performance on both simulated and real data and concluding that autoencoders provide researchers with a viable computational tool for response quality screening. Taking a different yet complementary approach, Kandanaarachchi (2022) proposed an unsupervised anomaly-detection ensemble framework grounded in IRT. By mapping IRT’s latent trait continuum to the anomaly detection problem, Kandanaarachchi’s IRT ensemble combines the outputs of multiple heterogeneous detection methods, including the 11 algorithms evaluated in the present study. It uses the IRT model’s discrimination and scaling parameters to downweight noisy, nondiscriminatory methods while accentuating sharper ones. The IRT ensemble was shown to outperform simple averaging, greedy selection, and other combination functions across synthetic data, the ODDS repository, and over 12,000 datasets from the KMASH repository. In the adjacent domain of intelligent tutoring systems (ITS), Gorgun and Bulut (2022) compared six unsupervised anomaly detection methods, including Isolation Forest, Local Outlier Factor (LOF), Gaussian Mixture Models, and Mahalanobis distance, for identifying aberrant responses in the ASSISTment ITS dataset. Their findings are instructive for the present study: most methods converged on similar sets of flagged responses, but LOF yielded markedly divergent results and failed to correlate with convergent validity criteria (students’ affective states), leading the authors to recommend using multiple detection methods in combination. Together, these studies establish that ML algorithms are applicable to the survey and assessment response domain and that both ensemble and multimethod approaches hold particular promise. However, none of these studies examined the differential sensitivity of individual algorithms to specific types of anomalous responding: Ding (2026) focused on a single algorithm (autoencoders), Kandanaarachchi (2022) focused on ensembling algorithms rather than characterizing their individual profiles, and Gorgun and Bulut (2022) compared algorithms in an ITS context without manipulating distinct anomaly types. This gap motivates the present study.
On the other hand, survey data are fundamentally different from the domains where these algorithms were developed. This suggests that the geometry of different anomaly types in the response space interacts with each algorithm’s detection mechanism in distinct ways, motivating a differential-sensitivity framework. For instance, random responding produces response vectors scattered throughout the feature space, so it is expected that nearly any method that measures distance, density, or reconstruction fidelity will capture them. Straightline responding, however, is a particular, low-variance pattern that may not be far from the data centroid; it could sit uncomfortably close to the mean response vector while remaining structurally anomalous. This kind of reasoning extends naturally to ML algorithms for detecting anomalies, each with a different inductive bias about what constitutes anomalous behavior.
The purpose of this study was to evaluate the differential effectiveness of machine-learning-based approaches in detecting different types of behaviorally oriented survey response anomalies. In the following sections, we provide a critical synthesis of 11 ML algorithms used for anomaly detection. These algorithms span four broad paradigms: distance-based, density-based, reconstruction-based, and tree- or threshold-based. For each algorithm, we discuss its theoretical foundations, established strengths and limitations in the general anomaly detection literature, and, where applicable, its relevance and challenges for detecting anomalous responses in survey data. Afterward, we conduct a simulation study to examine the differential sensitivity of these computational-based detection methods to anomaly types.
Distance-Based Methods
KNN-AGG: Aggregated k-Nearest Neighbor Distances
The k-nearest neighbors (KNNs) framework is among the most fundamental approaches to unsupervised anomaly detection. In this paradigm, an observation’s anomaly score is derived from its distance to its neighbors, under the assumption that anomalies are located far from the bulk of the data in the feature space (Ramaswamy et al., 2000). KNN-AGG extends the basic KNN approach by aggregating distances across a range of neighborhood sizes from k minimum to k maximum, rather than relying on a single fixed k value. This aggregation is intended to reduce sensitivity to the choice of k, a parameter that is notoriously difficult to optimize in unsupervised settings.
The strengths of KNN-based methods include their conceptual simplicity, nonparametric nature (no distributional assumptions are required), and their competitive performance across diverse benchmark datasets (Campos et al., 2016). The aggregation strategy in KNN-AGG provides additional robustness by smoothing out the idiosyncratic effects of any single neighborhood size. However, KNN-based approaches suffer from several well-documented limitations. First, they are fundamentally global outlier detectors: they measure how far a point is from the data in an absolute sense, which makes them less effective at identifying local anomalies embedded within varying-density clusters (Breunig et al., 2000). Second, the computational complexity scales quadratically with sample size (O(N2) for naive implementations), though this can be reduced to O(N log N) with KD-tree indexing (Kandanaarachchi et al., 2020). Third, KNN methods are sensitive to feature scales and can be adversely affected by the curse of dimensionality, as distances become less meaningful in high-dimensional spaces. In the context of survey data, which often features discrete ordinal items with a limited range, the distance-based approach may struggle to distinguish genuine response extremity from careless random patterns, particularly when survey items use the same Likert-type scale.
LDOF: Local Distance-Based Outlier Factor
The Local Distance-based Outlier Factor (LDOF; Zhang et al., 2009) was developed to address weaknesses of both global distance-based and local density-based approaches in scattered, real-world datasets. Rather than computing absolute distances or comparing local densities, LDOF measures the relative distance of an observation from its KNNs, capturing how much an object deviates from the local structure of those neighbors. Specifically, it compares the average distance from a point to its neighbors against the average pairwise distance among those neighbors.
The key advantage of LDOF is its reduced sensitivity to the choice of k compared with classical KNN and LOF methods (Zhang et al., 2009). It is also effective in datasets with scattered data points, where implicit data patterns make it difficult to apply density-based methods. However, LDOF’s computational complexity includes an additional O(k2) computation per point beyond the standard KNN search, resulting in an overall cost of O(N log N + Nk2) (Kandanaarachchi et al., 2020). For survey data, the relative distance measure may be advantageous in that it captures how respondents deviate from locally similar response profiles, though its performance on bounded ordinal data typical of Likert-type-scale items remains an open empirical question.
Density-Based Methods
LOF: Local Outlier Factor
The LOF (Breunig et al., 2000) is arguably the most widely cited and influential density-based anomaly detection algorithm. LOF computes the local density of each observation relative to its k-nearest neighborhood and assigns outlier scores by comparing a point’s density to that of its neighbors. An observation surrounded by points of substantially higher density receives a high LOF score, indicating it is anomalous relative to its local context. This local perspective allows LOF to detect anomalies that global methods would miss, for instance, a moderately distant point near a very dense cluster may be anomalous, whereas a similarly distant point in a sparse region may not.
LOF has been widely validated across numerous settings, including network intrusion detection, fraud detection, and classification benchmark datasets (Breunig et al., 2000; Campos et al., 2016). Its ability to handle varying-density distributions is a clear strength for survey data, where different subgroups of respondents may exhibit different patterns of endorsement. However, LOF has notable limitations. Its density estimation assumes approximately spherical local neighborhoods, which may not hold in practice (Tang et al., 2002). The choice of k strongly influences results, and LOF scores lack a consistent interpretation across datasets. A score of 1.1 may indicate an outlier in one setting but not in another (Kriegel et al., 2011). Furthermore, LOF’s accuracy degrades in high-dimensional spaces, and its O(N2) computational cost limits scalability. In large-scale survey studies with many items, both dimensionality and sample-size issues may prove challenging.
COF: Connectivity-Based Outlier Factor
The Connectivity-based Outlier Factor (COF; Tang et al., 2002) was introduced to address a specific limitation of LOF: its reliance on spherical density assumptions. COF incorporates the connectivity structure among data points (i.e., how points are linked through chains of nearest neighbors) into the density estimation. If two points have similar local densities but one is less connected to the broader data structure (i.e., its neighbors are themselves outlying), COF assigns a higher anomaly score to the less-connected point.
This connectivity-aware approach can detect outliers that deviate from nonspherical density patterns, making it theoretically advantageous for datasets with irregularly shaped clusters. In practice, COF has demonstrated improved detection when clusters have varying densities and lie close to each other (Tang et al., 2002). However, COF inherits many of LOF’s limitations, including sensitivity to k, interpretability challenges with the resulting scores, and high computational cost (O(N log N + Nk2)). Furthermore, the connectivity computation adds overhead and complexity. For survey data, COF’s advantage may be modest, given that Likert-type response vectors tend to produce relatively uniform local neighborhoods. However, it can be beneficial when multiple distinct careless response patterns (e.g., long strings, random responding, pattern responding) create nonspherical anomaly distributions.
INFLO: Influenced Outlierness
The Influenced Outlierness (INFLO; Jin et al., 2006) algorithm extends the LOF framework by incorporating each data point’s reverse neighborhood into the density estimation. While LOF considers only the KNNs of a point, INFLO also considers points for which the target observation is among their KNNs (the reverse nearest neighbors). By combining both the direct and reverse neighborhoods, INFLO aims to produce more stable and symmetric density estimates, particularly in regions where cluster boundaries create asymmetric neighborhood structures.
INFLO has shown competitive performance relative to LOF, particularly when datasets contain clusters of substantially different densities (Jin et al., 2006; Tang & He, 2017). Its consideration of reverse neighbors provides a more holistic view of a point’s relationship to the surrounding data. However, the reverse neighborhood computation increases computational cost, and like LOF, INFLO remains sensitive to the choice of k and suffers from the curse of dimensionality. The algorithm’s application in survey research is largely unexplored. However, its symmetric density estimation may be advantageous when the population of interest contains subgroups with distinctly different but legitimate response patterns (e.g., high versus low endorsers on a construct), where standard LOF might incorrectly flag members of minority response groups.
KDEOS: Kernel Density Estimation Outlier Score
KDEOS (Schubert et al., 2014) generalizes the density-based outlier detection framework by replacing the reachability-distance-based density estimation used in LOF with flexible kernel density estimation (KDE). KDEOS computes kernel density estimates for each data point over a range of k-nearest-neighborhood sizes, using a Gaussian kernel with a bandwidth determined by the reachability distances to neighboring observations. The resulting density scores are then normalized using z-scores, facilitating more interpretable, comparable outlier scores.
The use of kernel density estimation provides smoother, theoretically better-grounded density estimates than LOF’s reachability-distance formulation (Schubert et al., 2014). The normalization procedure further addresses the long-standing criticism that density-based outlier scores are difficult to interpret and compare across datasets. However, KDEOS introduces additional parameters (bandwidth and k-value range) that require tuning, and its computational complexity matches that of other KNN-based methods at O(N2). Like other density-based methods, it is less effective in high-dimensional spaces. For survey research applications, the kernel density framework may be susceptible to the discrete and bounded nature of Likert-type-scale responses, as Gaussian kernels assume continuous underlying distributions.
LDF: Local Density Factor
The Local Density Factor (LDF; Latecki et al., 2007) extends the LOF approach by computing local density estimates (LDE) using a Gaussian kernel function rather than the reachability distance. The LDF score for a point is derived from the ratio of the average density of its neighbors to its own density, analogous to LOF but with kernel-based density estimation. This kernel-based approach aims to produce more robust density estimates that are less influenced by individual extreme distances.
LDF generally outperforms LOF when kernel density estimation better captures the underlying data distribution (Latecki et al., 2007). However, it introduces additional complexity through the kernel bandwidth parameter, and its performance can be sensitive to this choice. LDF may not perform well on high-dimensional datasets or when data density is highly skewed, and the accuracy of density estimates is influenced by bandwidth selection. The algorithm requires empirically setting four parameters, which makes tuning challenging in unsupervised settings. For survey data, the continuous-distribution assumptions underlying the Gaussian kernel may be violated by the bounded, discrete structure of Likert-type items, potentially degrading density estimation quality.
Reconstruction-Based Methods
Autoencoders
Autoencoders are neural network architectures consisting of an encoder that compresses input data into a lower-dimensional latent representation and a decoder that reconstructs the original input from this representation. When trained on predominantly normal data, the autoencoder learns to reconstruct typical patterns well but produces high reconstruction errors for anomalous observations that deviate from learned patterns. This reconstruction error serves as the anomaly score (Liu et al., 2023).
Autoencoders offer several advantages for anomaly detection: they can model complex, nonlinear relationships in the data without explicit distributional assumptions; they scale relatively well to high-dimensional datasets; and they naturally perform dimensionality reduction, which can mitigate the curse of dimensionality that hampers distance- and density-based methods. Empirical comparisons have shown that autoencoders are competitive with, or even superior to, traditional methods across many benchmark datasets (Bank et al., 2023). However, autoencoders also present notable challenges. Their performance is sensitive to architectural choices (number of layers, latent dimension, activation functions) and training hyperparameters (learning rate, number of epochs), and there is no principled method for selecting these in unsupervised settings. They may also overfit to noise in the training data or fail to capture the full complexity of normal data variation, leading to false positives. In addition, autoencoders are often treated as “black box” models, which limits their interpretability, a concern in psychometric contexts where understanding why a response pattern is flagged as anomalous is important for substantive decision-making.
In survey research, autoencoders have received limited direct application. A key concern, however, is that autoencoders trained on contaminated data (i.e., data that already contains careless responses) may learn to partially reconstruct anomalous patterns, thereby reducing their sensitivity.
Boundary-Based Methods
OCSVM: One-Class Support Vector Machine
One-class support vector machines (OCSVM; Schölkopf et al., 2001) adapt the classical SVM classification framework for anomaly detection by treating all available data as belonging to a single class and learning a decision boundary that separates this data from the origin in a kernel-mapped feature space. New observations falling outside this boundary are classified as anomalies. The use of kernel functions (typically Radial Basis Functions) enables OCSVM to model complex, nonlinear decision boundaries.
OCSVM is theoretically well-founded and benefits from the robust optimization framework of SVMs. It does not require labeled anomaly examples for training and can handle nonlinear relationships through the kernel trick. However, OCSVM has several important limitations. First, its training complexity is O(N2) to O(N3), making it impractical for large datasets without approximation techniques. Second, OCSVM is highly sensitive to hyperparameter choices, such as the kernel type, kernel coefficient (gamma), and the regularization parameter (nu), and optimizing these in unsupervised settings is challenging, as performance can degrade substantially with poor parameter choices (Thomas et al., 2015). Third, while kernel mapping can handle high-dimensional data, the curse of dimensionality can still degrade generalization when the number of features is large relative to the sample size (Erfani et al., 2016).
In survey research contexts, OCSVM faces additional challenges. The discrete, bounded nature of Likert-type-scale data may not map well to the continuous feature spaces assumed by kernel functions. Furthermore, as Schroeders et al. (2021) noted, the data quality and structure in educational and psychological research are frequently not comparable to those in domains (e.g., network intrusion detection, image classification) where OCSVM has been most successful. Survey responses are inherently subject to measurement error, and careless responding manifests in idiosyncratic ways that may not form a coherent region outside the OCSVM’s learned boundary.
Tree-Based and Threshold-Based Methods
Isolation Forest
Isolation Forest (Liu et al., 2008) takes a fundamentally different approach to anomaly detection by exploiting the principle that anomalous points are, by definition, few and different, and therefore easier to isolate through random partitioning. The algorithm constructs an ensemble of randomized binary trees (isolation trees) that recursively partition the feature space using random splits. Anomalies, being sparse and distant from normal clusters, require fewer splits to be isolated and thus have shorter average path lengths across the ensemble.
Isolation Forest offers several notable advantages. It is computationally efficient, with linear time complexity O(N), and has lower memory requirements than distance- and density-based methods. It does not require distance computations, making it less susceptible to the curse of dimensionality than KNN-based approaches. Empirical evaluations have shown that Isolation Forest produces among the lowest failure rates across diverse benchmark datasets (Emmott et al., 2015). However, Isolation Forest has known weaknesses: it can struggle with datasets where anomalies are not globally isolated but rather are local anomalies within dense clusters; its random axis-aligned splits may be inefficient for detecting anomalies in datasets with correlated features; and the masking effect of large numbers of irrelevant features can influence its anomaly scores.
In survey research, Isolation Forest’s computational efficiency makes it attractive for large-scale data collection efforts. Its random partitioning approach may be well-suited to detecting globally aberrant response patterns (e.g., all-same responses, random responses that produce extreme feature values). However, its difficulty with local anomalies may be a potential weakness, since careless respondents may not always produce globally extreme patterns but may instead appear anomalous only relative to their local subgroup.
Stray: Search and Trace Anomaly Detection
The Stray algorithm (Talagala et al., 2021) employs extreme value theory (EVT) to provide a principled, data-driven threshold for anomaly detection. It defines an anomaly as an observation whose KNN distance with the maximum gap is significantly different from what would be expected under the Gumbel distribution, the maximum domain of attraction for the distribution of maximum neighbor gaps. This theoretical foundation allows Stray to automatically determine a significance threshold for declaring observations as anomalous, rather than relying on user-specified contamination rates.
Stray was developed to improve on the HDoutliers algorithm, addressing limitations in computational efficiency and detection accuracy in high-dimensional settings (Talagala et al., 2021). Its key strengths include: a strong theoretical grounding in EVT that provides statistical guarantees on detection performance; the ability to handle high-dimensional data through feature engineering; and implementation in an open-source R package that facilitates reproducibility. However, Stray’s performance can be variable across datasets. For instance, in additive manufacturing applications, its F1 score ranged from 0.09 to 0.92, suggesting sensitivity to data characteristics. Its reliance on the Gumbel distribution assumption may not hold in all settings, and like other KNN-based methods, it is affected by the curse of dimensionality.
In the context of survey research, Stray’s automatic threshold determination is particularly appealing because it eliminates the need for researchers to specify an expected contamination rate a priori, a parameter that is typically unknown. The EVT framework also provides a natural way to control the false discovery rate, which is important when decisions about data exclusion affect study validity and statistical power.
Cross-Cutting Themes and Considerations for Survey Research
Several themes emerge from this review that warrant attention when applying ML anomaly detection algorithms to survey data. First, the curse of dimensionality is a pervasive challenge. Surveys in educational and psychological measurement and social sciences often include dozens to hundreds of items, creating high-dimensional response vectors. While this high dimensionality is a strength for capturing nuanced response patterns, it degrades the performance of distance- and density-based methods where distances become increasingly uniform and less informative.
Second, survey data possess characteristics that distinguish them from the domains where most anomaly detection algorithms were developed and validated (e.g., network security and industrial process monitoring). Likert-type-scale responses are discrete, bounded, and ordinal rather than continuous, which violates the assumptions underlying many kernel-based methods (KDEOS, LDF, and OCSVM) and reconstruction-based approaches (autoencoders). The inherent measurement error in psychological assessment further complicates detection, as regular responding may exhibit substantial variability that mimics aspects of carelessness (Schroeders et al., 2021).
Third, the unsupervised nature of anomaly detection in survey research poses fundamental methodological challenges. Unlike network intrusion or fraud detection, where labeled examples of anomalies are sometimes available for validation, survey researchers typically lack a gold standard for identifying careless respondents. This makes model selection, hyperparameter tuning, and performance evaluation particularly difficult. As Schroeders et al. (2021) cautioned, supervised ML approaches for careless responding detection require labeled training data, extensive training phases, and are bound to specific item sets, limiting generalizability. The unsupervised methods reviewed here avoid this limitation but sacrifice the ability to leverage known examples of carelessness.
Fourth, the interpretability of anomaly scores varies markedly across methods. LOF and its variants produce relative scores that are difficult to compare across datasets, whereas Stray’s EVT-based threshold provides a principled significance cutoff. Autoencoders produce reconstruction errors that have an intuitive interpretation (i.e., how well the model reproduces the response pattern) but lack a theoretically grounded decision rule. For applied researchers in educational and psychological measurement, the interpretability and statistical properties of the detection criterion are critical considerations.
Fifth, computational scalability is relevant to large-scale assessments. Methods with quadratic or cubic complexity (LOF, COF, INFLO, KDEOS, LDF, LDOF, and OCSVM) may become prohibitive for datasets with tens of thousands of respondents, whereas Isolation Forest (linear complexity) and Stray (with an efficient nearest-neighbor search) offer better scalability. The autoencoder’s computational cost depends heavily on architecture, but is generally manageable with modern hardware.
Finally, an emerging perspective (Alfons et al., 2024) calls for open science practices in ML-based detection of careless responding. The present study contributes to this goal by providing a systematic evaluation of 11 algorithms for differential sensitivity to various anomaly types under controlled conditions relevant to measurement in education, psychology, and the social sciences. Whereas Ding (2026) established the viability of a single reconstruction-based method (autoencoders) for survey anomaly detection, Kandanaarachchi (2022) demonstrated the value of ensembling heterogeneous detectors via IRT, and Gorgun and Bulut (2022) compared multiple unsupervised methods in an ITS context, the present study fills a gap left by these contributions by characterizing the individual detection profile of each algorithm across six behaviorally distinct anomaly types in a controlled simulation. This differential-sensitivity perspective is necessary for informed algorithm selection and principled ensemble design: one must first understand what each algorithm detects (and misses) before combining them effectively.
In the following sections, we conducted a simulation study to examine the differential sensitivity to various anomaly types under controlled conditions. It should be noted that we are particularly interested in which algorithms are differentially sensitive to which anomaly types. Thus, the anomaly type is our primary manipulation in the simulation study. Although anomaly rate and number of items can be manipulated, they are secondary conditions that would tell us whether those differential sensitivities shift under different circumstances, an interesting question, but a separate one. Trying to answer everything in this study dilutes the message and creates an overwhelming number of conditions that are difficult to present clearly.
Simulation Study
Simulation Design
We generated a synthetic dataset of N = 3,000 respondents answering 85 five-point Likert-type items (1 = strongly disagree to 5 = strongly agree). Of these, 2,700 respondents (90%) were simulated as normal (attentive) respondents, and 300 (10%) were simulated with one of six anomalous response styles described below. The simulation was implemented in Python using NumPy and SciPy with a fixed random seed for full reproducibility. The complete code for the generated data file is available in Supplemental Materials.
Normal Response Generation
The normal response generation process consisted of four stages: (a) latent trait generation from a Gaussian mixture model, (b) nonlinear transformation of latent traits, (c) mapping latent traits to continuous item responses through six distinct functional forms, and (d) ordinal categorization via item-specific cutpoints. The complete simulation code is available in Supplemental Materials; the mathematical specification of each stage is detailed below.
Latent Trait Generation
Three continuous latent traits (F1, F2, F3) were drawn for each respondent from a three-component Gaussian mixture model with mixing proportions π = (.55, .30, .15). The component-specific parameters were as follows: Component 1 (55%): μ1 = (0.0, 0.0, 0.0) with a covariance matrix having unit or near-unit variances (1.0, 0.9, 0.8) and moderate correlations (r12 = .20, r13 = .00, r23 = .10). Component 2 (30%): μ2 = (1.5, −0.8, 0.5) with lower variances (0.8, 0.7, 0.6) and stronger correlations (r12 = .30, r13 = .10, r23 = .20). Component 3 (15%): μ3 = (−1.2, 1.2, −0.7) with variances (0.9, 0.8, 0.9) and selective correlations (r12 = .00, r13 = .20, r23 = .20). These parameters were chosen to produce a heterogeneous latent population with subgroups differing in both means (shifted by approximately 1–1.5 SD) and covariance structure, ensuring that the simulated data do not conform to a single multivariate normal distribution.
Nonlinear Transformation
After drawing the latent traits from the mixture, a sinusoidal warping was applied to the first two dimensions to induce nonlinearity that violates standard factor-analytic linearity assumptions. Specifically, letting θ = 0.8 × (F1 −F2), the transformed traits were computed as F1* = F1 + 0.6 sin(θ), F2* = F2 + 0.4 cos(θ), and F3* = F3 (unchanged). The sinusoidal amplitudes (0.6, 0.4) and coupling coefficient (0.8) were chosen to produce moderate nonlinearity—sufficient to violate linear factor models but not so extreme as to destroy the latent structure.
Item–Factor Assignment Structure
The 85 items were organized into six groups, each defined by a distinct functional form mapping latent traits to continuous item responses. The groups and their primary factor loadings were: Group 0 (Items 1–15, 15 items) loaded on F1* via a Gaussian copula with nonnormal marginals; Group 1 (Items 16–30, 15 items) loaded on F1* and F2* through a bimodal mixture with nonlinear item transforms; Group 2 (Items 31–45, 15 items) loaded on F1* and F3* via sigmoid, sine, and polynomial link functions; Group 3 (Items 46–59, 14 items) loaded on F2* and F3* with interaction effects and heteroscedastic noise; Group 4 (Items 60–72, 13 items) loaded on F1* and F2* with boundary-inflated cutpoints; and Group 5 (Items 73–85, 13 items) loaded on F3* with autoregressive residual dependencies. Across the full instrument, F1* was involved in 58 items (Groups 0, 1, 2, 4), F2* in 42 items (Groups 1, 3, 4), and F3* in 42 items (Groups 2, 3, 5). Groups 1 to 4 featured cross-loadings involving two factors and their interaction, producing a multidimensional structure.
Item Response Generation Models
Continuous latent responses were generated using a behaviorally oriented data-generating mechanism (Tourangeau et al., 2000) designed to produce realistically complex data (nonnormal marginals, nonlinear relationships, heteroscedasticity, serial dependence), ensuring that the simulation does not artificially favor any particular detection method. The exact mathematical model for each item group is as follows. In Group 0, each item’s continuous response was generated via a Gaussian copula: correlated uniform variates were produced from a 15-dimensional normal with AR(1)-like correlation (ρ = 0.5, with random jitter σ = 0.04), then transformed through item-specific marginal quantile functions randomly assigned as log-normal (40%), Cauchy (20%), gamma (20%), or logit-beta (20%). The copula output was tied to the global factor as Yj = Xj(copula) + 0.5 ×F1*. In Group 1, a bimodal latent composite was formed as Fbim = 0.7F1*+ 0.9F2*+ε, ε∼N(0, 0.6), shifted by +1.2 or −1.3 according to a Bernoulli component (55%/45%), with per-item loadings aj∼ U(0.6, 1.1) and error ε∼N(0, 0.8); selected items received quadratic (0.9x+ 0.15x2/3) or sinusoidal (0.8x+ 0.4 sin(x)) transformations. In Group 2, a latent predictor was computed as z = w1×F1*+w2×F3*+ 0.2 ×F1*×F3* (with w1, w2∼N(0.9, 0.3)), then mapped through cycled link functions: 2σ(z) − 1 (sigmoid), sin(z) + 0.3z (sine-linear), or 0.6z+ 0.2z2– 0.05z3/3 (polynomial), plus ε∼N(0, 0.4). In Group 3, each item was generated as z = a1×F2*+a2×F3*+ 0.5 ×F2*×F3* (with a1, a2∼ U(0.4, 1.0)) plus heteroscedastic noise with σ = 0.5 + 0.35(|F2*| +|F3*|). In Group 4, a shared latent score Zs = 0.7F1*– 0.5F2*+ε (ε∼N(0, 0.7)) was computed, and each item’s response was Zs +ε (ε∼N(0, 0.5)); boundary inflation in this group was achieved through narrower ordinal cutpoints (see below). In Group 5, an AR(1) residual process was used: Xj = 0.6 ×Xj–1+Ej (with Ej∼N(0, 1)), with a mild cubic skew transformation: Yj = 0.9Xj+ 0.1Xj3/3+ 0.4 ×F3*.
Ordinal Categorization
Continuous responses were discretized into five ordinal categories (1–5) using four item-specific cutpoints. Cutpoints were not fixed across items; instead, each item received unique cutpoints generated as cjk = δj+γj×bk, where b = (−1.4, −0.5, 0.5, 1.4) is a symmetric base template, δj∼N(0, 0.35) is an item-specific location shift (controlling item difficulty), and γj∼ U(0.8, 1.3) is an item-specific spread multiplier (controlling discrimination). For Group 4 (boundary-inflated items), γj was fixed at 0.6, producing narrower inner cutpoint spacing and thus more responses at scale endpoints 1 and 5. For reverse-keyed items (approximately 25% of items, with at least one per group), cutpoints were negated and reversed: cjk(rev) = −cj,5–k. The resulting cutpoints varied across items: the first cutpoint ranged from −2.29 to −0.30 (M = −1.32, SD = 0.48) and the fourth from 0.49 to 2.48 (M = 1.43, SD = 0.45), producing heterogeneous item characteristics.
Post hoc item analysis on the 2,700 normal respondents yielded a mean difficulty of −0.20 (SD = 0.48) and a mean corrected item–total correlation of .48 (SD = 0.22), indicating adequate variability in item characteristics.
Anomalous Response Styles
Three hundred respondents were randomly assigned to one of the six anomalous response styles, with proportions reflecting approximate prevalence in the survey literature: acquiescence (n = 72, 24%), extreme responding (n = 60, 20%), careless responding (n = 60, 20%), random responding (n = 48, 16%), straightlining (n = 36, 12%), and alternating (n = 24, 8%). All anomaly mechanisms were applied as post hoc modifications to valid, content-driven responses, ensuring that anomalies are defined in terms of response behavior rather than deviations from a specific measurement model. Each anomalous respondent received a continuous severity parameter drawn from a Beta(2, 2) distribution, clipped to the range .15 to .95, to control the intensity of the relevant mechanism (see Table 1).
Summary of Anomalous Response Styles, Sample Sizes, and Severity-Dependent Parameters.
Note. sev = severity parameter drawn from Beta(2, 2) clipped from .15 to .95. All anomaly mechanisms were applied as post hoc modifications to attentive ordinal responses.
Acquiescence
With probability .30 + .50 × severity, each response was shifted toward agreement (incremented for positively keyed items, decremented for reverse-keyed items), making acquiescence keying-direction-aware and content-independent, consistent with its theoretical characterization (Paulhus, 1991). The shift was implemented as Y ← min(5, Y + b) for positively keyed items and Y ← max(1, Y − b) for reverse-keyed items, where b was 1 (with probability .70) or 2 (with probability .30). Consequently, responses already at the agreement-side endpoint of the scale (5 for positively keyed items, 1 for reverse-keyed items) were unaffected by the manipulation, whereas responses at the disagreement-side endpoint (1 for positively keyed items, 5 for reverse-keyed items) were shifted toward agreement, consistent with the theoretical conceptualization of acquiescence as a directional pull toward the agreement pole regardless of item content.
Extreme Responding
Nonextreme responses (2, 3, or 4) were pulled toward the nearest scale endpoint with probability .30 + .60 × severity, preserving the original response direction for nonmidpoint categories.
Careless Responding
Careless responding was conceptualized as intermittent inattention with temporal structure, distinct from both pure random responding (which targets a scattered subset of items independently) and straightlining (which produces constant-value endorsement across page-blocks). Careless respondents attend to some items and disengage from others, with periods of disengagement tending to cluster temporally (Meade & Craig, 2012). To capture this pattern, a two-state Markov chain governed transitions between attentive and inattentive states across the sequence of items. Each respondent began in the attentive state. At each subsequent item, transition probabilities were P(attentive → inattentive) = .05 + .25 × severity and P(inattentive → inattentive) = .60 + .30 × severity, with the complementary probabilities governing the reverse transitions. Higher severity, therefore, produced both more frequent disengagement and longer dwell times in the inattentive state, generating contiguous runs of low-quality responses interspersed with attentive responses. While in the inattentive state, the response was either drawn uniformly at random from (1, . . ., 5) (with probability .60) or set equal to the previous item’s response (perseveration; with probability .40), reflecting empirical evidence that careless respondents show a mixture of random key-pressing and stuck repetition rather than a single behavior (Meade & Craig, 2012).
Random Responding
Random responders replaced a proportion of their items with responses drawn uniformly from 1 to 5. The proportion of affected items was .40 + .60 × severity, ranging from approximately 40% to nearly 100% of items. Items not affected retained their original attentive responses.
Straightlining
Straightlining was implemented at the page-block level (six pages total), reflecting evidence that straightlining typically occurs within screen pages (e.g., Zhang & Conrad, 2014). The number of affected pages was determined by rounding severity × 6. On each affected page, all items received a single constant response (selected uniformly from 1 to 5), with approximately 10% deviating by ±1.
Alternating Pattern Responding
Alternating responding was implemented as a contiguous run of items in which the respondent cycled between scale endpoints (e.g., 1, 5, 1, 5, . . .), reflecting the substantive characterization of this response style as a sustained zigzag pattern rather than as scattered, isolated zigzag values interspersed with valid responses. The length of the affected block was determined by max(4, round (severity × 85)), ensuring a minimum run of four consecutive items and scaling up to the full instrument at the highest severity. The starting position of the block was drawn uniformly from the eligible range. Within the block, items at even relative positions were assigned the value 1, and items at odd relative positions were assigned the value 5, with each value swapped to the opposite endpoint with probability .10 to introduce stochastic noise into the pattern. Items outside the block retained their original attentive responses.
Table 1 summarizes the six anomalous response styles, their sample sizes, and the severity-dependent parameters governing each mechanism.
Analytic Design
The analytic design comprised four sequential phases: (a) data preprocessing, (b) algorithm implementation with parameter tuning, (c) anomaly threshold determination, and (d) performance evaluation. All analyses were conducted in Python 3.12 using scikit-learn (Pedregosa et al., 2011), SciPy (Virtanen et al., 2020), NumPy (Harris et al., 2020), and PyTorch (Paszke et al., 2019). Table 2 summarizes each algorithm’s key parameters, tuning procedure, and final configuration. All codes are available in Supplemental Materials.
Summary of Algorithm Configurations and Threshold Strategies.
Note. CV = 5-fold stratified cross-validation. k = neighborhood size. For OCSVM and Isolation Forest, a sensitivity analysis was conducted by varying the contamination parameter across .05, .10, .15, .20, and .25. The architecture notation for the autoencoder denotes the number of units per hidden layer (encoder and decoder combined). The k parameter for neighborhood-based methods was selected by maximizing LOF area under the receiver operating characteristic curve (AUC-ROC) across k∈ (5, 10, 15, 20).
Data Preprocessing
Two scaling transformations were applied depending on algorithmic requirements. For the eight neighborhood-based algorithms and Stray, item responses were z-score standardized to ensure equal contribution to Euclidean distances. For the autoencoder, responses were rescaled to [0, 1] via min–max normalization, compatible with its Sigmoid output layer. OCSVM and Isolation Forest received the standardized data.
Algorithm Implementation and Parameter Tuning
Eleven unsupervised anomaly detection algorithms spanning four paradigms were implemented (see Table 2). All produced continuous anomaly scores per respondent; higher scores indicate greater anomalousness.
Neighborhood-Based Methods
Seven algorithms (KNN-AGG, LOF, COF, INFLO, KDEOS, LDF, and LDOF) share a neighborhood size parameter k. We selected k = 20 from candidates of 5, 10, 15, and 20 by maximizing LOF AUC-ROC against the true anomaly labels; this label-informed tuning was used solely for hyperparameter selection, not model fitting. A shared KNN structure (Euclidean distance) was pre-computed once and reused across all seven methods, ensuring that performance differences reflect scoring logic rather than inconsistent neighborhood definitions. Each algorithm then applied its own scoring mechanism: aggregated neighbor distances (KNN-AGG), local reachability density ratios (LOF), minimum spanning path costs (COF), reverse-neighborhood influence sets (INFLO), kernel density estimates (KDEOS, LDF), or relative neighbor-distance ratios (LDOF).
Autoencoder
A denoising autoencoder was implemented in PyTorch with a symmetric architecture (D→ 128 → 32 → 8 → 32 → 128 →D, where D = 85), selected from four candidates via 5-fold stratified cross-validation. The encoder used ReLU activations with 10% dropout; the decoder terminated with a Sigmoid activation. The bottleneck dimension of eight compressed the input approximately 10:1, forcing the network to learn only dominant patterns of normal responding. Critically, the autoencoder was trained exclusively on normal cases within each cross-validation fold (semi-supervised design). During training, 15% of item values were randomly replaced with the neutral midpoint as a denoising corruption (Vincent et al., 2010). Training used the Adam optimizer (learning rate = .001, batch size = 128) with early stopping (patience = 12 epochs, maximum = 500). Anomaly scores were defined as the per-respondent mean squared reconstruction error on held-out test folds.
One-Class Support Vector Machine
OCSVM was implemented using scikit-learn’s OneClassSVM with an RBF kernel (γ = “auto,” that is, 1/p, selected via 5-fold cross-validation over “scale”). The ν parameter was treated as a sensitivity parameter rather than a fixed value (see “Threshold Determination” section).
Isolation Forest
Isolation Forest was implemented using scikit-learn with n = 200 estimators, selected from 100, 200, and 300 by 5-fold cross-validation. The contamination parameter was treated as a sensitivity parameter. Anomaly scores were defined as negated decision function values.
Stray
Stray (Talagala et al., 2021) uses EVT to establish a principled anomaly threshold and to fit a Gumbel distribution to the maximum gaps in each observation’s sorted k-nearest-neighbor distances, yielding a p-value for each case and a continuous anomaly score (- log (p) and a principled binary classification (see below).
Cross-Validation Strategy
The eight neighborhood-based methods and Stray are transductive (scoring the full dataset without fitting an explicit model), so their scores were computed on the complete sample. The autoencoder, OCSVM, and Isolation Forest are inductive and were evaluated using 5-fold stratified cross-validation; final anomaly scores are the concatenation of out-of-fold predictions. For the autoencoder, only normal cases from each training fold were used for model fitting.
Anomaly Threshold Determination
We adopted a hybrid threshold strategy. For the eight neighborhood-based algorithms and the autoencoder, respondents with anomaly scores above the 97.5th percentile of the anomaly score distribution were classified as anomalous (for the autoencoder, this percentile was computed from training-set reconstruction errors for normal cases). We selected this threshold as a transparent, reproducible default rather than a theoretically optimal cutoff. We acknowledge a fundamental limitation of any fixed-percentile threshold: it is inherently relative to the data distribution and therefore does not adapt to the true contamination rate. By construction, a 97.5th percentile cutoff flags approximately 2.5% of respondents, regardless of the number of true anomalies. When contamination is below 2.5%, the threshold forces false positives; when contamination substantially exceeds 2.5% (as in the present simulation, where the true rate was 10%), the threshold imposes a hard ceiling on recall. We did not have access to a more defensible alternative for unsupervised use that does not require knowledge of the contamination rate. The fixed-percentile cutoff is therefore best understood as a conventional default analogous to a two-tailed significance level (α = .05), useful for transparent reporting but not optimal for any specific data set. To mitigate this limitation, we report AUC-ROC as our primary metric throughout the Results, as AUC-ROC evaluates discriminative capacity across the full range of possible thresholds and is therefore not affected by the choice of any single cutoff. The threshold-dependent metrics (detection rate and F1) should be interpreted in light of the constraints described above. For Stray, we used its native EVT-based threshold (p < .05; Talagala et al., 2021). For OCSVM and Isolation Forest, which classify observations with negative decision function values as anomalous, the decision boundary location depends on the contamination parameter (ν for OCSVM; contamination rate for Isolation Forest). Because this proportion is unknown in practice, we conducted a sensitivity analysis across .05, .10, .15, .20, and .25.
Performance Evaluation
Performance was evaluated at two levels. (1) Overall performance was assessed via AUC-ROC (primary metric), average precision, F1 at the algorithm-specific threshold, and specificity. (2) Differential sensitivity was assessed for each algorithm × anomaly type combination using type-specific AUC-ROC (computed on subsets containing only normal respondents and a single anomaly type), type-specific detection rate (recall at threshold), and type-specific F1. These metrics were arranged in algorithm × anomaly type matrices to visualize differential sensitivity patterns.
Results
Results are organized in four sections. First, we report overall detection performance across all anomaly types combined. Second, we present the central findings on differential sensitivity, the degree to which algorithms vary in their ability to detect specific anomaly types. Third, we conduct a sensitivity analysis to examine how the contamination parameter affects detection with OCSVM and Isolation Forest. Fourth, we report inter-algorithm agreement to assess convergence and method complementarity. Fifth, we report the Monte Carlo error from the single-replication design.
Overall Detection Performance
Figure 1 presents overall performance metrics for all 11 algorithms. Evaluated by AUC-ROC, the primary, threshold-free metric, all algorithms except Stray (.47) achieved AUC-ROC values above .83, with LOF (.95), LDOF (.95), INFLO (.94), LDF (.95), Autoencoder (.94), and Isolation Forest (.94) clustered at the top. OCSVM (.91), KNN-AGG (.92), KDEOS (.93), and COF (.83) also demonstrated strong overall discrimination. This generally high level of discriminative capacity across paradigms indicates that all algorithms except Stray can, in principle, distinguish anomalous from normal response patterns when evaluated across the full range of possible thresholds.
Overall performance of 11 algorithms across anomaly types using the ROC.
When evaluated at algorithm-specific thresholds, however, the picture diverged substantially, reflecting the fundamental tension between threshold conservatism and detection power (see Figure 2). The seven transductive algorithms using the 97.5th percentile threshold each flagged exactly 75 respondents (4.2% of the 1,800 cases), limiting their maximum achievable recall to .25 (75 out of 300 true anomalies). The autoencoder, also using the 97.5th percentile but computed on training-set reconstruction errors from normal cases, flagged 329 respondents—close to the true anomaly count of 300—and achieved the best balance of precision and recall. OCSVM and Isolation Forest, using their learned decision boundaries (ν = .25 and contamination = .25, respectively), flagged 771 and 629 respondents, achieving high recall at the cost of many false positives. Stray, using its EVT-based threshold, flagged 158 respondents.
Number of anomalies flagged by the algorithm.
Average precision (AP), which accounts for the precision–recall trade-off across thresholds, largely mirrored the AUC-ROC rankings (see Figure 1). Among algorithms using the conservative 97.5th percentile threshold, several—including LOF, INFLO, LDF, and LDOF—achieved perfect or near-perfect specificity (1.00), meaning every respondent they flagged was a true anomaly. However, these algorithms’ F1 scores were constrained to approximately .40 by the low recall ceiling imposed by their thresholds. The autoencoder achieved a notably higher F1 (approximately .78), benefiting from both high precision and substantially higher recall than the neighborhood-based methods.
Differential Sensitivity to Anomaly Types
The central question of this study is whether algorithms differ in their sensitivity to specific types of anomalous responding. Panel A in Figure 3 presents type-specific AUC-ROC values (one anomaly type vs. normal respondents), and Panel B in Figure 3 presents detection rates at each algorithm’s threshold. Together, these results reveal pronounced differential sensitivity patterns that map onto the theoretical mechanisms underlying each algorithmic paradigm.
Panel A: ROC value by algorithm and anomaly type. Panel B: Detection rate by algorithm and anomaly type.
Random Responding
Random response patterns were the most universally detectable anomaly type (see Figure 4, bottom-left panel). All algorithms except Stray achieved AUC-ROC ≥ .989 for random-versus-normal discrimination (Panel A in Figure 3), and the autoencoder, OCSVM, and Isolation Forest each detected 100% of random respondents at their respective thresholds (Panel B in Figure 3). Even the neighborhood-based algorithms with only 75 flags achieved detection rates of .42–.52. This near-universal detectability is theoretically expected: random responding produces item responses drawn from a uniform distribution, placing these cases far from the structured covariance patterns of normal respondents in multivariate space. Stray was the sole exception (.04 detection rate), as its gap-based EVT mechanism is insensitive to globally distant points.
Algorithm ranking by anomaly type.
Extreme Responding
Extreme response patterns (consistent endorsement of scale endpoints) were also highly detectable, with AUC-ROC values exceeding .98 for nine of 11 algorithms (Panel A in Figure 3; range: .534 for Stray to 1.000 for OCSVM and several density-based methods). Detection rates at threshold varied widely—from .03 (Stray) to 1.00 (OCSVM), reflecting the differential numbers of flags permitted by each algorithm’s threshold rather than its discriminative capacity. Among the neighborhood-based methods, LDOF achieved the highest detection rate (.55), while the autoencoder detected 95% of extreme cases and Isolation Forest 97% (Panel B in Figure 3). This strong performance across paradigms is consistent with the expectation that extreme responses generate vectors located at the periphery of the item response space, detectable by distance, density, reconstruction, and boundary-based mechanisms alike.
Careless Responding
Careless response patterns (random deviations from an otherwise valid profile) were well-discriminated by most algorithms at the AUC-ROC level (range: .885–.987 for 10 of 11 algorithms; .404 for Stray). However, detection rates at the 97.5th percentile threshold were modest for neighborhood-based methods (.07–.20), as more extreme anomaly types consumed their limited flag budget. The autoencoder achieved the highest detection rate (.90), followed by OCSVM and Isolation Forest (both .87; Panel B in Figure 3). The autoencoder’s strong performance on careless patterns aligns with its reconstruction-based logic: careless deviations disrupt item-level covariance patterns learned from normal data, producing elevated reconstruction error even when the deviations are modest in magnitude.
Acquiescence
Acquiescent responding (systematic endorsement of agree-side responses regardless of item content) was detectable at the AUC-ROC level, though with more variation than other types. AUC-ROC values ranged from .394 (Stray) to .987 (LDF), with most neighborhood-based methods achieving .88 − .98 (Panel A in Figure 3). However, detection rates at threshold (Panel B in Figure 3) revealed a sharp divide: the autoencoder (.76), OCSVM (.76), and Isolation Forest (.74) detected roughly three-quarters of acquiescent respondents, while all seven neighborhood-based methods detected 0–6%. This discrepancy between high AUC-ROC and low detection rate for the neighborhood-based methods reflects the threshold constraint: acquiescent cases scored below the 97.5th percentile cutoff because they were less extreme than random or alternating cases that dominated the top of the score distribution. The autoencoder’s strong acquiescence detection is notable, as it suggests that the deep network learned fine-grained covariance patterns disrupted by systematic agree-side endorsement.
Straightline Responding
Straightline response patterns (identical responses across all items) produced the most challenging detection problem. AUC-ROC values were uniformly moderate (Panel A in Figure 3), ranging from .580 (Stray) to .683 (KNN-AGG) for neighborhood-based methods, with OCSVM at .598, Isolation Forest at .674, and the autoencoder at .613. No algorithm achieved an AUC-ROC above .70 for straightline discrimination. Detection rates at threshold (Panel B in Figure 3) were correspondingly low: OCSVM achieved the highest rate (.44), followed by Isolation Forest (.42), the autoencoder (.22), and the neighborhood-based methods (.06–.14). Stray detected only 11% of straightline cases. This is a notable finding: straightline respondents endorse a single response option for all items, producing vectors that lie along the diagonal of the item space. These vectors may not be far from the normal data cloud in Euclidean distance (particularly if the endorsed value is near the scale midpoint), making them difficult to detect by any distance-, density-, or reconstruction-based method. The uniform difficulty across algorithmic paradigms suggests that straightline detection may require purpose-built indicators (e.g., within-person variance) rather than general-purpose anomaly detection algorithms.
Alternating Responding
Alternating response patterns (cycling between low and high responses) were detected with near-perfect AUC-ROC (1.000) by the autoencoder, OCSVM, Isolation Forest, and, notably, by several neighborhood-based methods, including KNN-AGG, LOF, KDEOS, and LDF (Panel A in Figure 3). This high discrimination translated into strong detection rates (Panel B in Figure 3): the autoencoder, OCSVM, and Isolation Forest achieved 100% detection, while KNN-AGG and KDEOS each detected 88%, LOF 71%, and LDF 67%. Stray was the only method to perform poorly (.08 detection, .555 AUC). The strong performance of neighborhood-based methods on alternating patterns is theoretically informative: alternating response vectors occupy a distinctive region of multivariate space (high variance, negative pairwise correlations), making them globally distant from normal respondents, precisely the type of anomaly that distance- and density-based methods are designed to detect.
Summary
Across anomaly types, three broad patterns of differential sensitivity emerged (see Figure 4 for ranked comparisons). First, globally deviant patterns (random, extreme, and alternating responding) were detectable by most algorithms, as they produced response vectors that were distant from the normal data distribution across multiple metrics. Notably, alternating patterns were detectable even by neighborhood-based methods, distinguishing them from other structurally anomalous types. Second, moderate-difficulty patterns (careless responding and acquiescence) were effectively detected by the autoencoder, OCSVM, and Isolation Forest, but poorly detected by the neighborhood-based methods under the conservative 97.5th percentile threshold. The high AUC-ROC values for these types across paradigms suggest that the detection failures of neighborhood-based methods are primarily threshold artifacts rather than a discriminative limitation. Third, straightline responding emerged as the most difficult anomaly type, with no algorithm achieving an AUC-ROC above .70, underscoring the limitations of general-purpose anomaly detectors for patterns that are structurally incoherent but not necessarily distant in multivariate space.
Assessing Stability
To assess whether the complex generative process (three-component Gaussian mixture, sinusoidal warping, Gaussian copula with nonnormal marginals, heteroscedastic interactions, and AR(1) residual dependencies) is itself a source of instability, the specific concern about complex simulations of this kind, we conducted a targeted Monte Carlo stability check on the data-generating process by running the data-generating process 30 times, each time with a different random seed. Each seed produces a completely independent dataset. We recorded the across-replication variability of the key distributional and structural characteristics of the generated data. By comparing the 30 resulting datasets, we found that the data-generating process itself is stable. Specifically, the mixture-component allocation rates recovered their targets closely (M = .551, .298, .151 against targets of .55, .30, .15; all across-replication SDs < .01), the overall response-category distribution was stable (across-replication SDs = 0.5–1.5 percentage points per category), the top five eigenvalues of the item correlation matrix had coefficients of variation between 2.2% and 5.7%, and the mean severity values across anomaly styles were stable at M≈ .50 with across-replication SDs of .02–.05.
Figure 5 presents these stability diagnostics visually: item means and corrected item–total correlations are plotted with ±1 SD across-replication error bars (top panels), the top-10 eigenvalues of the item correlation matrix show tight clustering across seeds (bottom-left panel), and mean severity per anomaly style is virtually identical across replications (bottom-right panel). The full stability report, including per-item and per-eigenvalue across-replication SDs and coefficients of variation, is available in Supplemental Materials. These results indicate that the upstream data-generating process is not itself a substantial source of Monte Carlo error.
Multireplication stability of the data-generating process across 30 independent seeds.
To assess the robustness of the reported results to Monte Carlo error, the specific concern that complex generative processes can produce unstable outcomes, we conducted a full replication study in which the entire pipeline (data generation and 11-algorithm benchmark) was repeated across 30 independent random seeds. The overall AUC-ROC values were highly stable across replications. The maximum across-replication coefficient of variation (CV) among the 10 non-Stray algorithms was 3.85% (COF), with most algorithms showing CVs at or below 1.5% (e.g., LOF: M = .968, SD = .009, CV = 0.90%; LDF: M = .964, SD = .010, CV = 1.01%; Autoencoder: M = .948, SD = .011, CV = 1.11%; Isolation Forest: M = .909, SD = .014, CV = 1.53%). Stray exhibited greater variability (M = .530, SD = .027, CV = 5.16%), consistent with its generally poor and unstable performance on survey data noted in the Discussion. Type-specific AUC-ROC values were similarly stable: detection of random responding was virtually invariant across seeds (SD < .001 for most algorithms), while straightline responding, the most challenging anomaly type, showed the largest across-replication variability (SD = .06–.09), consistent with its near-chance AUC-ROC levels. These results confirm that the differential sensitivity patterns reported in this study, the central finding that algorithms differ systematically in their ability to detect specific anomaly types, are robust to sampling variability and are not artifacts of a single draw from the generative process.
Discussion
This study examined the differential sensitivity of 11 ML algorithms to six types of anomalous survey responding. The central finding is clear and consequential: algorithms differ substantially in their ability to detect specific anomaly types, and these differences are systematically related to the detection mechanism each algorithm employs. No single algorithm dominated across all anomaly types, and the pattern of differential sensitivity carries direct implications for how researchers should approach anomaly detection in survey data.
Theoretical Implications
Detection Mechanisms and Anomaly Geometry
The differential sensitivity patterns observed in this study can be understood through the lens of anomaly geometry—the spatial and structural properties of different anomaly types in the multivariate item response space. Distance- and density-based methods (e.g., LOF, KNN-AGG, and LDOF) are fundamentally sensitive to positional anomalies: regions of the feature space with low density or high distance. Random and extreme responding produce such cases because they generate response vectors far from the normal data cloud. Alternating responding also falls into this category, as the rapid cycling between scale extremes produces vectors with distinctive variance and correlation profiles that place them in sparsely populated regions.
Reconstruction-based methods (e.g., autoencoders) are sensitive to structural anomalies: cases whose item-level covariance patterns deviate from those learned from normal data during training. The autoencoder’s strong performance on careless responding (.90 detection rate) and acquiescence (.76) reflects its ability to capture subtle covariance disruptions that do not necessarily produce large Euclidean distances. Careless deviations disrupt the learned covariance structure, even when modest in magnitude, and acquiescent responding introduces systematic covariance patterns that differ from those of valid respondents who attend to item content. These structural signatures are more accessible to a model that explicitly learns and tests against the covariance structure of normal responding.
Boundary- and tree-based methods (OCSVM, Isolation Forest) demonstrated broad detection coverage, achieving strong performance across most anomaly types. This versatility likely reflects the complementary nature of their detection mechanisms: OCSVM learns a boundary in kernel space that separates normal from anomalous regions, capturing both positional and structural anomalies to the extent that the RBF kernel maps them to distinguishable regions. Isolation Forest’s random partitioning mechanism is inherently sensitive to patterns that are easily separable from the majority of the data, regardless of whether the separation arises from distance, density, or structural distinctiveness.
The Threshold–Sensitivity Trade-Off
A recurring theme across the results is the tension between threshold conservatism and detection sensitivity. The 97.5th percentile threshold, while transparent and reproducible, imposed a hard ceiling on the number of respondents that could be flagged (75 out of 1,800), thereby creating a competitive dynamic among anomaly types for inclusion in the flagged set. Globally deviant cases (random, extreme, alternating) consistently dominated the upper tail of the score distribution, leaving little room for subtler anomalies. This finding has implications for the design of anomaly detection pipelines: a single conservative threshold may be insufficient when multiple anomaly types are present, as it will preferentially detect the most extreme cases at the expense of less extreme but substantively important ones.
The sensitivity analysis for OCSVM and Isolation Forest reinforced this point. Overall discriminative capacity (AUC-ROC) was robust to the contamination parameter (.89–.91 for OCSVM; comparable stability for Isolation Forest), but type-specific detection rates varied substantially. Increasing the contamination parameter from .05 to .25 improved detection of acquiescence (from 10% to 76%–77%) and careless responding (from 33%–35% to 87%–91%) at the cost of increased false positives. This trade-off is domain-dependent: in high-stakes assessments where false positives carry serious consequences, conservative thresholds may be preferable; in large-scale surveys where moderate false positive rates are tolerable, more liberal thresholds will capture a broader range of anomaly types.
Stray and the Limits of EVT-Based Detection
Stray’s uniformly poor performance across anomaly types (.47 overall AUC-ROC) was one of the more surprising findings. In the general anomaly detection literature, Stray’s extreme-value-theory approach is effective for identifying isolated outliers in moderate-dimensional data (Talagala et al., 2021). Its failure in the present context likely reflects a mismatch between the algorithm’s assumptions and the structure of survey data. Stray’s gap-based mechanism identifies observations whose distance to their nearest neighbor is statistically unlikely under a Gumbel distribution fitted to the gap distribution. In survey data with discrete, bounded response scales and relatively low dimensionality (85 items), the gap distribution may not conform well to the Gumbel assumption, leading to poorly calibrated p-values and, consequently, poor detection. In addition, many anomaly types in survey data produce response vectors that are not isolated, in the sense that Stray targets, careless and acquiescent respondents, may cluster together rather than appear as isolated outliers.
Practical Implications for Applied Researchers
The findings of this study yield several concrete recommendations for researchers who wish to incorporate ML-based anomaly detection into their survey data quality procedures.
No Single Algorithm Suffices
The differential sensitivity patterns documented here mean that any single algorithm will systematically miss certain anomaly types. Researchers who rely exclusively on one method—however well it performs on average—risk leaving specific forms of invalid responding undetected. The inter-algorithm agreement analysis reinforced this point: the autoencoder, OCSVM/Isolation Forest, and neighborhood-based methods each flagged partially overlapping but distinct sets of respondents. An ensemble or multimethod approach, in which respondents flagged by multiple algorithms are treated as high-confidence detections and respondents flagged by only one algorithm are reviewed further, offers a practical strategy for achieving broader coverage across anomaly types.
Combine ML Methods With Traditional Indicators
The difficulty that all algorithms face in straightline detection suggests that general-purpose anomaly detectors should be supplemented with purpose-built response quality indicators. Traditional indices such as the longstring index (which directly measures consecutive identical responses) and within-person response variance are well-suited to detecting straightline patterns that ML algorithms miss. A recommended workflow would involve (a) computing traditional indices (longstring, within-person variance, psychometric synonym/antonym correlations) to screen for straightline and other pattern-specific anomalies; (b) applying one or more ML algorithms (e.g., an autoencoder and Isolation Forest) to detect anomalies that are multivariate in nature; and (c) integrating evidence from both sources, flagging respondents who exceed thresholds on either traditional or ML-based indicators.
Threshold Selection Requires Deliberation
The results demonstrate that threshold choice has a first-order effect on which anomaly types are detected. The conservative 97.5th percentile threshold, while protecting against false positives in low-contamination settings, is structurally unable to recover from a contamination rate that substantially exceeds 2.5%. In our simulation, where the true contamination was 10%, this rule yielded near-zero detection of acquiescence and careless responding for neighborhood-based methods because the limited flag budget was consumed by more globally deviant anomaly types. We therefore caution applied researchers against treating any fixed-percentile rule as adaptive: by construction, it always flags the same proportion of respondents regardless of how many true anomalies are present, producing forced false positives when contamination is low and missed detections when it is high. Researchers should consider their tolerance for false positives, the expected prevalence of anomalies in their setting, and the substantive cost of misclassification when selecting thresholds. In exploratory data cleaning, where the goal is to identify and review potentially problematic respondents rather than automatically remove them, a more liberal threshold (e.g., the 90th percentile) may be appropriate. In confirmatory analyses where exclusion decisions must be justified, a more conservative threshold combined with manual review of flagged cases provides a defensible approach. More generally, adaptive or data-driven threshold selection methods that do not require pre-specifying the contamination rate—for example, knee-detection on the sorted score distribution or unsupervised cluster separation criteria—represent an important direction for methodological development.
The Autoencoder Warrants Serious Consideration
Among the algorithms tested, the autoencoder achieved the best balance of breadth and precision. It flagged 329 respondents—close to the true anomaly count of 300—and achieved the highest F1 score, with strong detection of careless (.90), extreme (.95), random (1.00), alternating (1.00), and acquiescent (.76) responding. Its detection of straightline patterns (.22) was modest, consistent with the general difficulty of this anomaly type, yet still exceeded that of neighborhood-based methods. The autoencoder’s semi-supervised design, in which the model is trained exclusively on normal response patterns, provides a natural framework for applied use: researchers can train the model on a clean subset (or the full dataset, assuming most respondents are valid) and flag cases with elevated reconstruction error. Computing the 97.5th percentile on training-set reconstruction errors of normal cases (rather than on the full mixed sample) provides an interpretable cutoff that is at least anchored to the distribution of attentive responding rather than to the contaminated test sample, although it remains a fixed-percentile rule and inherits the limitations described above.
OCSVM and Isolation Forest as Broad-Spectrum Detectors
When high recall is prioritized over precision, for example, in initial screening before manual review, OCSVM and Isolation Forest offer high detection rates across most anomaly types. Their contamination parameter provides a convenient lever for adjusting the sensitivity–specificity trade-off, and the sensitivity analysis demonstrated that their overall discriminative capacity (AUC-ROC) is robust to this choice. Applied researchers can set the contamination parameter based on prior expectations about the prevalence of anomalous responding in their data (e.g., 5%–15% in online samples; Huang et al., 2015; Meade & Craig, 2012).
Limitations and Future Directions
Several limitations qualify the interpretation and generalizability of these findings. First, all evaluation metrics reported in this study, including the primary metric, AUC-ROC, require known ground-truth labels indicating which respondents are truly anomalous. Such labels are available only because we used simulated data with explicitly generated anomaly types. In real-world applications, ground-truth anomaly status is unknown, so AUC-ROC and other supervised evaluation metrics cannot be computed. Simulation provides controlled conditions for evaluating algorithm performance, but the generalizability of these findings to real survey data remains an open question. In real data, anomaly types may be more ambiguous, overlapping, or qualitatively different from the clean, discrete categories simulated here. For example, a real respondent may exhibit partial acquiescence combined with intermittent carelessness—a hybrid pattern that was not represented in our simulation design. Future research should complement simulation-based benchmarking with validation against real survey datasets, using independent indicators of response quality (e.g., response time, instructed response items, and retest consistency) as approximate ground truth.
Second, the six anomaly types examined here, random, extreme, careless, acquiescence, straightline, and alternating responding, represent theoretically motivated but idealized patterns. Each type was simulated in its pure form, with all anomalous respondents in a given category exhibiting the target behavior across the entire survey. In practice, anomalous responding is often intermittent, partial, or mixed (Meade & Craig, 2012). A respondent may answer attentively for the first half of a survey and then switch to straightline responding out of fatigue. The present results characterize algorithm sensitivity to pure anomaly types; detection accuracy for partial or mixed anomalies remains to be investigated.
Third, the simulation was based on a single instrument (85 Likert-type items). The generalizability of the differential sensitivity patterns to instruments with different structures (e.g., fewer items, different scale lengths, and multidimensional vs. unidimensional) is unknown. Survey instruments with fewer items provide less multivariate information for anomaly detection, potentially degrading performance for all methods. Instruments with binary or polytomous items may produce different anomaly geometries than 5- or 7-point Likert scales. Future benchmarking studies should systematically vary instrument properties to establish boundary conditions for ML-based anomaly detection.
Fourth, although AUC-ROC provides a threshold-free summary of discriminative capacity, applied decisions about which respondents to flag or exclude are inherently threshold-dependent. Our use of the 97.5th percentile for most algorithms was a conventional default chosen for transparency and reproducibility, not an optimal or theoretically “principled” choice. As is true of any fixed-percentile threshold, this rule is data-relative and does not adapt to the true contamination rate: it will always flag approximately 2.5% of respondents, even if every respondent is attentive (forcing false positives) and even if half are anomalous (producing substantial false negatives). The clearest manifestation in our results is the hard 25% recall ceiling on the seven transductive neighborhood-based methods, which constrained their type-specific detection rates for acquiescence and careless responding. The decision boundaries for OCSVM and Isolation Forest face an analogous problem: their contamination parameters must be set a priori. We therefore caution that the threshold-dependent metrics reported here (detection rate, F1) should be interpreted as illustrative of behavior at a single defensible threshold setting, not as estimates of detection performance at an optimal cutoff. Future work should pursue adaptive or data-driven threshold selection methods that do not require knowledge of the contamination rate, such as those based on score-distribution inflection points (knee/elbow detection), unsupervised cluster separation metrics, or extreme-value-theory cutoffs of the kind Stray attempts to provide. Encouragingly, the central conclusions of the present study regarding differential sensitivity to anomaly type are based on AUC-ROC, which is invariant to threshold choice.
Fifth, several algorithmic parameters were tuned using ground-truth labels (e.g., selecting k = 20 for neighborhood-based methods by maximizing LOF AUC-ROC). While this approach is standard in benchmarking studies and does not constitute information leakage into the algorithmic scores (which are computed unsupervised), it means the reported performance may not transfer directly to applied settings where label information is unavailable for tuning. In practice, researchers would need to rely on rules of thumb, cross-validation on proxy outcomes, or domain knowledge to set these parameters.
Sixth, our simulation included 300 anomalous cases out of 3,000 total (10% contamination rate), distributed unevenly across six types. This base rate is used because 5%–10% is commonly reported in laboratory studies (Meade & Craig, 2012), but may be lower than rates encountered in some online panels. The absolute and relative base rates of anomaly types affect both algorithm performance and threshold calibration. Future research should examine how varying contamination rates, both overall and by type, affect the differential sensitivity patterns observed here.
Conclusion
ML algorithms for anomaly detection offer considerable promise for identifying invalid survey responses, but they are not interchangeable. The differential sensitivity patterns documented in this study demonstrate that each algorithm’s detection mechanism creates systematic blind spots for certain anomaly types. Globally deviant patterns are detectable by any method; structurally subtle patterns require algorithms attuned to covariance disruptions (autoencoders) or kernel-space separability (OCSVM); and the most challenging pattern—straightline responding—eludes general-purpose anomaly detectors entirely. These findings argue for a multimethod approach that combines ML-based anomaly detection with traditional response quality indicators, informed by an understanding of which anomaly types each method is (and is not) equipped to detect. By explicitly accounting for the differential sensitivity of available tools, this study provides a framework for researchers to make transparent, defensible decisions about survey data quality screening.
Supplemental Material
sj-docx-2-epm-10.1177_00131644261448404 – Supplemental material for Beyond One-Size-Fits-All: A Differential Sensitivity Framework for Machine Learning–Based Detection of Anomalous Survey Responses
Supplemental material, sj-docx-2-epm-10.1177_00131644261448404 for Beyond One-Size-Fits-All: A Differential Sensitivity Framework for Machine Learning–Based Detection of Anomalous Survey Responses by Cody Ding in Educational and Psychological Measurement
Supplemental Material
sj-docx-3-epm-10.1177_00131644261448404 – Supplemental material for Beyond One-Size-Fits-All: A Differential Sensitivity Framework for Machine Learning–Based Detection of Anomalous Survey Responses
Supplemental material, sj-docx-3-epm-10.1177_00131644261448404 for Beyond One-Size-Fits-All: A Differential Sensitivity Framework for Machine Learning–Based Detection of Anomalous Survey Responses by Cody Ding in Educational and Psychological Measurement
Supplemental Material
sj-docx-4-epm-10.1177_00131644261448404 – Supplemental material for Beyond One-Size-Fits-All: A Differential Sensitivity Framework for Machine Learning–Based Detection of Anomalous Survey Responses
Supplemental material, sj-docx-4-epm-10.1177_00131644261448404 for Beyond One-Size-Fits-All: A Differential Sensitivity Framework for Machine Learning–Based Detection of Anomalous Survey Responses by Cody Ding in Educational and Psychological Measurement
Supplemental Material
sj-docx-5-epm-10.1177_00131644261448404 – Supplemental material for Beyond One-Size-Fits-All: A Differential Sensitivity Framework for Machine Learning–Based Detection of Anomalous Survey Responses
Supplemental material, sj-docx-5-epm-10.1177_00131644261448404 for Beyond One-Size-Fits-All: A Differential Sensitivity Framework for Machine Learning–Based Detection of Anomalous Survey Responses by Cody Ding in Educational and Psychological Measurement
Supplemental Material
sj-ipynb-1-epm-10.1177_00131644261448404 – Supplemental material for Beyond One-Size-Fits-All: A Differential Sensitivity Framework for Machine Learning–Based Detection of Anomalous Survey Responses
Supplemental material, sj-ipynb-1-epm-10.1177_00131644261448404 for Beyond One-Size-Fits-All: A Differential Sensitivity Framework for Machine Learning–Based Detection of Anomalous Survey Responses by Cody Ding in Educational and Psychological Measurement
Footnotes
Consent to Participate
Not applicable.
Consent for Publication
Not applicable.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statements
The codes used in this study are available in Supplemental Materials.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
