Prediction of specific energy consumption in electric arc furnaces: A data-centric multi-stage collaborative filtering framework

Abstract

In the context of electric arc furnace (EAF) steelmaking, a typical process industry scenario, the conventional model-centric artificial intelligence (MCAI) paradigm faces a critical bottleneck caused by low-quality raw industrial data. Specifically, the low quality of raw industrial data results in prediction models for specific energy consumption (SEC, i.e. energy consumption per ton of steel) exhibiting both low accuracy and high volatility (i.e. poor robustness). To address this challenge, this study embraces the data-centric artificial intelligence (DCAI) paradigm and proposes an innovative four-stage multi-stage collaborative filtering (MCF) framework. This framework serves as a practical implementation of the DCAI philosophy, systematically refining the EAF SEC dataset to construct a ‘golden dataset’ through four progressive and synergistic stages: preliminary feature pruning, global instance denoising, collaborative feature selection, and boundary instance condensation (removal of ambiguous samples near decision boundaries). A core design principle of the framework is its adherence to a collaborative logic: stabilising the feature space prior to refining the instance distribution. Rigorous evaluation based on the resultant ‘golden dataset’ demonstrates that the MCF framework not only significantly enhances model prediction accuracy, as evidenced by metrics such as the hit rate under critical error tolerances and root mean square error (RMSE), but also substantially reduces the volatility of this accuracy, measured by the standard deviation (std.) of these metrics. Consequently, industrial-grade robustness is achieved. A decisive finding of this study is that a fundamental Lasso model, when applied to the ‘golden dataset’, comprehensively surpasses the overall performance (encompassing both accuracy and robustness) of a complex, hyperparameter-tuned XGBoost model trained on the original, unrefined data. This research demonstrates that the MCF framework, by concurrently improving prediction accuracy and ensuring prediction stability, offers an effective technical solution for implementing the DCAI paradigm in process industries such as EAF steelmaking. Furthermore, it strategically validates that investing in data quality is a higher-leverage, high-value pathway than complex model tuning.

Keywords

data-centric artificial intelligence electric arc furnace steelmaking specific energy consumption prediction multi-stage collaborative filtering data cleaning model reliability model robustness

Introduction

As a key process in the modern steel industry, the energy consumption per ton of steel in electric arc furnace (EAF) steelmaking is a core indicator determining production costs and carbon emission efficiency. Consequently, accurate prediction of this metric is of significant strategic importance for process optimisation and energy conservation.^1–3 A review of the field's history reveals that EAF energy consumption prediction methods have evolved from mechanistic analysis to data-driven approaches, and from simple statistics to complex intelligent models. This evolution reflects the industry's persistent pursuit of prediction accuracy and robustness, while highlighting the persistent disconnect between academic research and industrial practice.

Evolution of EAF energy consumption prediction methods and paradigm shifts

Period of mechanistic and statistical empirical models (approx. 1990s–early 2000s)

Early research primarily relied on integrating thermodynamic mechanism models with statistical regression methods. In 1992, based on production data from 14 AC EAF, Köhle⁴ pioneered the establishment of an empirical formula for electrical energy consumption using multiple linear regression (MLR), laying the foundation for data-driven modelling in this field. He later systematically summarised the latest progress in EAF energy consumption modelling at the 7th European Electric Steelmaking Conference (EEC) in 2002.⁵ Concurrently, at the same conference, Pfeifer and Kirschen⁶ proposed a research framework contrasting thermodynamic analysis with statistical models, exploring the differences and complementarities between energy balance and statistical prediction. Targeting specific furnace types, Conejo and Cárdenas⁷ presented a statistical model study on the energy consumption characteristics of a 100% direct reduced iron EAF at AISTech 2006. Furthermore, Sandberg et al.⁸ optimised EAF energy efficiency through statistical process evaluation, while Czapla et al.⁹ utilised multiple regression combined with process parameters to optimise power consumption. Research during this period was characterised by the selection of 5 to 10 key process variables (e.g., scrap weight, power-on time, oxygen injection volume) based on expert experience to establish linear or polynomial regression models with clear physical meaning. The advantage of these models lies in their high interpretability, but they struggle to handle complex non-linear interactions and high-dimensional data.

Period of traditional machine learning adoption (approx. mid-to-late 2000s–2010s)

With increased computational power and data accumulation, researchers began introducing machine learning methods such as support vector machines (SVM), random forests (RF), and artificial neural networks (ANN). Notably, international researchers, including Chinese scholars, explored the application of neural networks and chaos theory in EAF modelling during this period.¹⁰ Carlsson et al.¹¹ systematically reviewed the application of statistical models and machine learning methods (including ANN, deep learning, and RF) in EAF energy consumption prediction. Logar et al.¹² developed region-based mathematical models for scrap melting process control and energy optimisation. Tomažič et al.¹³ employed methods such as SVM, k-nearest neighbors (k-NN), and Gaussian Processes to predict energy consumption per heat, combining fuzzy logic to enhance model adaptability. Research in this stage began to demonstrate the advantages of non-linear modelling yet still relied heavily on manual feature engineering and domain knowledge, without forming a systematic data quality engineering system. Notably, a substantial gap persisted between the complexity of academic research and the simple statistical models actually applied in industry during this period.

Period of model-centric AI arms race (approx. 2016–2022)

In recent years, the industry has widely attempted to adopt the Model-Centric Artificial Intelligence (MCAI) paradigm, seeking to improve prediction accuracy by constructing increasingly complex models. Gradient Boosting Decision Tree models such as XGBoost and LightGBM, as well as deep neural networks (DNNs),^14,15 have been widely applied to modelling EAF operational parameters. This paradigm centres research and development on the continuous iteration of models, aiming to overcome data-related challenges through model complexity.^16,17 However, in complex industrial environments like EAF steelmaking, the inherent characteristics of raw industrial data – namely, high noise, high redundancy, and low quality – pose a major bottleneck that MCAI approaches cannot easily overcome.^18,19 High noise levels and abnormal operating conditions cause models to learn erroneous patterns; high redundancy and feature collinearity increase the risk of overfitting; and low-quality data prevents even the most advanced MCAI models from learning true physical laws, ultimately leading to unstable performance and poor reliability.^20,21

Rise of the data-centric paradigm (2019–present)

To break through the limitations of MCAI, the AI field has advocated a paradigm shift towards data-centric AI (DCAI).^22,23 This paradigm proposes redirecting the focus from the endless iteration of model algorithms to the systematic and engineered improvement of data quality.^24,25 DCAI's core tenet is that high-quality data outweighs model complexity; optimising data to enhance model performance and robustness is a higher-leverage strategy.^26,27 Zha et al.²⁴ systematically elucidated the three main steps of DCAI: training data development, inference data development, and data maintenance. In the industrial sector, DCAI has already demonstrated potential in scenarios such as predictive maintenance and quality control, significantly improving model robustness through data cleaning, denoising and label correction.^28,29 However, in complex process industry scenarios like EAF steelmaking, a concrete and systematic methodology for effectively implementing the DCAI philosophy is still lacking. Existing literature often focuses on the application of single cleaning techniques (e.g. anomaly detection via Isolation Forest³⁰ or feature selection via Lasso³¹), overlooking the necessity of multi-stage collaboration.³²

Research motivation and core contributions

To address this gap, this study proposes an innovative four-stage multi-stage collaborative filtering (MCF) framework as a specific technical vehicle for the DCAI philosophy in process industries. The core contributions of this framework are as follows:

Technical innovation: The MCF framework successfully constructs a ‘golden dataset’ suitable for EAF energy consumption prediction through four progressive and collaborative stages: preliminary feature pruning, global instance denoising, collaborative feature selection, and boundary instance condensation. The core design follows the collaborative logic of ‘stabilising the feature space first, then purifying the instance distribution’, ensuring a systematic and effective data purification process.

Performance enhancement: Rigorous evaluations based on the ‘golden dataset’ demonstrate that the MCF framework not only significantly increases the prediction hit rate within critical error tolerances but also substantially reduces the volatility of prediction accuracy (as measured by the standard deviation of root mean squared error (RMSE)), achieving a fundamental transformation from ‘unstable average accuracy’ to ‘industrial-grade robustness’.

Strategic value: Through a decisive asymmetric comparative experiment, this study compellingly demonstrates the core strategic value of the DCAI paradigm: a basic Lasso model applied to the ‘golden dataset’ comprehensively outperforms a complex, hyperparameter-tuned XGBoost model trained on the original data in terms of both accuracy and robustness. This confirms that investing in data quality is a higher-leverage pathway than investing in model tuning.

Research limitations and paper structure

It implies that the MCF framework constructed in this study and the verification experiments rely strictly on production data collected from two specific 130-ton DC EAFs. While the design philosophy is generalisable, its direct applicability and optimal parameter configuration in other EAF types (such as AC EAFs) or other process industries (such as chemicals and cement production) require further verification. This article does not claim that the model possesses universal cross-equipment transferability, but rather aims to provide a reliable technical solution and empirical benchmark for DCAI applications in EAF scenarios.

The rest of this article is structured as follows: the second section elucidates the background of the dataset used in this study and systematically analyses the quality challenges faced by raw industrial data. The third section provides a detailed exposition of the MCF framework. The fourth section presents a series of rigorous, progressive experimental designs. The fifth section systematically presents and discusses the results of these experiments, while also addressing the limitations of the current study and suggesting directions for future research. Finally, the sixth section provides a conclusion to the article.

Data foundation and quality challenges

This chapter elucidates the background of the industrial dataset utilised in this study and systematically analyses its inherent, unrefined quality challenges. This analysis not only serves as the direct motivation for introducing the MCF framework but also forms the logical cornerstone of the entire research design.

Dataset overview

The dataset used in this study was sourced from a steel enterprise in China, collected over a 6-month period from two similar 130-ton direct current (DC) EAFs. It comprises production records from 2000 valid smelting heats, each with 130 features. This dataset represents the production of approximately 120 different steel grades during this period, thereby exhibiting broad operational diversity and representativeness. The feature set is comprehensive, covering five key process dimensions of EAF steelmaking: initial charge characteristics, electrical energy input parameters, blowing and chemical characteristics, auxiliary material addition features, and in-process monitoring features. The core target variable for this research is the specific energy consumption (SEC, i.e. energy consumption per ton of steel) (kWh/t), a key performance indicator for EAF energy efficiency and economic viability.

To ensure the objectivity and rigour of all subsequent data purification effect assessments, a baseline strategy of ‘minimal preprocessing’ was intentionally adopted before the formal introduction of the MCF framework. Only a single, essential baseline operation was performed on the raw data: rows with a feature missing rate exceeding 50% were removed. Based on this criterion, a total of 46 records were excluded, accounting for only 2.3% of the total raw data. Preliminary statistical analysis indicated that the distribution of the removed samples across key process parameters (e.g. heat type and scrap addition) showed no significant difference from that of the retained samples, suggesting that the missingness primarily originated from sporadic cross-sensor data acquisition failures rather than systematic absence under specific operating conditions. Consequently, this operation maximally mitigates the risk of introducing selection bias while cleaning invalid data. The sole purpose of this step was to ensure that any subsequent machine learning models could run without input errors, rather than to perform any substantive intervention on the data quality. This design deliberately avoids common preprocessing steps such as feature pre-selection based on expert domain knowledge, complex outlier smoothing or data standardisation.

The core rationale for this strategic choice is to strictly adhere to the principle of controlled variables in scientific experimentation, aiming to establish a fair and highly challenging ‘raw baseline’. Consequently, any improvements in model performance observed in subsequent sections can be uniquely and unequivocally attributed to the systematic effects of the proposed MCF framework, thereby unambiguously quantifying its independent contribution. This design is intended to demonstrate that a well-engineered, systematic DCAI framework like MCF should inherently possess the capability to automatically identify and handle redundancy, anomalies and noise from highly complex raw data, without relying on prior, domain-specific human intervention.

Quality challenges of raw industrial data

Under the aforementioned ‘minimal preprocessing’ strategy, the raw industrial data exhibits typical quality deficiencies, including high noise, redundancy and low quality. Its inherent, multi-dimensional quality issues pose severe constraints on the performance of any data-driven model. These challenges, which are the specific targets the MCF framework is designed to systematically address, are concentrated on the following three levels:

High-dimensional redundancy and the collinearity trap: The original feature set contains significant multicollinearity (i.e. high linear correlation between features) and informational overlap.^33,34 This not only unnecessarily increases the computational burden of the model but, more critically, can lead to overfitting to noise during the training process, thereby severely compromising the model's generalisation ability and the reliability of its predictions.

Interference from abnormal operating conditions and outliers: Transient sensor failures, data transmission errors or sporadic, highly atypical operating conditions in the industrial setting generate anomalous samples and strong outliers within the dataset.^35,36 This ‘bad data’ can severely distort the true distribution of the data; it poses strong interference to the model's learning process and deflecting it from an accurate understanding of the core physical principles.

Pervasive noise and blurred decision boundaries: The combination of pervasive measurement noise, process fluctuations and the inherent distribution of samples in the feature space leads to blurred decision boundaries between different ranges of the target variable (SEC).³⁷ This ambiguity makes it difficult for the model to learn clear and robust decision rules, which directly impacts its prediction accuracy and generalisation capability in critical application scenarios.

Multi-stage collaborative filtering framework: a systematic implementation of the DCAI paradigm

This chapter provides a detailed exposition of the MCF framework, which underpins the core of this research. As the principal technical vehicle for the DCAI paradigm in the context of EAF steelmaking, MCF is designed to progressively purify an initial dataset, $D_{R a w}$ , composed of an original feature set X and an original instance set S, into a ‘golden dataset’ characterised by high information density and consistency. This is accomplished through a systematic engineering process. The design philosophy of the MCF framework adheres to a progressive, coarse-to-fine logic. The overall process is partitioned into two primary stages: global purification and local refinement. Critically, each stage strictly implements the principle of ‘first optimising the feature space, then purifying the instance space’. This hierarchical design ensures that subsequent instance purification operations are always conducted on an optimised and more reliable feature space. Such an approach fundamentally mitigates the risk of erroneously discarding valid samples due to the failure of distance metrics in high-dimensional, redundant spaces, which is a prerequisite for ensuring the accuracy and robustness of the entire framework.³⁸ The architecture of the MCF framework is illustrated in Figure 1.

Figure 1.

Architectural diagram of the multi-stage collaborative filtering (MCF) framework.

Stage 1: preliminary feature pruning

The objective of this stage is to perform a rapid initial purification of the original feature set X, eliminating features with minimal contribution and high informational redundancy. This process yields a more streamlined and robust intermediate feature set, $X^{'}$ , thereby reducing data dimensionality and enhancing the computational efficiency of subsequent algorithms. This stage employs a serial dual-filtering mechanism. First, low variance filtering is applied to calculate the variance, $V a r (X_{i})$ of each feature $X_{i}$ according to equation (1). In this equation, n represents the total number of samples, $x_{i j}$ the value of the j-th sample for the i-th feature, and $\bar{x_{i}}$ the mean of that feature. This process removes ‘near-constant features’ with a variance below a predefined threshold $θ_{v a r}$ (set to 0.01). This threshold setting is based on the empirical distributional characteristics of the features, designed to filter out static signals that remain constant across the vast majority of samples and lack statistical explanatory power for energy consumption fluctuations.

Subsequently, to address the problem of linear dependency among features, the variance inflation factor (VIF) is introduced for multicollinearity diagnosis. As formulated in equation (2), the VIF of a feature $X_{i}$ measures the extent of its variance inflation, which is fundamentally determined by the coefficient of determination, $R_{i}^{2}$ . This $R_{i}^{2}$ , as defined by equation (3), corresponds to the coefficient of determination from an auxiliary linear regression model in which $X_{i}$ is the dependent variable and all other features act as independent variables. In equation (3), n is the total number of samples; $x_{i, k}$ and $\bar{x_{i}}$ the observed and mean values of feature $X_{i}$ , respectively and $\hat{x_{i, k}}$ the predicted value of $x_{i, k}$ from the auxiliary model. An $R_{i}^{2}$ value approaching 1 signifies that feature $X_{i}$ is highly predictable from other features, causing its VIF to approach infinity and indicating severe multicollinearity. The feature with the highest VIF is iteratively removed until all remaining features have a VIF below the threshold $θ_{vif}$ (set to 15). Although statistical literature typically regards a VIF > 10 as indicative of severe multicollinearity,³⁹ this study adopts a slightly more lenient threshold of 15 to account for the inherent physical coupling among industrial parameters (e.g. voltage and current). This represents a strategic trade-off between eliminating harmful collinearity and preserving physical information, thereby effectively breaking the collinearity trap.

This dual serial filtering mechanism operates exclusively on the feature dimension, purifying the initial feature set X into the more streamlined and robust intermediate set $X^{'}$ , while the original instance set S remains unaltered. By establishing a feature space with reduced dimensionality and weaker collinearity, this step provides a more reliable foundation for the distance metrics required in the subsequent global instance denoising stage. Consequently, $X^{'}$ and S jointly form the updated dataset that serves as the input for the second stage of the MCF framework.

V a r (X_{i}) = \frac{1}{n} \sum_{j = 1}^{n} (x_{i j} - \bar{x_{i}})^{2}

(1)

{VIF}_{i} = \frac{1}{1 - R_{i}^{2}}

(2)

R_{i}^{2} = 1 - \frac{\sum_{k = 1}^{n} {(x_{i, k} - \hat{x_{i, k}})}^{2}}{\sum k = 1^{n} {(x_{i, k} - \bar{x_{i}})}^{2}}

(3)

Stage 2: global instance denoising

Following the preliminary purification of the feature space in stage 1, which yielded a more reliable feature subset $X^{'}$ , the focus of the MCF framework shifts from the feature dimension to the instance dimension. The objective of this stage is to identify and remove global, high-dimensional outlier samples from the dataset within this more robust feature space defined by $X^{'}$ .

To accomplish this, the isolation forest (IF) algorithm is employed. The core hyperparameter of this algorithm is the ‘contamination rate’ (denoted as $ν$ ), which pre-specifies the expected proportion of anomalous samples in the dataset. Based on an exploratory analysis of the raw industrial data and an empirical estimation of the frequency of abnormal operating conditions in typical smelting processes, this study sets $ν$ to 0.08. This parameter choice is grounded in a retrospective analysis of the steel plant's historical production logs, indicating that the proportion of abnormal conditions resulting from cold starts, sensor disconnections, or extreme unplanned operations typically falls within the 5% to 10% range. Therefore, setting $ν$ aligns with the domain's prior probability, effectively capturing global outliers while minimising the risk of erroneously removing valid samples representing extreme operating conditions. Based on the core principle that ‘anomalies are more susceptible to isolation’, the IF algorithm efficiently identifies and isolates these outlier instances by partitioning the data via random hyperplanes.

This process operates exclusively on the instance dimension, purifying the instance set from S to $S^{'}$ while leaving the feature set $X^{'}$ intact. The resulting sets, $X^{'}$ and $S^{'}$ , jointly constitute the updated dataset that serves as the input for the third stage of the MCF framework.

Stage 3: collaborative feature selection

After addressing feature redundancy and global instance noise in the preceding stages, the statistical relationships between the features and the target variable have become more distinct. Building on this foundation, the focus of this stage reverts to the feature space, aiming to distil a core feature subset, $X^{″}$ , from the intermediate set $X^{'}$ . This subset is intended to comprise features with the highest predictive stability and mechanistic reliability relative to the target variable Y. To achieve this, an innovative dual-engine collaborative strategy was designed to integrate the complementary perspectives of linear and non-linear models.

The first engine employs the LassoCV, which integrates the Lasso model with a cross-validation. As a form of regularised linear regression, the efficacy of Lasso's feature selection capability lies in its unique objective function, presented in equation (4). This function minimises the traditional sum of squared errors term, $| Y - X β |_{2}^{2}$ (where Y is the target variable vector, X the feature matrix and $β$ the feature coefficient vector), while simultaneously imposing an L1 regularisation penalty, $α \sum_{j} | β_{j} |$ . The core principle of this penalty term, governed by the hyperparameter $α$ , is to penalise the sum of the absolute values of all feature coefficients, $β_{j}$ . During model optimisation, this penalty compels the coefficients corresponding to features with negligible predictive contribution to be precisely shrunk to zero, thereby achieving automated feature elimination. The cross-validation component of LassoCV is responsible for identifying the optimal $α$ value in a data-driven manner, ensuring the robustness and reliability of the feature selection process. This engine ultimately yields a set of core features based on a linear perspective, designated as $Top - K_{Lasso}$ .

Concurrently, the second engine utilises a high-performance LightGBM model, which excels at capturing non-linear and complex interaction effects among features. This engine ranks feature importance based on information gain (IG), calculated as shown in equation (5). In this formula, D represents the dataset, a a specific feature and $D_{v}$ the subset of D where feature a has the value v. $Ent (D)$ and $Ent (D_{v})$ are the entropies of the dataset D and its subset, respectively. A higher IG value indicates that feature a contributes more significantly to reducing the uncertainty of the dataset, thus implying greater importance. This process generates another set of core features based on a non-linear perspective, $Top - K_{LGBM}$ .

The decision logic epitomises the essence of ‘collaboration’: priority is given to the intersection of features identified by both engines. If the size of this intersection, $k_{intersect}$ , is less than the predefined number of core features, $K_{core}$ (set to 20 in this study; this setting relies on the diminishing marginal returns of feature importance, where top-ranked features capture the vast majority of information gain, while tail features offer limited contribution and potentially introduce noise. Additionally, the number of candidate features for LightGBM, $Top_K_{LGBM}$ , is set to 30 to provide sufficient redundancy, ensuring flexibility and robustness when supplementing features in cases of insufficient intersection), the remaining $K_{core} - k_{intersect}$ features are supplemented according to the feature importance ranking provided by the LightGBM engine. This strategy ensures that the final feature set $X^{″}$ is not only highly condensed but also possesses dual robustness, validated from both linear and non-linear modelling perspectives. The instance set $S^{'}$ remains unchanged. The resulting sets, $X^{″}$ and $S^{'}$ , jointly form the updated dataset that serves as the input for the fourth stage of the MCF framework.

min_{β} {| Y - X β |_{2}^{2} + α \sum_{j} | β_{j} |}

(4)

IG (D, a) = Ent (D) - \sum_{v \in Values (a)} \frac{| D_{v} |}{| D |} Ent (D_{v})

(5)

Stage 4: boundary instance condensation

With the low-dimensional and high-information-density core feature subset $X^{″}$ established, the MCF framework enters its final and most refined instance purification stage. The objective of this stage is to address a more subtle data quality issue: ambiguous instances located near the decision boundaries of different target intervals. In contrast to stage 2, which removed global outliers, this stage is dedicated to achieving a local ‘condensation’ of data clusters to enhance model robustness in critical decision regions.

To this end, the edited nearest neighbors (ENN) algorithm is innovatively adapted for this regression task. First, through an equal-frequency binning strategy, the continuous target variable (energy consumption per ton of steel) is discretized into M ordered intervals. In this study, we set $M = 5$ to partition the energy consumption distribution into five ordered intervals representing typical operating conditions (ranging from extremely low to extremely high). This ensures full coverage of the energy consumption fluctuation range during the production process, thereby assigning a temporary class label to each sample. Subsequently, within the feature space defined by $X^{″}$ , each sample is evaluated against its k nearest neighbours. The setting of $k = 5$ follows the classic configuration of the k-NN algorithm, striking a balance between local sensitivity and resistance to noise interference.⁴⁰ If the temporary class of a sample is inconsistent with the majority class of its neighbours, the sample is identified as a boundary noise point and is consequently removed. Through this “editing” process, the internal structure of the final dataset becomes clearer, and the boundaries between different target value intervals are rendered more distinct. At this point, the entire collaborative filtering process is concluded, and the resulting high-purity instance set constitutes the ‘golden dataset’, which serves as the cornerstone for all subsequent model evaluations. To systematically present the implementation details of the MCF framework and ensure the transparency and reproducibility of this research, Table 1 provides a comprehensive summary of the core methods, key hyperparameters, and specific setting values involved in the aforementioned four stages. This parameter system forms the foundation for all subsequent experimental evaluations.

Table 1.

Configuration of core methods and hyperparameters for each stage of the MCF framework.

Stage	Core method	Hyperparameter	Set value
Stage 1	Low variance filter	$θ_{var}$	0.01
	Variance inflation factor	$θ_{vif}$	15
Stage 2	Isolation forest	$ν$	0.08
Stage 3	LassoCV	$K_{core}$	20
	LightGBM	$Top - K_{LGBM}$	30
Stage 4	Edited nearest neighbours	$M$	5
		$k$	5

MCF: multi-stage collaborative filtering.

Experimental design

This chapter details the series of progressive experiments designed for the systematic evaluation of the DCAI paradigm and its core technical vehicle, the MCF framework. The structure of this chapter follows a logical progression: demonstration of core advantages, validation of internal mechanisms, assessment of subsequent benefits, and finally, elucidation of its strategic value. This approach is intended to clearly present how each experiment is executed through specific, reproducible steps, thereby ensuring the rigour and transparency of the research.

Evaluation protocol

To ensure the fairness and comparability of all subsequent experiments, we first defined an evaluation baseline that was consistently applied throughout the study. This baseline includes a model library and a metric system, which together form the foundation for all experimental assessments.

To comprehensively and impartially evaluate the generalizability of the MCF framework across algorithms of varying complexity, this study constructed a benchmark model library covering three major paradigms: ‘classical statistical regression’, ‘traditional machine learning’, and ‘modern ensemble learning’. The selection basis is grounded in a deep analysis of data characteristics and industrial application scenarios:

Data modality adaptability: The input data for this study consists of static tabular data from discrete smelting heats, where each record represents a summary of process parameters for an independent heat. During modelling, heats are treated as independent and identically distributed (i.i.d.) samples. Since the data itself does not contain temporal dependencies requiring modelling, deep learning models specialised in capturing long-range sequential information (such as long short-term memory and recurrent neural network) are ill-suited for this task and were therefore excluded from the model library.

Performance advantage on tabular data: Although DNNs possess powerful fitting capabilities, Gradient Boosting Decision Tree (GBDT) models, represented by XGBoost and LightGBM, typically exhibit superior robustness, faster training efficiency and stronger interpretability compared to standard DNNs when applied to medium-scale (2000 samples) high-dimensional industrial tabular data.⁴¹ Consequently, this study selected XGBoost and LightGBM as representatives of ‘high-performance black-box models’.

Historical baseline comparison: To address the field's interest in traditional methods, ridge regression and lasso regression were specifically introduced. These two models represent the ‘simple statistical regression models’ widely used in early research, serving as a foundational baseline to quantify the magnitude of the performance leap the DCAI paradigm can deliver relative to the traditional linear fitting paradigm.

Ultimately, the benchmark model library comprises six models of varying complexity: Ridge, Lasso, support vector regression, RF, XGBoost, and LightGBM. In all experiments, unless explicitly stated otherwise in the hyperparameter tuning comparison in the ‘Strategic value assessment: comparative efficacy of the DCAI and MCAI paradigms’ section, the hyperparameters of these models remained strictly fixed. This ensures that data quality is the sole variable influencing changes in model performance.

To quantify model performance multidimensionally, a comprehensive evaluation metric system was established, encompassing three primary aspects: prediction accuracy, prediction hit rate and model robustness. Prediction accuracy is measured using standard statistical metrics, including RMSE, mean squared error (MSE), mean absolute error (MAE), and the coefficient of determination ( $r^{2}$ ), with their formulas provided in equations (6) through (9), respectively. In these equations, $y_{i}$ is the actual value, $\hat{y_{i}}$ the predicted value, $\bar{y}$ the mean of the actual values and N the total number of samples. To more closely align with the evaluation criteria of practical industrial applications, we introduced the prediction hit rate metric. This is defined as the proportion of samples for which the absolute error between the predicted and actual values falls within a specific threshold. This study focuses on evaluating the hit rates at three critical gradients: ±5, ±10 and ±15 kWh/t, representing high-precision, conventional and lenient industrial tolerances, respectively. Furthermore, model robustness is quantified by the standard deviation ( $σ_{Metric}$ ) of key prediction accuracy metrics (e.g., RMSE) across multiple cross-validation runs, as shown in equation (10). This measure is used to assess the volatility and reliability of the model's performance, where R and K represent the number of repetitions and folds in the cross-validation, respectively.

RMSE = \sqrt{\frac{1}{N} \sum_{i = 1}^{N} {(y_{i} - \hat{y_{i}})}^{2}}

(6)

MSE = \frac{1}{n} \sum_{i = 1}^{n} (y_{i} - \hat{y_{i}})^{2}

(7)

MAE = \frac{1}{N} \sum_{i = 1}^{N} | y_{i} - \hat{y_{i}} |

(8)

r^{2} = 1 - \frac{\sum_{i = 1}^{N} {(y_{i} - \hat{y_{i}})}^{2}}{\sum_{i = 1}^{N} {(y_{i} - \bar{y})}^{2}}

(9)

\begin{matrix} σ_{Metric} = \sqrt{\frac{1}{R \times K} \sum_{r = 1}^{R} \sum_{k = 1}^{K} {({Metric}_{r, k} - \bar{Metric})}^{2}} \end{matrix}

(10)

Core advantage assessment: Enhancement of prediction hit rate

This study is designed to directly and robustly quantify the core value of the MCF framework from a perspective of paramount importance to industrial applications. The primary scientific objective is to systematically validate whether the ‘golden dataset’, refined through the MCF framework, significantly enhances the prediction hit rate of various models under different error tolerances compared to the original raw data. To achieve this objective, the experiment contrasts two core datasets representing extreme states of data quality: $D_{Raw}$ , representing the original industrial data, and the ‘golden dataset’ $D_{s 1234}$ , embodying the ideal state achieved after comprehensive processing via the DCAI paradigm. Specifically, in the evaluation procedure, all six benchmark models are independently trained and assessed on both datasets. A rigorous 10-repetition, 10-fold cross-validation methodology is employed to obtain robust performance estimations. For each model-dataset combination, the average prediction hit rate is calculated at three critical error tolerance thresholds: ±5, ±10 and ±15 kWh/t. This direct comparison of a model's performance before and after data processing intuitively illustrates the resulting difference in hit rates, thereby compellingly demonstrating the pronounced superiority of the MCF framework in enhancing both the industrial applicability and reliability of the models.

Internal mechanism validation: Framework efficacy and synergy

Following the demonstration of the MCF framework's significant external efficacy via the hit rate metric, this section delves into its internal mechanisms to investigate the scientific validity and inherent rationale behind its success. This experiment encompasses two progressive scientific objectives. First, a ‘progressive efficacy’ assessment is conducted to quantify the positive and cumulative contribution of each processing stage within the MCF framework to the enhancement of data quality. Second, through “synergistic necessity” ablation studies, it aims to demonstrate that the framework's stages are not merely a simple stack of filters but rather exhibit indispensable, sequence-dependent synergistic interactions. To this end, a specialised hierarchical dataset system was constructed to precisely mirror the MCF processing pipeline. This system originates from the raw data $D_{Raw}$ and sequentially applies the four stages of the MCF framework ( $S_{1}$ , $S_{2}$ , $S_{3}$ and $S_{4}$ ) to generate a series of intermediate and final datasets ( $D_{s 1}$ , $D_{s 12}$ , $D_{s 123}$ and $D_{s 1234}$ ). Furthermore, comparative datasets for the ablation study, namely $D_{s 13}$ (omitting $S_{2}$ ) and $D_{s 124}$ (omitting $S_{3}$ ), were generated by selectively bypassing critical stages. The resulting hierarchical dataset structure is detailed in Table 2.

Table 2.

Table of progressive dataset construction for ablation experiments.

Dataset symbol	Dataset category	MCF stage of application	Features/examples
$D_{Raw}$	Baseline	None	130/2000
$D_{S 1}$	Progressive sequence	$S_{1}$	118/2000
$D_{S 12}$	Progressive sequence	$S_{1} + S_{2}$	118/1824
$D_{S 123}$	Progressive sequence	$S_{1} + S_{2} + S_{3}$	20/1166
$D_{S 1234}$	Progressive sequence	$S_{1} + S_{2} + S_{3} + S_{4}$	20/906
$D_{S 13}$	Ablation contrast	$S_{1} + S_{3}$ (skip $S_{2}$ )	20/2000
$D_{S 124}$	Ablation contrast	$S_{1} + S_{2} + S_{4}$ (skip $S_{3}$ )	118/1419

MCF: multi-stage collaborative filtering.

The evaluation procedure for this experiment is detailed as follows: to validate ‘progressive efficacy’, all six models from the benchmark library are sequentially trained and evaluated on the hierarchical dataset series ( $D_{Raw} \to D_{S 1} \to D_{S 12} \to D_{S 123} \to D_{S 1234}$ ). By employing a rigorous 10-repetition, 10-fold cross-validation, we quantify and track whether all prediction accuracy metrics for each model exhibit a distinct monotonic improvement trend as the data processing stages advance.

To verify ‘synergistic necessity’, we conduct an ablation study utilising all six models from the benchmark library. The core of this procedure involves two sets of direct performance comparisons, with RMSE and $r^{2}$ selected as the performance indicators: (1) a comparison of model performance on $D_{s 123}$ (complete pipeline) versus $D_{s 13}$ ( $S_{2}$ ) and (2) a comparison on $D_{s 1234}$ (complete pipeline) versus $D_{s 124}$ ( $S_{3}$ omitted). For each model-dataset combination involved in these comparisons, a strict 10-repetition, 10-fold cross-validation is executed to obtain robust performance scores. A significant degradation in model performance observed on the ablation datasets serves as direct evidence of the synergy and sequential dependence among the framework's stages.

Subsequent benefit assessment (I): Enhancement of model robustness

This experiment is designed to quantify and substantiate a core value transformation introduced by the DCAI paradigm: a strategic shift in the objective of machine learning applications from the conventional pursuit of ‘unstable average accuracy’ to ensuring the ‘high degree of robustness’ essential for industrial-grade applications. The central hypothesis is that a model trained on a dataset systematically purified by the MCF framework will exhibit significantly lower volatility in its predictive performance compared to its performance when trained on the noisy, original dataset. To validate the universality of this hypothesis, all six models from the benchmark library were utilised, with two evaluation configurations established for each model based on the $D_{Raw}$ and $D_{S 1234}$ datasets, respectively. The specific evaluation protocol involved executing a complete 10-repetition, 10-fold cross-validation procedure for each model-dataset combination, thereby generating 100 independent performance scores (RMSE and $r^{2}$ ). The analytical focus of this study is not the mean of these scores but rather the dispersion of their distribution. By calculating the standard deviation ( $σ_{Metric}$ ) of these 100 scores for each case, as detailed in equation (10), we systematically compare the stability difference exhibited by the same model on the two datasets. This approach serves to intuitively and compellingly reveal the fundamental role of the DCAI paradigm in suppressing model uncertainty at its source and achieving industrial-grade robustness.

Subsequent benefit assessment (II): Comparative evaluation of data cleaning strategies

This experiment is designed to objectively establish the technical superiority of the MCF framework by conducting a fair and comprehensive comparative performance analysis against benchmark methods widely utilised in academia and industry. For this purpose, the MCF framework is treated as an integrated, synergistic data cleaning strategy. Three representative benchmark strategies were designed for comparison: strategy A, a purely feature-centric approach, performs feature selection based solely on mutual information; strategy B, an instance-centric method, utilises the classic local outlier factor (LOF) algorithm to identify and remove outliers and strategy C simulates a simple, non-synergistic multi-step process by first executing strategy A and then strategy B. These three strategies were respectively applied to the original dataset $D_{Raw}$ , generating three new datasets: $D_{feature}$ , $D_{instance}$ and $D_{combined}$ . These datasets, together with the baseline $D_{Raw}$ and the MCF-processed $D_{S 1234}$ , constitute the five datasets for this comparative evaluation. For ease of subsequent discussion and graphical presentation, these five strategies (and their corresponding datasets) are designated as follows: strategy E ( $D_{Raw}$ ), strategy A ( $D_{feature}$ ), strategy B ( $D_{instance}$ ), strategy C ( $D_{combined}$ ) and strategy D ( $D_{S 1234}$ ). The evaluation procedure is centred on these five datasets. To ensure a fair comparison and assess the strategies’ effectiveness across different model types, the linear Lasso and the high-performance non-linear XGBoost were selected as evaluation models. These two models were combined with each of the five datasets, thereby constructing 10 independent ‘model-dataset’ evaluation combinations (e.g. the ‘Lasso on $D_{feature}$ ’ combination, the ‘XGBoost on $D_{S 1234}$ ’ combination). For each of these ten combinations, a rigorous 10-repetition, 10-fold cross-validation procedure was executed, yielding 100 independent performance scores. The analytical focus of this experiment is on the mean of these scores. By calculating the final average RMSE and $r^{2}$ for each combination and systematically comparing the average performance of the same model across the five strategically processed datasets, this analysis clearly and quantitatively reveals the pronounced superiority of the synergistic MCF strategy over the other benchmark methods.

Strategic value assessment: Comparative efficacy of the DCAI and MCAI paradigms

This experiment is designed to elevate the research perspective to a strategic level by addressing a long-standing, fundamental question in the field of AI research and development through a direct comparative experiment: For a given industrial problem, should limited R&D resources – including computational power, time and personnel – be preferentially allocated to the systematic enhancement of data quality (the DCAI paradigm), or to the exhaustive optimisation of model architectures and hyperparameter tuning (the MCAI paradigm)? To address this question equitably, two experimental groups with distinct but equivalently evaluated protocols were established. The DCAI group represents the ‘invest-in-data’ strategy, focusing on assessing the performance of baseline models with fixed, default hyperparameters on the high-quality ‘golden dataset’ $D_{S 1234}$ . In contrast, the MCAI group represents the ‘invest-in-model’ strategy, which involves first dedicating substantial computational resources to a large-scale grid search on the noisy and redundant original dataset $D_{Raw}$ to identify an ‘optimal’ set of hyperparameters for peak performance, and subsequently evaluating the performance of this hyperparameter-optimised model.

The evaluation procedure is structured to ensure absolute fairness in the final performance comparison between the two paradigms. To this end, corresponding DCAI and MCAI comparison configurations were constructed for both the Lasso and XGBoost models. The preparation for the MCAI configurations involved an intensive grid search on the $D_{Raw}$ dataset to obtain an ‘optimal’ hyperparameter set, whereas their DCAI counterparts utilised the default, untuned model parameters. Critically, once the model configurations were finalised – finely tuned for the MCAI group and default for the DCAI group – all configurations proceeded to an identical final evaluation phase. Specifically, for each ‘model-paradigm’ combination, including ‘DCAI-Lasso on $D_{S 1234}$ ’ versus ‘MCAI-Lasso on $D_{Raw}$ ’ and ‘DCAI-XGBoost on $D_{S 1234}$ ’ versus ‘MCAI-XGBoost on $D_{Raw}$ ’, a rigorous 10-repetition, 10-fold cross-validation was executed to generate 100 independent performance scores. The final analysis focuses on cross-paradigm comparisons within the same model class. By systematically comparing both predictive accuracy and robustness metrics, this experiment clearly reveals the substantial disparity in cost-effectiveness between the two paradigms, thereby offering strategic insights for selecting R&D pathways in industrial intelligence applications.

Baseline comparison setup with traditional statistical regression models

To address the discussion within the field regarding model complexity versus data quality, and to verify the superiority of the DCAI paradigm over early empirical statistical methods, this study introduces a dedicated set of baseline comparison experiments. These experiments simulate the classic statistical modelling workflow for EAF energy consumption per ton prediction from the 2000s, constructing a ‘Traditional Statistical Baseline’ that represents typical practices from early works such as Köhle and Conejo.^4–9

The baseline workflow comprises the following steps:

Domain knowledge-based feature selection: In contrast to the automated collaborative mechanism of MCF, this baseline exclusively selects eight core variables recognised in the literature as input features: total scrap weight, pig iron addition, power-on time, oxygen blowing volume, carbon injection amount, smelting interval time, average voltage and average current. This selection directly references the variable sets used in classic studies by Köhle and Conejo.

Simple statistical regression modeling: MLR and ridge regression are employed as predictors. These methods possess high interpretability and were mainstream choices in early research.

The performance of the aforementioned traditional baseline on the raw dataset will be compared against the same model type processed by the MCF framework (MCF + Ridge) and a high-performance model (MCF + XGBoost). All experiments adhere to the evaluation protocol detailed in the ‘Evaluation protocol’ section (10 repetitions of 10-fold cross-validation) to quantify the independent contribution of data quality engineering to prediction performance.

Results and discussion

This chapter aims to systematically present and thoroughly discuss the experimental results derived from the designs detailed in chapter 4. In a progressive manner, it validates the core value, internal mechanisms and strategic position of the DCAI paradigm within the complex industrial scenario of EAF energy consumption prediction. The discussion strictly adheres to the logical sequence established in chapter 4: commencing with an empirical demonstration of core advantages, proceeding to a scientific analysis of internal mechanisms, followed by a quantitative assessment of multi-dimensional benefits, and culminating in a strategic reflection on R&D paradigms. Finally, through a comparative benchmarking against traditional statistical regression baselines, this chapter confirms the superiority of the proposed DCAI framework over both existing and historical methods.

Core advantage validation: Significant leap in prediction hit rate

The primary finding of this research, which also serves as the most direct and compelling evidence for the data-centric philosophy, is manifested in the fundamental improvement of model prediction hit rates. As detailed in Table 3, all benchmark models, when trained on the ‘golden dataset’ ( $D_{S 1234}$ ) processed by the MCF framework, achieved a transformative enhancement in their prediction hit rates compared to their performance on the original raw dataset ( $D_{Raw}$ ). A detailed analysis of these results reveals several key trends. First, a decisive breakthrough was achieved in the high-precision prediction interval (±5 kWh/t) for the SEC (i.e. energy consumption per ton of steel). On the raw data, even the sophisticated XGBoost model lingered at a low hit rate of only 45.32%, offering limited value for practical production guidance. Conversely, on the ‘golden dataset’, this metric was substantially enhanced for all models, including the structurally simpler linear ones. For instance, the hit rate for XGBoost increased to 66.73%, consistently reaching a level acceptable for industrial application. This compellingly demonstrates that high-quality data is the absolute cornerstone for achieving high-precision predictions, with its impact far outweighing that of increased model complexity. Second, within a generally acceptable industrial error tolerance (e.g. ±15 kWh/t), models trained on the ‘golden dataset’ universally achieved hit rates exceeding 90%, exhibiting a pronounced reliability advantage, whereas models trained on the raw data fell far short of this benchmark and exhibited inconsistent performance. Finally, the data purification effect of the MCF framework holistically elevated both the lower and upper performance bounds of all models, indicating that enhanced data quality empowers all downstream models to realise their full potential. This result irrefutably proves that systematic data purification, rather than the mere pursuit of model complexity, is the core driver for fundamentally enhancing the accuracy and reliability of models in practical industrial applications.

Table 3.

Comparison table of prediction hit rates between datasets.

	±5 kWh/t/%		±10 kWh/t/%		±15 kWh/t/%
	$D_{Raw}$	$D_{s 1234}$	$D_{Raw}$	$D_{s 1234}$	$D_{Raw}$	$D_{s 1234}$
Lasso	33.41	51.12	40.13	56.23	60.12	82.22
Ridge	34.54	52.23	42.22	57.34	62.23	83.32
SVR	36.45	54.32	44.32	57.12	63.32	84.23
LightGBM	46.23	65.12	55.32	80.12	70.23	90.12
XGBoost	45.32	66.73	54.33	81.23	71.33	92.23
RF	44.77	68.23	53.54	83.32	74.32	91.14

RF: random forests; SVR: support vector regression.

Internal mechanisms of performance enhancement: analysis of framework efficacy and synergy

Following the direct observation of the substantial enhancement in prediction hit rates, this study delves further into the internal mechanisms of the MCF framework to elucidate its scientific validity. As illustrated by the ‘progressive efficacy’ evaluation in Figure 2, the key performance metrics of all models exhibited a clear and consistent trend of improvement as the processing stages of the MCF framework advanced (from $D_{Raw}$ to $D_{S 1234}$ ). Specifically, the prediction error metrics (RMSE, MSE and MAE) for all models demonstrated a stepwise monotonic decrease, while the coefficient of determination ( $r^{2}$ ) showed a steady monotonic increase. This robustly proves that each stage of the MCF framework – feature pruning, global denoising, feature refinement and boundary concentration – makes a positive and cumulative contribution to data quality. However, the ablation study results, presented in Table 4, further reveal that these stages are not a simple linear superposition but possess an indispensable and sequence-dependent synergy. When comparing $D_{S 123}$ (complete pipeline) with $D_{S 13}$ ( $S_{2}$ – global denoising omitted), the performance of all models deteriorated significantly; for instance, the RMSE of XGBoost sharply increased from 7.12 to 12.05, while its $r^{2}$ dropped from 0.85 to 0.79. This profoundly reveals that without the initial removal of anomalous noise points via global denoising, these outliers would severely distort the true statistical relationships between features and the target variable. Consequently, the subsequent feature refinement process ( $S_{3}$ ) would be critically misguided, leading to the selection of a suboptimal feature combination. Similarly, a comparison between $D_{S 1234}$ (complete pipeline) and $D_{S 124}$ ( $S_{3}$ – feature refinement omitted) also showed a marked decline in model performance; for example, the RMSE for Lasso increased from 8.33 to 9.21. This indicates that without first stabilising and refining the core feature space through feature selection, the very concept of ‘neighbourhood’ based on distance metrics becomes unreliable within the high-dimensional, redundant space. This, in turn, prevents the final boundary instance concentration ( $S_{4}$ ) from being executed accurately, leading to the erroneous removal of samples. These findings fundamentally explain why a simple imitation of a multi-step cleaning strategy is markedly inferior to the MCF framework: it lacks the core synergistic logic we designed of ‘first stabilising the feature space, then purifying the instance distribution’.

Figure 2.

Evaluation chart of framework stage effectiveness: (a) MSE, (b) RMSE, (c) MAE and (d) $r^{2}$ . MAE: mean absolute error; MSE: mean squared error; RMSE: root mean squared error.

Table 4.

Performance comparison table for ablation experiments.

	RMSE				$r^{2}$
	$D_{S 13}$	$D_{S 123}$	$D_{S 124}$	$D_{S 1234}$	$D_{S 13}$	$D_{S 123}$	$D_{S 124}$	$D_{S 1234}$
Lasso	15.39	9.42	9.21	8.33	0.67	0.69	0.68	0.71
Ridge	16.03	9.45	10.12	9.21	0.63	0.69	0.66	0.70
SVR	16.41	12.79	12.08	10.11	0.53	0.57	0.52	0.58
LightGBM	11.81	7.12	7.10	6.32	0.81	0.88	0.76	0.91
XGBoost	12.05	7.12	8.86	6.93	0.79	0.85	0.77	0.89

RMSE: root mean squared error; SVR: support vector regression.

Subsequent benefit I: A paradigm shift from pursuing accuracy to ensuring robustness

High-quality data not only yields superior average prediction accuracy but, more importantly, facilitates a paradigm shift in model reliability, transitioning from quantitative improvement to a qualitative leap. As illustrated by the robustness evaluation in Figure 3, models trained on the original dataset, $D_{Raw}$ , exhibit high standard deviations for both RMSE and $r^{2}$ metrics. This indicates that their predictive performance fluctuates drastically across different data subsets, signifying poor reliability and rendering them untrustworthy for critical industrial decision making. However, upon switching the training data to the $D_{S 1234}$ dataset, which was purified by the MCF framework, a cliff-like drop was observed in the standard deviations of both RMSE and $r^{2}$ for all models. This significant change signifies that the predictive outcomes of the models have become highly stable and reliable. The essence of this transformation lies in the MCF framework's ability to systematically remove noise, redundancy and ambiguous boundaries, thereby substantially reducing the intrinsic uncertainty within the data itself. Consequently, the models are enabled to learn more generalisable principles that are closer to the underlying physical essence, rather than merely fitting to stochastic fluctuations in the data. In practical production scenarios, a model that consistently delivers predictions within a reasonable error margin is of far greater applied value than one that achieves high average accuracy but occasionally produces egregiously inaccurate predictions. Therefore, the transition from ‘pursuing unstable average accuracy’ to ‘guaranteeing industrial-grade robustness’, as realised by the MCF framework, represents a critical step in advancing predictive models from laboratory research to frontline production applications.

Figure 3.

Comparison chart of model robustness metrics.

Subsequent benefit II: Comprehensive performance superiority of the MCF strategy

To objectively evaluate the comprehensive performance of MCF as a holistic data cleansing strategy, a comparative analysis was conducted against benchmark strategies commonly employed in academia and industry. As illustrated by the experimental results in Figure 4, the MCF framework (strategy D) demonstrated overwhelming technical superiority. Compared to the baseline performance of strategy E (utilising the original data, $D_{Raw}$ ), the performance enhancements from single-dimensional approaches were markedly limited. Specifically, both strategy A (focusing solely on feature selection via Mutual Information) and strategy B (concentrating exclusively on instance-level outlier detection via LOF) yielded marginal or even detrimental results, exposing the inherent limitations of such unilateral methods. Furthermore, even strategy C, which emulates a multi-step procedure by serially combining strategies A and B, performed significantly worse than the MCF framework due to the absence of a synergistic design. For both the Lasso and XGBoost test models, the MCF framework (strategy D) consistently achieved the lowest prediction error, as measured by RMSE, and the highest coefficient of determination ( $r^{2}$ ). These results, which comprehensively surpass all other comparative strategies including the baseline, provide compelling evidence for the scientific validity and superiority of the ‘multi-stage cooperative filtering’ design inherent to MCF. The framework is not a mere agglomeration of techniques but rather an integrated, organic system wherein each stage is intricately linked and mutually reinforcing. Its superior performance, therefore, stems from this intrinsic synergistic logic rather than from a simple concatenation of individual technologies.

Figure 4.

Performance comparison chart of data cleaning strategies: (a) RMSE and (b) $r^{2}$ . RMSE: root mean squared error.

Strategic value proposition: The cost-effectiveness of the DCAI paradigm

The most transformative and practically significant finding of this research, illustrated in Figure 5, emerges from the cross-paradigm cost-effectiveness analysis. This quantitative comparison reveals two pivotal discoveries. First, from the perspective of comparing analogous models, data quality constitutes the definitive factor for achieving superior performance and robustness. Specifically, in direct comparisons of MCAI-Lasso versus DCAI-Lasso and MCAI-XGBoost versus DCAI-XGBoost, the DCAI groups (employing default parameters with ‘golden data’) consistently exhibited substantially lower error metrics than the MCAI groups (utilising fine-tuned parameters with raw data). Concurrently, a drastic reduction in the standard deviation of performance was observed. This provides direct evidence that high-quality data serves as the core lever for achieving a quantum leap in model performance and ensuring industrial-grade robustness. Second, when comparing simple versus complex models across these paradigms, the DCAI framework demonstrates an overwhelming strategic advantage. The most compelling experimental result is that the ‘foundational DCAI’ approach, represented by DCAI-Lasso (a basic model trained on ‘golden data’), surpassed the ‘advanced MCAI’ approach, represented by MCAI-XGBoost (a complex model trained on raw data), in terms of both prediction accuracy and robustness. This asymmetric comparison compellingly demonstrates the immense strategic value of the DCAI paradigm: investing in systematic data quality enhancement is a more highly leveraged strategy than engaging in the perpetual ‘arms race’ of model architecture optimisation and hyperparameter tuning. Consequently, this study provides a clear roadmap for the research and development of industrial intelligent systems, establishing that a return to fundamentals – solidifying the data foundation – is the strategic cornerstone for the large-scale, high-value deployment of data-driven intelligent technologies.

Figure 5.

Benefit comparison chart between DCAI and MCAI paradigms: (a) MSE, (b) RMSE, (c) MAE and (d) $r^{2}$ . DCAI: data-centric artificial intelligence; MAE: mean absolute error; MCAI: model-centric artificial intelligence; MSE: mean squared error; RMSE: root mean squared error.

Comparative analysis with traditional statistical regression models

Table 5 presents the comparative results on predictive performance between the Traditional Statistical Baseline (limited domain knowledge-based features + raw data) and MCF-enhanced models. This baseline reflects the typical performance of the early statistical paradigm.

Table 5.

Performance comparison between traditional statistical baselines and MCF-enhanced models.

Model scenario	Feature set	RMSE (kWh/t)	R ²
Traditional Baseline (MLR)	8 Core Features (Domain Knowledge)	58.42	0.32
Traditional Baseline (Ridge)	8 Core Features (Domain Knowledge)	52.78	0.35
MCF + Ridge	Optimal Features (MCF Selected)	9.21	0.70
MCF + XGBoost	Optimal Features (MCF Selected)	6.93	0.89

MCF: multi-stage collaborative filtering; MLR: multiple linear regression; RMSE: root mean squared error.

The results indicate that traditional statistical regression models exhibited poor performance under raw data and limited features ( $R^{2}$ ranging from approximately 0.3–0.35, RMSE>50 kWh/t), which is consistent with early literature.^4–7 Relying solely on a small set of variables selected by domain expertise and linear assumptions makes it difficult to effectively capture the complex non-linear dynamics and operational fluctuations of modern DC EAFs.

In contrast, models empowered by the MCF framework achieved significant improvements in the same metrics: the RMSE of the Ridge model decreased to 9.21 kWh/t, with $R^{2}$ increasing to 0.70; the XGBoost model further achieved an RMSE of 6.93 kWh/t and an $R^{2}$ of 0.89. This comparison highlights the critical role of systematic feature optimisation and data purification in enhancing predictive performance.

Limitations and future outlook

While this research provides robust empirical evidence for the application of the DCAI paradigm in process industries and validates the effectiveness of the MCF framework, it is imperative to objectively acknowledge its limitations and delineate directions for future research.

First, the scope of validation in this study is subject to certain constraints. The construction of the MCF framework and the verification experiments rely strictly on production data collected from two specific 130-ton DC EAFs. Consequently, the features and patterns learned by the models are intrinsically tied to the specific equipment parameters, raw material structures and operational habits of these furnaces. This study has not verified the direct applicability of the model to other types of EAFs (such as AC EAFs) or other process industries (such as chemical and cement production). Direct transfer to new scenarios without adaptation may limit model performance.

Second, regarding hyperparameter configuration, certain critical hyperparameters within the current MCF framework (e.g. the contamination rate of the Isolation Forest and the VIF threshold) depend on preliminary statistical analysis of the specific dataset and domain expertise. Although these settings are effective in the current scenario, the framework currently lacks a fully automated mechanism for hyperparameter optimisation. Consequently, manual recalibration may be required when facing new scenarios with significantly different data distributions.

Finally, the application scenario and data modality are relatively limited. This study focuses primarily on discrete tabular statistical data from individual heats, without incorporating the rich high-frequency time-series signals (such as current and voltage waveforms) or unstructured data available in industrial settings. To some extent, this limits the potential for leveraging deep learning models to mine finer-grained dynamic features.

Looking ahead, based on the aforementioned limitations, we will focus on the following research directions: (1) cross-domain generalisation and transfer learning, aiming to verify the framework's adaptability across different equipment and industries to enhance universality; (2) automated hyperparameter optimisation, exploring the introduction of AutoML or meta-learning mechanisms to reduce the cost of manual intervention and (3) multi-modal data fusion, integrating time-series and image data to construct a more holistic and intelligent industrial prediction system.

Conclusion

Addressing the data quality bottleneck in the prediction of energy consumption per ton of steel for EAF steelmaking, this study proposed and systematically validated a DC MCF framework. The primary conclusions are as follows:

This study successfully designed and validated an innovative four-stage MCF framework. Ablation experiments decisively demonstrated the synergistic necessity of its internal stages: without first removing outliers via global instance denoising ( $S_{2}$ ), the subsequent feature selection ( $S_{3}$ ) is severely misguided; similarly, without first stabilising the core feature space through feature selection ( $S_{3}$ ), the subsequent boundary instance condensation ( $S_{4}$ ) cannot be executed accurately. This confirms that the MCF is not a mere aggregation of techniques but an organic, integrated system adhering to the core logic of ‘first stabilising the feature space, then purifying the instance distribution’.

Furthermore, the DCAI paradigm achieves a fundamental shift in the reliability of data-driven applications. Quantitative results show that training models on the ‘golden dataset’ refined by the MCF framework leads to a substantial reduction in the standard deviation of prediction performance. This marks a pivotal transition in the value proposition of data-driven models, moving from the pursuit of potentially unstable average accuracy to the assurance of high robustness and reliability, which are indispensable for industrial production.

Finally, the DCAI paradigm establishes a core strategic value for the research and development of intelligent systems. The central finding from our cross-paradigm comparison is that a basic, untuned Lasso model trained on the ‘golden dataset’ comprehensively surpasses a complex XGBoost model – which was meticulously optimised on raw data at significant computational cost – in both robustness and prediction accuracy. This asymmetric outcome provides compelling evidence that investing in systematic data quality enhancement is a more highly leveraged strategy than model tuning and represents an indispensable pathway for the high-value implementation of industrial intelligent applications.

Limitations and future research directions: While this study confirms the effectiveness of the MCF framework on specific EAF datasets, future research will focus on three strategically significant directions to broaden its industrial applicability: (1) cross-domain generalization and transfer learning: To address the limitations of single-source validation on DC EAFs, we aim to collect datasets from diverse furnace types (e.g. AC EAFs) and other process industries (e.g. cement and chemicals), exploring transfer learning techniques to verify cross-scenario adaptability; (2) multi-modal data fusion: We plan to integrate high-frequency time-series sensor data (e.g. current and voltage waveforms) to enable deep learning models, such as long short-term memories or transformers, to capture finer-grained dynamic process features; (3) adaptive hyperparameter optimisation: We will develop AutoML modules based on meta-learning or Bayesian optimisation to reduce the framework's reliance on domain expertise, thereby achieving adaptive configuration across different industrial scenarios.

Footnotes

ORCID iD

Yangchun Wang

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Hay

Echterhof

Visuri

, et al. Development of an electric arc furnace simulator based on a comprehensive dynamic process model. Processes 2019; 7: 852.

Abadi

Tang

Rashidi

. A review of simulation and numerical modeling of electric arc furnace (EAF) and its processes. Heliyon 2024; 10: e32157.

World Steel Association. Steel's contribution to a low carbon future and climate resilient societies – world steel position paper. Brussels: World Steel Association, 2020, ISBN: 978-2-930069-83-8.

Köhle

. Effects on the electric energy and electrode consumption of arc furnaces. Stahl Eisen 1992; 112: 59–67.

Köhle

. Recent improvements in modelling energy consumption of electric arc furnaces. In: Proceedings of the 7th European Electric Steelmaking Conference (EEC), Venice, Italy, 2002, pp. 305–314.

Pfeifer

Kirschen

. Thermodynamic analysis of EAF energy efficiency and comparison with a statistical model of electric energy demand. In: Proceedings of the 7th European Electric Steelmaking Conference (EEC), Venice, Italy, 2002, pp. 1.413–1.428.

Conejo

Cárdenas

. Energy consumption in the EAF with 100% DRI. In: Proceedings of the AISTech 2006 Conference, Vol. 1, Cleveland, OH, USA, 2006, pp. 529–535.

Sandberg

Bentell

Undvall

. Energy optimisation of electric arc furnaces by statistical process evaluation. In: Proceedings of the 7th European Electric Steelmaking Conference (EEC), Venice, Italy, 2002, pp. 435–443.

Czapla

Karbowniczek

Michaliszyn

. The optimisation of electric energy consumption in the electric arc furnace. Arch Metall Mater 2008; 53: 559–565.

10.

Wang

Jin

Zhu

. Modeling and prediction of electric arc furnace based on neural network and chaos theory. In: Lecture Notes in Computer Science, Vol. 3498, 2005, pp. 819–826. Springer. DOI: 10.1007/11427469_130

11.

Carlsson

Samuelsson

Jönsson

. Predicting the electrical energy consumption of electric arc furnaces using statistical modeling. Metals (Basel) 2019; 9: 959.

12.

Logar

Dovžan

Škrjanc

. Modeling and validation of an electric arc furnace: part 1, heat and mass transfer. ISIJ Int 2012; 52: 402–412.

13.

Tomažič

Andonovski

Škrjanc

, et al. Data-driven modelling and optimization of energy consumption in an electric arc furnace. Metals (Basel) 2022; 12: 816.

14.

Chen

Liu

Kumar

, et al. Energy consumption modelling using deep learning technique – a case study of EAF. Procedia CIRP 2018; 72: 1063–1068.

15.

Meng

Finley

, et al. LightGBM: a highly efficient gradient boosting decision tree. In: Advances in Neural Information Processing Systems 30 (NIPS), Long Beach, USA, 2017, pp. 3146–3154.

16.

Chen

Guestrin

. XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, USA, 2016, pp. 785–794. DOI: 10.1145/2939672.2939785

17.

Cheng

Møller

, et al. Data issues in industrial AI system: a meta-review and research strategy. Comput Ind 2025; 173: 104361.

18.

Saparrat

Monti

Ibarra

. AI Application to melting temperature prediction in an electric arc furnace. In: Proceedings of the AISTech 2020 Conference, Pittsburgh, PA, USA, 2020, pp. 40–45. DOI: 10.33313/380/060.

19.

Niyayesh

Uygun

. Predicting endpoint parameters of electric arc furnace-based steelmaking using artificial neural network. Int J Adv Manuf Technol 2024; 138: 155–167.

20.

Yang

Wang

. Taming silent failures: a framework for verifiable AI reliability. IEEE Reliab Mag 2025; 2: 46–55.

21.

Nguyen

Shui

Giri

, et al. Reliably detecting model failures in deployment without labels. In: Advances in Neural Information Processing Systems 39 (NeurIPS), San Diego, USA, 2025. DOI: 10.48550/arXiv.2506.05047.

22.

Jarrahi

Memariani

Guha

. The principles of data-centric artificial intelligence. Commun ACM 2023; 66: 84–92.

23.

Jakubik

Vössing

Kühl

, et al. Data-centric artificial intelligence: a survey. Bus Inf Syst Eng 2024; 66: 507–515.

24.

Zha

Bhat

Lai

, et al. Data-centric artificial intelligence: a survey. ACM Comput Surv 2025; 57: 1–42.

25.

Mazumder

Banbury

Yao

, et al. Dataperf: benchmarks for data-centric AI development. In: Advances in Neural Information Processing Systems 36 (NeurIPS), New Orleans, USA, 2023, pp. 5320–5347.

26.

Luley

Deriu

Yan

, et al. From concept to implementation: the data-centric development process for AI in industry. In: Proceedings of the 2023 10th IEEE Swiss Conference on Data Science (SDS), 2023, pp. 73–76. IEEE. DOI: 10.1109/SDS57534.2023.00017

27.

Yepmo

Smits

Lesot

, et al. CADI: contextual anomaly detection using an isolation forest. In: Proceedings of the 39th ACM/SIGAPP Symposium on Applied Computing, Avila, Spain, 2024, pp. 1–8. DOI: 10.1145/3605098.3635969

28.

Reimann

Hay

Echterhof

, et al. Application and evaluation of mathematical models for prediction of the electric energy demand using plant data of five industrial-size EAFs. Metals (Basel) 2021; 11: 1348.

29.

Gajic

Savić-Gajic

Savić

, et al. Modelling of electrical energy consumption in an electric arc furnace using artificial neural networks. Energy 2016; 108: 132–139.

30.

Liu

Ting

Zhou

. Isolation forest. In: Proceedings of the 2008 IEEE International Conference on Data Mining, 2008, pp. 413–422. IEEE. DOI: 10.1109/ICDM.2008.17

31.

Tibshirani

. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B (Methodol) 1996; 58: 267–288.

32.

Efron

Hastie

Johnstone

, et al. Least angle regression. Ann Stat 2004; 32: 407–451.

33.

Jung

Cole

. Predictive modeling of high-entropy alloys and amorphous metallic alloys using machine learning. J Chem Inf Model 2024; 64: 7313–7336.

34.

Zhao

Zhang

Liaw

, et al. Machine learning-based computational design methods for high-entropy alloys. High Entropy Alloys Mater 2025; 3: 41–100.

35.

Sahar

Rauf

Murtaza

, et al. Anomaly detection in laser powder bed fusion using machine learning: a review. Results Eng 2023; 17: 100803.

36.

Mattera

Nele

. Machine learning approaches for real-time process anomaly detection in wire arc additive manufacturing. Int J Adv Manuf Technol 2025; 137: 2863–2888.

37.

Coelho

Zhang

Van Ingelgem

, et al. Reviewing machine learning of corrosion prediction in a data-oriented perspective. npj Mater Degrad 2022; 6: 8.

38.

Kordos

. Data selection for neural networks. Schedae Inform 2016; 25: 153–164.

39.

Kutner

Nachtsheim

Neter

, et al. Applied linear statistical models. 5th ed. New York: McGraw-Hill/Irwin, 2005.

40.

Wilson

. Asymptotic properties of nearest neighbor rules using edited data. IEEE Trans Syst Man Cybern 1972; 2: 408–421.

41.

Shwartz-Ziv

Armon

. Tabular data: deep learning is not all you need. Inf Fusion 2022; 81: 84–90.