A novel restoration transformer for blast furnace top infrared images

Abstract

Infrared (IR) images captured by the blast furnace (BF) top system suffer from severe degradation, including low contrast and texture loss, due to dust, steam, and thermal glare. These issues critically impair image-based monitoring and visual algorithms. Traditional IR enhancement methods only address simple noise, while visible light dehazing algorithms fail on pseudo-colour IR images with low contrast and unnatural textures. To overcome these limitations, we propose an IR image restoration transformer called IRReFormer, a deep fusion model targeting BF top IR image degradation characteristics. IRReFormer integrates the advantages of both traditional IR enhancement and visible light dehazing algorithms, incorporating three key components: (1) a frequency feature enhancement self-attention for improving global feature representation, (2) a discrete wavelet transform block for multi-scale frequency decomposition, and (3) a spatial detail enhancement block for progressive texture recovery. Experimental results on the restoration of BF top IR hazy images demonstrate that the proposed method achieves superior performance with peak signal-to-noise ratio (PSNR) of 25.40 dB and structural similarity index measure (SSIM) of 0.831, surpassing the current state-of-the-art methods (PSNR 24.77 dB, SSIM 0.816) on standard evaluation metrics.

Keywords

blast furnace discrete wavelet transform frequency feature enhancement infrared image restoration spatial detail enhancement

Introduction

Infrared (IR) imaging exhibits superior interference resistance and lower dependence on lighting conditions compared to visible light imaging, making it widely applicable in remote sensing, healthcare, and industrial inspection.^1,2

Within industrial applications, the blast furnace (BF) is a critical scenario for IR imaging. Operating under high temperatures, pressure, and complex reactions within a large, sealed vessel, the BF interior presents harsh conditions. Significant dust, steam, and variable lighting impede visible light imaging. Consequently, IR imaging systems³ installed at the BF top (BF top imaging systems) are essential for internal monitoring.

Real-time IR video from these systems enables operators to monitor burden surface geometry, lining wear, and chute operation while providing localised temperature data. This information facilitates an understanding of gas flow distribution and intensity, enabling precise operational decisions.

For instance, precise chute angle configuration is critical for stable BF operation. However, due to mechanical wear, material buildup, or transmission errors, the actual chute angle may deviate from the preset value during operation. Such deviations can adversely affect material distribution and furnace stability. Traditional measurement methods^4–7 employ mechanical devices (e.g. cross-shaped rods) to directly measure inclination angles or utilise sensors to sample points on the chute surface for geometric angle calculation. However, these approaches can only be implemented during planned BF shutdowns or overhauls, which typically occur only two to three times annually,⁸ failing to meet real-time industrial monitoring requirements. To address this, He et al.⁹ developed a real-time chute angle diagnosis algorithm based on a Res-LSTM model, which maps features from BF top IR images to angle values. Nevertheless, the performance of such vision-based algorithms is susceptible to image quality.

The harsh BF environment, particularly during charging, introduces significant noise into IR images. The process of adding materials and blowing hot air generates substantial dust, steam, and thermal glare. In contrast, images captured during charging intervals are clearer due to reduced disturbance as reactions stabilise. As shown in Figure 1, comparing clear (interval) and hazy (charging) images at chute angles of $20 \circ$ , $30 \circ$ , and $40 \circ$ reveals a frequency-mixed degradation: dust or steam scattering causes low-frequency blurring (e.g. at edges), while thermal glare introduces high-frequency noise in low-frequency backgrounds.

To quantify the impact of image degradation on visual task accuracy, we applied the Res-LSTM chute angle detection algorithm.⁹ Results showed a drastic accuracy drop from 99.79% on clear interval images to 54.13% on degraded charging-phase images. This underscores the urgent need for effective BF top IR image restoration algorithms to provide high-quality input for subsequent visual tasks such as chute angle detection.

Current research on industrial image restoration can be broadly divided into two categories: IR enhancement and visible-light dehazing. Traditional IR enhancement algorithms^10,11 often fail under complex BF conditions, resulting in blurring or distortion. Deep learning-based IR enhancement methods, such as those for low-light enhancement¹² and super-resolution,¹³ struggle with the complex, mixed noise in BF top IR images due to data scarcity and domain differences. On the other hand, visible-light dehazing algorithms^14–16 offer valuable insights for handling haze-like degradation, but they are designed for natural visible images and face a modality gap when applied to pseudo-colour IR images with low contrast and sparse textures.

To bridge these gaps, we propose IRReFormer, a novel restoration algorithm for BF top IR images that integrates principles from both IR enhancement and visible-light dehazing. The main contributions are:

We propose the IR image restoration former (IRReFormer), integrating joint spatial, frequency, and wavelet domain processing. IRReFormer excels at restoring BF top IR images, significantly improving quality for downstream vision tasks. Experiments on a BF IR dataset demonstrate its superiority over existing methods.

To address inherent IR challenges – low resolution, contrast, and colour texture – we developed a frequency domain enhancement module combining spatial and frequency features via windowed self-attention. Alongside two important modules – a wavelet transform module and a spatial detail enhancement (SDE) module – they collectively manage multi-scale features and mitigate noise.

Our algorithm delivers practical impact: Applying it to degraded BF top IR images increased chute angle detection accuracy from 54.13% to 99.65% in challenging industrial environments. The framework enables reliable real-time vision tasks in harsh industrial environments such as BF by providing downstream algorithms with quality-restored imagery that meets operational requirements.

Related work

IR image enhancement algorithms

Traditional IR image enhancement algorithms primarily utilise mathematical models, physical principles, and signal processing techniques. Histogram-based methods include contrast-limited adaptive histogram equalisation (CLAHE),¹⁰ which performs localised equalisation within image subregions while constraining contrast amplification, and its enhanced variant CLAHE-DGC¹⁷ that dynamically adjusts clip limits based on texture complexity and applies dual gamma correction. Filtering-based approaches such as the weighted guided filter (IE-WGF)¹⁸ decompose images into base and detail layers, using CLAHE for base-layer contrast balancing and nonlinear functions for detail enhancement. Physics-inspired methods such as the gravity-based IE-GFLIN¹¹ leverage gravitational force analysis to distinguish noise from edges, combining dual-threshold filtering with lateral inhibition networks for edge sharpening and contrast adjustment. However, these methods often rely on simplified assumptions that do not hold in complex BF environments, leading to unsatisfactory performance.

Deep learning-based approaches have significantly advanced IR enhancement: For low-light conditions, HWMNet¹⁹ employs semi-wavelet attention blocks in a hierarchical model to enhance wavelet-domain features, while MMFFNet¹² integrates multi-scale adaptive feature fusion with local refinement modules for brightness recovery. In super-resolution, PSRGAN¹³ utilises deep wavelet residual blocks under data constraints, and DASRGAN²⁰ implements domain-adaptive mechanisms targeting texture refinement and noise suppression. Nevertheless, these methods primarily target general brightness enhancement or resolution improvement, and often amplify noise when applied to severely degraded BF top IR images.

Visible light image dehazing algorithms

Traditional dehazing methods rely on physical models such as the atmospheric scattering model,²¹ which assumes uniform illumination – invalid in BF environments. Recent deep learning approaches have evolved through three paradigms: convolutional neural networks (CNNs) as seen in DEA-Net,¹⁶ which combines detail-enhanced convolution with content-guided attention; generative adversarial networks exemplified by EPDN²² using multi-resolution generators without physical priors; and transformer-based methods, including MB-T²³ that achieves linear complexity through Taylor-expansion attention approximation, Dehaze-F¹⁵ with enhanced spatial aggregation, and SFSNiD²⁴ addressing night fog via frequency interaction and semi-supervised learning. Data-centric innovations such as distribution alignment²⁵ and cross-dataset registration²⁶ further improve model generalisation.

Despite their success in visible-light images, these methods face fundamental challenges when applied to IR images due to differences in imaging principles, contrast, and texture characteristics. The unique, frequency-mixed degradation in BF top IR images necessitates a tailored approach that synergistically combines insights from both IR enhancement and dehazing.

Our approach

Data preparation

Supervised learning enables precise model training but requires real-world paired datasets – a significant challenge due to the impracticality of simultaneously capturing hazy and clear images under identical conditions.

Notably, we leverage the rotational consistency of the BF chute to overcome this limitation. As shown in Figure 2, the chute rotates at constant angular velocity $ω$ with a period of 7.86 s, generating 393 frames per cycle. During charging phases, images exhibit haze from glare, dust, and moisture; whereas inter-charging intervals yield clear frames. By recording continuous IR video of chute operation, we exploit positional correspondence: Every 393 frame captures identical chute positions, enabling reliable extraction of real-world paired hazy-clear images. Dataset processing details are provided in the ‘Dataset and metrices’ section.

Network structure

Despite leveraging chute-based positional correspondence, acquiring real-world paired datasets under complex BF conditions remains challenging. To address data scarcity, our IRReFormer employs a modified U-Net architecture²⁷ optimised for effective learning from limited samples. As illustrated in Figure 3, the framework integrates hierarchical feature extraction with multi-scale supervision to resolve the hazy-to-clear image mapping.

The three-layer U-Net processes multi-scale inputs through progressive refinement. An initial convolutional layer (ConvI) projects images into feature space. The encoder pathway successively applies convolutional layers with downsampling, halving spatial dimensions to extract high-level semantics. Conversely, the decoder pathway fuses two information streams via upsampling and $1 \times 1$ convolutions: (1) high-resolution details from encoder skip connections and (2) contextual semantics from prior decoder outputs. This dual-stream integration, finalised by a convolutional output layer (ConvO), preserves critical textures while reconstructing clear images.

A crucial innovation is multi-scale supervision during training, where intermediate outputs provide hierarchical guidance to iteratively refine global structures and local details. Though only the final output is retained at inference, it benefits from integrated multi-scale feature optimisation.

As shown in Figure 4, each encoder (decoder) block integrates three core components: the frequency feature enhancement self-attention (FFE-SA) block, discrete wavelet transform (DWT) block with deformable convolution, and SDE block. This multi-domain design synergistically combines principles from visible-light dehazing and IR image enhancement to address the distinct challenges of BF top IR restoration. The FFE-SA block adopts a frequency-domain decoupling strategy, akin to advanced dehazing methods, to separate and process the low-frequency blur (from dust or steam) and high-frequency noise (from thermal glare). Concurrently, the DWT block enhances multi-scale local details, and the SDE block sharpens edges through differential convolutions – both mechanisms are core to IR contrast and detail enhancement. By fusing frequency-aware degradation decoupling with spatial and wavelet-domain detail recovery, our model effectively restores images affected by the complex, mixed interference characteristic of harsh BF environments. The computational operations are detailed in Table 1.

Table 1.

The symbols for the computational operations.

Symbols	Operations
$λ$	ReLU
$δ$	Sigmoid
$s f$	Softmax
$τ$	Tensor mean
$ψ$	Tensor maximum
$S_{↑}$	Upsample
$S_{↓}$	Downsample
$\otimes$	Hadamard product
$⊙$	Matrix multiplication
$\oplus$	Addition
©	Concatenation
$C_{s}^{k}$	Standard convolution with kernel size $k$ and stride $s$
${DConv}_{k \times k}$	Deformable convolution with kernel size $k \times k$
DEConv	Detail enhancement convolution (combination of five different convolutions)

FEE-SA block

Considering that IR images lack texture information, processing solely in the spatial domain makes it difficult to effectively extract important features from the images. Moreover, the noise in hazy images is coupled and has inconsistent frequencies. In BF top IR images, this manifests as a mixture of low-frequency blur (from dust/steam scattering) and high-frequency noise (from thermal glare), which are challenging to separate in the spatial domain. Therefore, we designed the FFE-SA block, which integrates the FFE block into the shifting-window self-attention mechanism.²⁸ The Fourier transform projects features into the frequency domain, where the phase spectrum primarily encodes structural edges while the amplitude spectrum reflects energy distribution and is more susceptible to noise. The FFE block then applies frequency-domain attention to enhance phase-related structures and suppress noise-corrupted amplitude components. This “decouple-and-enhance” strategy is particularly effective for the complex, frequency-mixed degradations encountered in BF environments. This block improves the model’s ability to handle important features and noise in BF top IR images. The structure of the FFE-SA block is shown in Figure 5, while the detailed structure of the FFE block alone is illustrated in Figure 6.

We first project the input spatial domain feature $x$ to its frequency domain feature $F$ by the Fourier transformation,²⁹ as shown in equation (1).

F (u, v) = \sum_{h = 0}^{H - 1} \sum_{w = 0}^{W - 1} x (h, w) \exp (- 2 π j (u \frac{h}{H} + v \frac{w}{W})),

(1)

where

(h, w)

and

(u, v)

represent the coordinates in the spatial and frequency domain, respectively.

H

and

W

are the height and width of each feature map

x

j

is the imaginary unit.

e

is the base of the natural logarithm.

The amplitude $A$ and phase $P$ can be obtained according to the real and imaginary parts of $F$ . To enhance important features and suppress noise interference in the frequency domain, an attention map is generated. Specifically, as for the input feature $f \in {A, P}$ , the input undergoes a depthwise separable convolution (DSConv) process. Initially, depthwise convolution applies a $3 \times 3$ kernel to each input channel independently. Following this, the ReLU activation function is applied to introduce nonlinearity. Subsequently, pointwise convolution ( $1 \times 1$ convolutions) combines the outputs from the depthwise convolution step, adjusting the feature map depth by integrating information across channels. By employing DSConv, we significantly reduce the computational load, enabling faster processing and lower memory usage. The calculation method is shown in equation (2):

f * = C_{1}^{1} (λ (C_{1}^{3} (f))) .

(2)

After obtaining the $f *$ , the mean and maximum values of the $f *$ are calculated separately to capture both the global information and salient features of the image within a local area. By applying DSConv followed by Sigmoid activation functions, a normalised attention map is generated. The output of the Sigmoid function, which ranges from 0 to 1, controls the weight or importance of the information. A value close to 0 indicates that the feature is largely suppressed or ignored, whereas a value close to 1 signifies that the feature is fully weighted and considered. The resulting attention map $α$ is then multiplied by the input feature map to enhance important features. This multiplication process emphasises critical features while suppressing less relevant ones, leading to the final output $f^{'}$ . The calculation method is shown in equation (3).

\begin{aligned} α & = δ (C_{1}^{1} (λ (C_{1}^{3} (τ (f *), ψ (f *))))), \\ f^{'} & = α \otimes f . \end{aligned}

(3)

Finally, following the processing flow from $f$ to $f^{'}$ , the enhanced feature $f^{'}$ is transformed back into its original spatial domain to obtain $x_{f}$ , as formulated in equation (4):

x_{f} = F^{- 1} (f^{'}),

(4)

where the

F^{- 1} (\cdot)

is the inverse Fourier transformation operation.³⁰ Here, we define the calculation from equation (1) to equation (4) as FFE.

In the FFE-SA block, another core component is the shifting window self-attention module. Conventional window-based self-attention mechanisms partition the input feature maps into non-overlapping fixed windows and perform self-attention operations within each window. However, this approach may not adequately capture the contextual information across different windows. In contrast, the shifted window mechanism shifts the windows at certain layers, allowing adjacent windows to share information, thereby enhancing the connectivity between local regions. Due to the shifted windows crossing original window boundaries, the model can better model long-range dependencies at deeper levels without increasing the receptive field as in traditional CNNs.

After the process of FFE, we apply three independent linear transformations ( $1 \times 1$ convolutions) to map the enhanced features $x_{f}$ to query $Q_{f}$ , key $K_{f}$ , and value $V_{f}$ . Then, the local features are learned in the spatial domain using a shifting window self-attention module. The self-attention is computed within $8 \times 8$ non-overlapping windows as

S A (Q_{f}, K_{f}, V_{f}) = s f (\frac{Q_{f} ⊙ K_{f}^{T}}{\sqrt{d}} + B) ⊙ V_{f},

(5)

where

d

and

B

denote the dimensionality and position bias, respectively, and

K_{f}^{T}

represents the transpose of

K_{f}

Finally, the original feature is transferred by the residual connection as

f s = S A (Q_{f}, K_{f}, V_{f}) \oplus x,

(6)

where the feature

f s

is obtained by applying the shifting window self-attention mechanism to the input features (

Q_{f}, K_{f}, V_{f}

), resulting in enhanced features which are then combined with the original input

x

The computations from equation (1) to equation (6) encompass the full operations of the FFE-SA block. The FFE block enhances features in the frequency domain and facilitates interaction between frequency and spatial domains, significantly improving the model’s ability to handle complex features.

DWT block based on deformable convolution

Whereas the FFE-SA block excels at FFE, its reliance on the Fourier transform introduces a global attribute that limits its effectiveness in capturing locally important features. In contrast, the wavelet transform possesses excellent localisation characteristics, which can compensate for the shortcomings of the FFE-SA block. Moreover, the wavelet transform provides a multi-resolution representation, capturing image details at different scales. Furthermore, considering the changes in shape and size of the chute, a reference object inside the BF during its operation, introducing the DWT block is particularly suitable for processing IR images of the BF top.

The structure of the DWT is shown in Figure 7. For the input feature graph $f s$ , two deformable convolutions,³¹ ${DConv}_{3 \times 3}$ and ${DConv}_{7 \times 7}$ , are first applied to obtain different scale features, and the deformable convolution can adapt to the irregular shape features of the chute in the BF top images. The calculation method is shown in equation (7):

\begin{aligned} f s_{1} & = {DConv}_{3 \times 3} (f s), \\ f s_{2} & = {DConv}_{7 \times 7} (f s) . \end{aligned}

(7)

Here, $f s_{1}$ uses a $3 \times 3$ deformable convolutional kernel to capture more detailed local features, while $f s_{2}$ uses a $7 \times 7$ kernel to obtain more extensive global features.

Then, the small-scale feature map $f s_{1}$ is processed through the wavelet transform block (WTB).^32,33 The WTB uses the Haar wavelet basis function. By performing filtering operations in both horizontal and vertical directions, we obtain four distinct subbands. Among them, $f s_{L L}$ contains the low-frequency information of the image, $f s_{L H}$ contains the horizontal details of the image, $f s_{H L}$ contains the vertical details of the image, $f s_{H H}$ contains the diagonal details of the image. Thus, the feature $f s_{wavelet}$ is obtained by coupling them using the concatenation operation, as shown in equation (8):

Then, we use a feature modulation block to apply the small-scale features $f s_{wavelet}$ as modulation parameters to the large-scale features. Specifically, the feature modulation block first processes the wavelet-domain features $f s_{wavelet}$ through convolutional layers and an up-sampling operation to generate a spatial attention map. This map is then element-wise multiplied with the large-scale feature map $f s_{2}$ , enabling the high-frequency details (e.g. edges and textures) extracted by the wavelet transform to selectively enhance the corresponding regions in the coarse-scale feature map. This modulation mechanism allows the model to adaptively fuse multi-scale information: the wavelet-based high-frequency details guide the refinement of local structures, while the large-scale features provide contextual semantics. As a result, the model’s capability to restore fine details in BF top IR images is significantly improved. The calculation method is shown in equation (9):

f s_{2}^{^{'}} = f s_{2} \otimes S_{↑} (C_{1}^{1} (λ (C_{1}^{1} (f s_{wavelet})))) .

(9)

By adjusting the large-scale feature map $f s_{2}$ to $f s_{2}^{^{'}}$ , the ability of the model to deal with multi-scale features and locally important features is enhanced. Finally, the integrated feature map $f s_{fusion}$ is obtained by combining feature maps $f s_{1}$ , $f s_{wavelet}$ and $f s_{2}^{^{'}}$ , as shown in equation (10):

SDE block

High-frequency features are important for the restoration of hazy images. Moreover, in the context of BF top IR images, the edges and contours of targets are continuously changing, which necessitates enhancing the model’s ability to handle critical features. Therefore, an SDE block is proposed.

The structure of the SDE block is shown in Figure 8. The core component of the SDE block is a detail enhancement convolution (DEConv) that integrates five types of difference convolutions.³⁴ The difference convolutions combine the edge feature detection capabilities of classical edge detection operators with the powerful feature learning ability of convolutional networks, which can pay more attention to the gradient information of the image and improve the precision of edge detection. The DEConv consists of five different convolutions that are computed in parallel. Among these, the central difference convolution (CDC) is suitable for capturing local fine edges or textures, the angular difference convolution (ADC) is appropriate for handling diagonal edges, the rotational difference convolution (RDC) is effective for detecting edge contours with rotational symmetry, the horizontal difference convolution (HDC) is adept at detecting horizontal edges, and the vertical difference convolution (VDC) is designed for detecting vertical edges. The weights and biases of the DEConv are calculated as shown in equation (11):

\begin{aligned} w_{DEConv} & = w_{CDC} + w_{ADC} + w_{RDC} + w_{HDC} + w_{VDC}, \\ b_{DEConv} & = b_{CDC} + b_{ADC} + b_{RDC} + b_{HDC} + b_{VDC} . \end{aligned}

(11)

In the SDE block, the input feature $f s$ first undergoes feature extraction via DEConv layers, which include various types of difference convolution (CDC, ADC, RDC, HDC, and VDC). Following this, ReLU activation introduces non-linearity, and another DEConv layer further refines the features. Finally, the processed features are added to the original input feature $f s$ through a residual connection, yielding the enhanced output feature $f s^{^{'}}$ . This process effectively captures detail and edge information in images, boosting the model’s ability to handle key features. The calculation method is shown in equation (12):

f s^{^{'}} = f s \oplus DEConv (λ (DEConv (f s))) .

(12)

Finally, to enhance the model’s ability to handle complex features at multiple scales, we introduce feature interaction in both the wavelet and spatial domains, as shown in Figure 4. The input spatial feature map $x$ is processed by the FFE-SA block, generating the enhanced spatial feature $f s$ . The output $f s$ undergoes two independent processing steps: one through the DWT block, where it is processed in the wavelet domain to enhance feature information, resulting in $f s_{fusion}$ ; and the other through the SDE block, where it is processed in the spatial domain to enhance feature information, resulting in $f s^{^{'}}$ . The two sets of features are combined to achieve feature interaction and fusion with $f s$ , as shown in equation (13):

In summary, the FFE-SA block enhances the model’s ability to handle the overall structure and long-range dependencies in BF top IR images; the DWT module improves the model’s capability to process local and multi-scale features within the images; and the SDE module strengthens the model’s ability to handle important detail features. By integrating the enhanced features from the frequency domain, wavelet domain, and spatial domain, a richer and more comprehensive feature representation can be generated, thereby improving the model’s ability to handle complex coupled noise and its robustness.

Experimental methods

Dataset and metrics

Using the chute as a reference object, we synchronised IR video acquisition with its operational cycles via the BF top imaging system. The image acquisition module consists of a lens and a thermal imager. Its main parameters are shown in Table 2.

Table 2.

Specifications of the IR camera.

Parameter	Value
Image resolution	$384 \times 288$
Image frequency	50 Hz
Pixel size	$17 μ m$
Noise equivalent temperature difference (NETD)	$\leq 50 mK@25 \circ C$
Wavelength	8–14 $μ$ m
Lens focal length	3.3 mm
Field of view (horizontal $\times$ vertical)	$100 \circ \times 79 \circ$
Temperature measurement range	0 $\circ$ C to 1600 $\circ$ C

The chute completes a full rotation every 7.86 s (measured by laser sensors), and the camera’s 50 fps frame rate yields 393 frames per cycle. By aligning frames across consecutive cycles, images at identical chute positions were grouped into 393-frame sequences. During material charging (hazy conditions) and stable intervals (clear conditions), continuous video recordings were segmented into these groups. Each group pairs hazy images (captured during active material distribution) with clear counterparts (acquired in charge-free intervals). The video data used to construct the dataset were collected over an extended period, spanning multiple months and production days.

The dataset spans 21 chute angles ( $20 \circ$ – $40 \circ$ ), with 900 paired sequences per angle, totalling 18,900 pairs. To rigorously prevent data leakage, we re-partitioned the entire dataset based on temporal isolation. The split ensures that all data in the test set comes from a later operational period than the data in the training set, guaranteeing complete temporal separation. This division was performed at the level of independent charging cycles. These were divided into 15,120 training pairs and 3,780 testing pairs.

To evaluate the effectiveness of the IRReFormer algorithm, we quantitatively compared it with other algorithms using this dataset. We employed four complementary metrics to comprehensively assess the restoration quality from multiple perspectives: pixel-level fidelity, structural integrity, perceptual similarity, and local gradient preservation.

The peak signal-to-noise ratio (PSNR)³⁵ quantifies pixel-wise reconstruction accuracy by measuring the ratio between the maximum possible signal power and the power of the noise (error) between the restored and ground-truth images. It serves as a fundamental indicator of overall image fidelity.

The structural similarity index measure (SSIM)³⁵ evaluates the perceived structural similarity between two images by comparing their luminance, contrast, and structure. It is more aligned with human visual perception than PSNR and effectively assesses the preservation of textures and contours.

To provide a deeper assessment of perceptual quality, we also employ the learned perceptual image patch similarity (LPIPS)³⁶ and the gradient magnitude similarity deviation (GMSD).³⁷ LPIPS measures high-level perceptual consistency by computing the distance between deep feature representations extracted from a pre-trained neural network; lower LPIPS values indicate better perceptual realism. GMSD quantifies the distortion in local gradient structures by analysing the standard deviation of gradient magnitude similarity; a lower GMSD value signifies superior preservation of edge sharpness and local details.

Together, these four metrics offer a holistic and multi-level evaluation: PSNR focuses on low-level pixel accuracy, SSIM on mid-level structural integrity, LPIPS on high-level semantic and perceptual consistency, and GMSD on the fidelity of local gradient and edge information. This comprehensive set of indicators ensures that the effectiveness of our restoration algorithm is rigorously validated across all critical dimensions of image quality.

The restoration model was trained on a Huawei server equipped with two Intel 6132 14-core CPUs, 128 GB RAM, and two NVIDIA Tesla V100-SXM2-32GB GPUs. The specific configuration parameters for training are detailed in Table 3.

Table 3.

Settings of the restoration model.

Parameter	Value
Image_size	$256 \times 256 \times 3$
Batch_size	4
Epoch	100
Optimizer	Adam
Learning_rate	0.0001
Loss function	L1 + MS-SSIM (weights $0.7 : 0.3$ )

Experimental results

Table 4 presents the evaluation results of various IR image enhancement and dehazing algorithms on the BF top IR images dataset. The $↑$ represents a larger value, a higher quality. For IR image enhancement algorithms, the methods in the table include classic traditional enhancement algorithms as well as deep learning-based low-light enhancement and super-resolution algorithms; for dehazing algorithms, all methods listed are among the most advanced from the past two years.

Table 4.

Performance of different algorithms for the BF top IR image dataset.

	Methods	PSNR (dB) $↑$	SSIM $↑$	LPIPS $↓$	GMSD $↓$	T (ms) $↓$
IST1430005-struc0x.png IR enhancement\c:IMG	CLAHE-DGC¹⁷	13.42	0.334	0.584	0.531	1.46
	IE-GFLIN¹¹	12.57	0.362	0.572	0.528	4.85
	IE-WGF¹⁸	15.04	0.375	0.533	0.406	6.04
	PSRGAN¹³	14.78	0.438	0.496	0.478	32.37
	DASRGAN²⁰	18.49	0.627	0.407	0.430	40.61
	HWMNet¹⁹	22.32	0.677	0.341	0.268	19.84
	MMFFNet¹²	11.82	0.336	0.585	0.541	22.75
Dehazing	MB-TaylorFormer²³	16.45	0.506	0.482	0.391	38.71
	ITBdehaze²⁵	22.28	0.636	0.335	0.272	48.97
	DehazeFormer¹⁵	22.48	0.691	0.326	0.240	61.32
	DEA-Net¹⁶	20.71	0.584	0.374	0.279	33.26
	CrossDehaze²⁶	22.43	0.667	0.328	0.251	42.12
	SFSNiD²⁴	24.70	0.808	0.186	0.193	64.86
	IRReFormer (ours)	25.38	0.828	0.168	0.175	57.53

BF: blast furnace; IR: infrared; PSNR: peak signal-to-noise ratio; SSIM: structural similarity index measure; LPIPS: learned perceptual image patch similarity; GMSD: gradient magnitude similarity deviation; IRReFormer: infrared image restoration former. Bold values represent the optimal results in each category.

Quantitative evaluations confirm our algorithm’s superiority across multiple metrics, including PSNR, SSIM, LPIPS, and GMSD. IRReFormer achieves the best performance not only in PSNR (25.40 dB) and SSIM (0.831), but also in LPIPS (0.168) and GMSD (0.175). These results indicate that our method excels in both pixel-level fidelity, structural integrity, perceptual quality, and edge preservation.

Traditional IR enhancement methods (e.g. CLAHE-DGC,¹⁷ IE-GFLIN,¹¹ and IE-WGF¹⁸) rely on assumptions (e.g. uniform degradation) that are violated by the dynamic scattering and glare in BF environments, leading to distortion, over-enhancement, and artefacts. Deep learning-based super-resolution methods (e.g. PSRGAN¹³ and DASRGAN²⁰) amplify both details and the inherent noise, causing edge blurring and structural discontinuity. The low-light enhancer MMFFNet¹² lacks a mechanism to suppress mixed-frequency interference, resulting in overexposure and noise amplification. HWMNet¹⁹ improves wavelet-domain features but amplifies glare and fails to decouple noise.

Among dehazing algorithms, MB-TaylorFormer²³ prioritises computational efficiency but neglects noise suppression. ITBDehaze²⁵ and CrossDehaze²⁶ suffer from domain shift issues caused by visible-light pretraining, while DehazeFormer¹⁵ and DEA-Net¹⁶ lack IR-specific adaptations. Although SFSNiD²⁴ achieves relatively better performance by leveraging similarities between night fog and BF conditions, it still exhibits edge blurring.

Inference times are reported in Table 4. IRReFormer requires 57.53 ms per image, exceeding the 20 ms acquisition interval but feasible for real-time monitoring via skip-frame processing (e.g. processing every third frame). This enables continuous high-quality monitoring even during charging, overcoming the limitation of methods restricted to charge-free intervals.

Visual results (Figure 9) show that traditional and super-resolution methods exhibit distortion, blurring, or overexposure, while dehazing algorithms cause edge warping. In contrast, IRReFormer confirms its robustness for BF top IR restoration. Specifically, as shown in the zoomed-in patches, IRReFormer effectively restores sharp chute contours and internal structural textures, which are critical for accurate visual inspection and measurement in BF operations.

Ablation study

To validate the efficacy of the FFE, DWT, and SDE blocks in IRReFormer, ablation studies were conducted on the BF top IR images test set (Table 5). The baseline model (BASE: three-layer U-Net with self-attention only) achieved 23.27 dB PSNR, 0.764 SSIM, 0.210 LPIPS, and 0.203 GMSD. Incorporating individual modules improved performance and pairwise combinations of modules further enhanced metrics, while the complete model integrating all components achieved optimal performance at 25.40 dB PSNR, 0.831 SSIM, 0.168 LPIPS, and 0.175 GMSD.

Table 5.

Results of ablation study.

BASE	FFE	DWT	SDE	PSNR (dB) $↑$	SSIM $↑$	LPIPS $↓$	GMSD $↓$	$T$ (ms) $↓$
✓				23.28	0.762	0.210	0.203	48.69
✓	✓			24.09	0.790	0.188	0.191	51.43
✓		✓		24.07	0.792	0.186	0.187	52.36
✓			✓	24.02	0.788	0.191	0.189	50.70
✓	✓	✓		24.41	0.801	0.179	0.182	55.44
✓	✓		✓	24.42	0.803	0.183	0.185	53.87
✓		✓	✓	24.69	0.812	0.181	0.183	54.71
✓	✓	✓	✓	25.38	0.828	0.168	0.175	57.53

BASE: baseline model; FFE: frequency feature enhancement; DWT: discrete wavelet transform; SDE: spatial detail enhancement; PSNR: peak signal-to-noise ratio; SSIM: structural similarity index measure; LPIPS: learned perceptual image patch similarity; GMSD: gradient magnitude similarity deviation. Bold values represent the optimal results in each category.

These results quantitatively demonstrate each module’s critical contribution to reconstruction quality. The FFE-SA block operates in the frequency domain to decouple and enhance structural (phase) information while suppressing noise-corrupted amplitude components, addressing mixed-frequency degradations. The DWT block captures multi-scale local details through the wavelet transform and uses high-frequency details to modulate large-scale features, thereby enhancing edge and texture recovery. The SDE block employs differential convolutions to focus on spatial gradients, effectively sharpening edges and contours. Together, these modules provide complementary enhancements: FFE-SA for global frequency-aware restoration, DWT for multi-scale detail reconstruction, and SDE for local edge refinement, resulting in a synergistic improvement in both quantitative metrics and visual quality.

Engineering application

Take the BF chute inspection as an example, which is a critical task for ensuring stable production since the chute, a core component inside the BF, must be set at the correct angle. This task involves using the Res-LSTM model to extract features from these IR images and map them to the corresponding chute angles, thereby diagnosing whether the chute angle matches the preset angle. However, deep learning-based detection algorithms heavily rely on the quality of the BF top IR images.

To systematically evaluate the adverse impact of hazy BF top IR images on chute detection accuracy, we designed four experimental groups with distinct training–testing configurations:

Group I (clear-only): The model is trained exclusively on pristine clear images (100% of the training set) and tested on both clear and hazy images to establish baseline performance.

Group II (hazy-only): The model is trained solely on unprocessed hazy images (100% of the training set) and tested on both clear and hazy images to isolate degradation effects.

Group III (mixed hazy): The model is trained on a hybrid dataset (50% clear + 50% hazy images) mimicking real-world operational variability, with testing on both clear and hazy images.

Group IV (mixed restored): The model is trained on a combination of 50% clear images and 50% algorithmically restored images (restored from hazy images using the IRReFormer algorithm), and tested on both clear and restored images to assess recovery efficacy.

The experiments utilised a dataset of 21,600 clear-hazy-restoration triplets spanning chute angles from 20°to 40°. Hazy images were captured under dynamic material distribution conditions, while their corresponding clear counterparts were acquired during static, charge-free intervals. Restoration images were generated by processing hazy inputs with the IRReFormer algorithm to reconstruct visual fidelity. To ensure statistical rigour, the dataset was stratified into 17,280 training triplets (80%) and 4,320 testing triplets (20%), preserving a proportional representation of all three image types. All models underwent uniform training for 20 epochs (we observed that the validation loss plateaued around 15 epochs, indicating convergence, as shown in Figure 10; training for 20 epochs ensured adequate learning), with quantitative results detailed in Table 6.

Table 6.

Accuracy of different experimental groups.

Accuracy (%)	Group I	Group II	Group III	Group IV
Clear	99.79	5.00	99.86	99.17
Hazy (restoration)	0.83	71.18	54.13	99.65

Bold values represent the optimal results in each category.

As quantified in Table 6, Group I (trained exclusively on clear images) achieved 99.79% accuracy on clear test images but catastrophically failed on hazy images (0.83%). Group II (trained solely on hazy images) attained 71.18% hazy-test accuracy, revealing inherent information loss in degraded data. Group III (50% clear + 50% hazy training) maintained high clear-image accuracy (99.86%) yet suffered significant hazy-test degradation (54.13%), demonstrating that mixed training cannot resolve fundamental haze-induced feature corruption.

Crucially, Group IV (50% clear + 50% IRReFormer-restored training) achieved 99.65% accuracy on restored test images – surpassing Group III’s hazy-test performance by 45.52 percentage points and approaching clear-image benchmarks (99.17%). This restoration-driven accuracy recovery confirms that IRReFormer effectively mitigates operational disturbances (dust, steam, and lighting variations), significantly enhancing the performance of visual tasks such as detection in industrial settings such as BF.

Conclusion

We present IRReFormer, an IR restoration algorithm specifically designed for BF top imaging. The framework integrates three synergistic components: (1) the FFE-SA block optimises multi-frequency feature representation while prioritising critical structural details; (2) the DWT block enables hierarchical extraction of global–local multi-scale features; (3) the SDE block employs differential convolutions to sharpen edges and suppress artefacts. This cross-domain fusion mechanism ensures robust performance in complex industrial environments.

Experimental results demonstrate IRReFormer’s superiority, achieving a PSNR of 25.40 dB and SSIM of 0.831 – outperforming conventional IR enhancement and visible-light dehazing methods. In downstream chute angle detection tasks, restoration accuracy increased from 54.13% (hazy inputs) to 99.65%, confirming practical utility in real-world BF operations.

While developed for BF top images, the proposed IRReFormer has the potential to be adapted to other industrial IR imaging scenarios, such as kiln monitoring, thermal inspection of machinery, and other high-temperature processes that suffer from similar mixed degradations (e.g. smoke, dust, and glare). However, direct application may be limited by domain-specific characteristics, including varying sensor properties, temperature ranges, and scene layouts. Future work will explore domain adaptation and fine-tuning techniques to enhance the model’s generalisation across different industrial environments. Additionally, we will focus on unsupervised or semi-supervised learning paradigms to reduce the dependency on paired data and improve adaptability in data-scarce scenarios. Continuous learning capabilities will also be investigated to address evolving industrial conditions.

Footnotes

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the National Natural Science Foundation of China under Grant No. 52474364, Grant No. 52074185, and Grant No. 61902269; the Science and Technology Major Project of WuHan (2023020302020572); and the Priority Academic Program Development of Jiangsu Higher Education Institutions, China.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

ORCID iDs

Linling Xu

Helan Liang

Bingji Yan

References

Jiang

Yang

. IIANET: Information interactivity attention network with adversarial learning for infrared small object detection. Comput Vis Image Underst 2023; 237: 103839.

Zhu

Xiao

Lew

, et al. Lifelong visible–infrared person re-identification via replay samples domain-modality-mix reconstruction and cross-domain cognitive network. Comput Vis Image Underst 2025; 254: 104328.

Xiang

. Application of infrared thermal imaging system on furnace top in blast furnace. Plant Eng Consult 2023; 64–66.

Chen

L-F

B-Q

Gao

Q-G

, et al. Blast furnace distribution chute angle measurement frock. CN218673457U, 2023. Patent.

Wang

. A measuring device for the inclination angle of blast furnace chutes. CN215909851U, 2022. Patent.

B-Q

Wang

Y-S

. Method for measuring blanking angle of distribution chute of blast furnace by using laser scanner. CN115652006A, 2023. Patent.

Zhang

S-W

Chen

G-Q

ShiF

, et al. Blast furnace distribution chute tilting angle measuring device and method. CN116356095A, 2023. Patent.

. A novel method for measuring the swing angle of a blast furnace burden distribution chute. CN108893571A, 2018. Chinese patent.

Liang

Yan

, et al. Angle diagnosis of blast furnace chute through deep learning of temporal images. IEEE Trans Instrum Meas 2024; 73, 1–11.

10.

Zuiderveld

. Contrast limited adaptive histogram equalization. In: Graphics gems, 1994, pp. 474–485.

11.

Katircioglu

Çay

Cingiz

. Infrared image enhancement model based on gravitational force and lateral inhibition networks. Infrared Phys Technol 2019; 100: 15–27.

12.

Zhu

Chen

Wang

, et al. MMFF-NET: Multi-layer and multi-scale feature fusion network for low-light infrared image enhancement. Signal Image Video Process 2023; 18: 1089–1097.

13.

Huang

Jiang

Lan

, et al. Infrared image super-resolution via transfer learning and PSRGAN. IEEE Signal Process Lett 2021; 28: 982–986.

14.

Wang

Zhu

Huang

, et al. TMS-GAN: A twofold multi-scale generative adversarial network for single image dehazing. IEEE Trans Circ Syst Video Technol 2022; 32: 2760–2772.

15.

Song

Qian

, et al. Vision transformers for single image dehazing. IEEE Trans Image Process 2023; 32: 1927–1941.

16.

Chen

. DEA-Net: Single image dehazing based on detail-enhanced convolution and content-guided attention. IEEE Trans Image Process 2024; 33: 1002–1015.

17.

Chang

Jung

, et al. Automatic contrast-limited adaptive histogram equalization with dual gamma correction. IEEE Access 2018; 6: 11782–11792.

18.

Luo

Zhang

. Infrared image enhancement algorithm based on weighted guided filtering. In: 2021 IEEE 2nd international conference on information technology, big data and artificial intelligence, 2021, Vol. 2, pp. 332–336. DOI: 10.1109/ICIBA52610.2021.9688030.

19.

Fan

Liu

. Half wavelet attention on M-Net+ for low-light image enhancement. In: 2022 IEEE international conference on image processing, 2022, pp. 3878–3882. DOI: 10.1109/ICIP46576.2022.9897503.

20.

Huang

Miyazaki

Liu

, et al. Target-oriented domain adaptation for infrared image super-resolution. ArXiv 2023; abs/2311.08816.

21.

Nishita

Miyawaki

Nakamae

. A shading model for atmospheric scattering considering luminous intensity distribution of light sources. In: Proceedings of the 14th annual conference on computer graphics and interactive techniques 1987, pp. 303–310. DOI: 10.1145/37402.37437.

22.

Chen

Huang

, et al. Enhanced pix2pix dehazing network. In: 2019 IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 8152–8160. DOI: 10.1109/CVPR.2019.00835.

23.

Qiu

Zhang

Wang

, et al. MB-TaylorFormer: Multi-branch efficient transformer expanded by Taylor formula for image dehazing. In: 2023 IEEE/CVF international conference on computer vision, 2023, pp. 12756–12767. DOI: 10.1109/ICCV51070.2023.01176.

24.

Cong

Gui

Zhang

, et al. A semi-supervised nighttime dehazing baseline with spatial-frequency aware and realistic brightness constraint. In: 2024 IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 2631–2640. DOI: 10.1109/CVPR52733.2024.00254.

25.

Liu

, et al. A data-centric solution to nonhomogeneous dehazing via vision transformer. In: 2023 IEEE/CVF conference on computer vision and pattern recognition workshops, 2023, pp. 1406–1415. DOI: 10.1109/CVPRW59228.2023.00145.

26.

Shi

Weng

Lin

, et al. Crossdehaze: Scaling up image dehazing with cross-data vision alignment and augmentation. ArXiv 2024; abs/2407.14823. DOI: 10.48550/arXiv.2407.14823.

27.

Ronneberger

Fischer

Brox

. U-Net: Convolutional networks for biomedical image segmentation. ArXiv 2015; abs/1505.04597. DOI: 10.48550/arXiv.1505.04597.

28.

Liu

Lin

Cao

, et al. Swin transformer: Hierarchical vision transformer using shifted windows. In: 2021 IEEE/CVF international conference on computer vision, 2021, pp. 9992–10002. DOI: 10.1109/ICCV48922.2021.00986.

29.

Guo

Zhou

, et al. Exploring Fourier prior for single image rain removal. In: International joint conference on artificial intelligence, 2022, pp. 935–941. DOI: https://doi.org/10.24963/ijcai.2022/131.

30.

Zhou

Huang

Guo

, et al. Fourmer: An efficient global modeling paradigm for image restoration. In: International conference on machine learning, 2023, Vol. 202, pp. 42589–42601.

31.

Xiong

Chen

, et al. Efficient deformable convnets: Rethinking dynamic and sparse operator for vision applications. In: 2024 IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 5652–5661. DOI: 10.1109/CVPR52733.2024.00540.

32.

Jiang

Luo

Fan

, et al. Low-light image enhancement with wavelet-based diffusion models. ACM Trans Graph (ToG) 2023; 42: Article No. 238.

33.

Huang

Miyazaki

Liu

, et al. Irsrmamba: Infrared image super-resolution via mamba-based wavelet transform feature modulation model. ArXiv 2024; abs/2405.09873. DOI: 10.48550/arXiv.2405.09873.

34.

Liu

, et al. Pixel difference networks for efficient edge detection. 2021 IEEE/CVF international conference on computer vision, 2021, pp. 5097–5107. DOI: 10.1109/ICCV48922.2021.00507.

35.

Horé

Ziou

. Image quality metrics: PSNR vs. SSIM. In: 2010 20th international conference on pattern recognition, 2010, pp. 2366–2369. DOI: 10.1109/ICPR.2010.579.

36.

Zhang

Isola

Efros

, et al. The unreasonable effectiveness of deep features as a perceptual metric. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, 2018, pp. 586–595.

37.

Xue

Zhang

Mou

, et al. Gradient magnitude similarity deviation: A highly efficient perceptual image quality index. IEEE Trans Image Process 2014; 23: 684–695.