An automated steel plates fault diagnosis system using adaptive faster region convolutional neural network

Abstract

In today’s world, Steel plates play essential materials for various industries like the national defense industry, chemical industry, automobile industry, machinery manufacturing, etc. However, some defects may occur in a few plates during the manufacture of stainless-steel plates which directly impact the quality of the stainless-steel plate. If the faulted plate detection can be done manually, then it leads to errors and a time-consuming process. Hence, a computerized automated system is necessary to detect the abnormalities. In this paper, a novel Adaptive Faster Region Convolutional Neural Networks (AFRCNN) scheme has been proposed for automatic fault detection of stainless-steel plates. The proposed AFRCNN scheme comprises three phases: identification, detection, and recognition. Primarily, the damaged plates are identified using Region Proposal Network and Fully Convolutional Neural Network functioning as a combined process under AFRCNN. In the next phase, the number corresponding to the particular plate is recognized through the standard Automated Plate Number Recognition approach with the support of the character recognition technique. The simulation results manifest that the proposed AFRCNN scheme obtains a superior classification accuracy of 99.36%, specificity of 99.24%, and F1-score of 98.18% as compared with the existing state-of-the-art schemes.

Keywords

Fault detection stainless steel plates convolutional neural network classification region proposal network

1 Introduction

In the case of manufacturing a material or products, the quality of the refined products should be monitored and its quality at the surface should be validated before its usage. In recent years, if such abnormalities are found, the industries will reject such types of raw materials that are used in the production process [1]. Sometimes, the abnormalities are due to the poor quality of the raw material utilized for manufacturing. If this problem is not at all rectified, then it will cause crucial damage in the entire process. Hence, the detection of defects should be timed earlier in order to reduce the loss in the production cost [2].

In general, there will be some imperfections at the surface of the metals. If it is prolonged, then its effects will be more crucial. Therefore, a significant concern is needed to identify the abnormalities of small dimensions at the initial stage [3]. Even though the production is performed at a high speed, there should not be such problems in the output. Some of the small dimension abnormalities are cracks, scratches, holes, pits, and bumps. The effective quality control methods are computed to achieve an excellent surface evaluation. Furthermore, the testing can be carried out at different environmental conditions like temperature the results will be varied. Thus, fault identification in the stainless-steel plates is a major challenging task [4].

A huge number of stainless-steel plate defects occur for different products. Their distinctiveness differs from industry to industry and from worker to worker. Besides, these faults vary arbitrarily owing to the distinctions in the product production progression [5]. For instance, in the production procedure of hot-rolled strips, the key faults such as hole, craze, rolled in scale, and oil spot. Traditionally, the recognition of stainless-steel plate defects mostly implements the technique of manual diagnosis. This technique is easily influenced by the slanted factors of the testing workers. In addition, the detection accuracy of the investigation results is poorer and consumes more time during real-time inspection. Henceforth, it is necessary to employ the automated fault identification of stainless-steel plates on the production line [6].

In recent years, computer-aided stainless steel plate fault inspection mechanism has received widespread interest as a non-vicious and fully automated outcome to facilitate or replace manual inspectors [7 –9]. The most frequently utilized technique is to initially construct the high-level demonstration of faults through several feature learning approaches. It employs any one of the filters like Gabor filters, contrast filters, wavelet filters, etc. to eradicate the noise in the input. Subsequently, the faults are categorized with the aid of classifier models such as thresholding methods, deep learning, and neural networks. Nonetheless, the thresholding methods remain an unsolved crisis to design the finest influential features to acquire high-accuracy recognition of stainless-steel plate faults.

In the last few decades, more researchers have investigated computer-aided diagnosis based on deep learning due to the attractiveness of deep neural networks in fault classification [10]. It has the ability to learn attributes that have a high level of generalization and strong characteristics. Further, deep neural networks can acquire more precise fault classification than early techniques. It builds a depiction of all input data by executing a large number of learning layers and has better abstraction. Owing to the evolution of deep neural networks, the effectiveness and accuracy of computer-aided processes have been extensively improved. This technique can automatically learn the features from bottom level to top-level and complete the process of fault identification accurately [11].

The deep neural networks approaches have the ability to detect stainless steel plate faults and replace the conventional and machine learning approaches [12]. Nevertheless, an automatic fault identification process has major challenges owing to the complexity of faults: (a) Low contrast: The continuation of dust and the varying light intensity cause low contrast in the product production. (b) Intra-class difference: The contour of the faults is always uneven in industrial production. (c) Inter-class similarity: Because of the improbability of the production task, some types of faults have tiny differences.

Based on the aforesaid analysis, the development of the diagnosis and classification approaches has always been a difficult process in stainless steel plate fault diagnosis systems. In addition, the recent deep neural networks-based fault diagnosis system has the constraints of lesser classification accuracy and there is still a bunch of scope for accuracy enhancement [13].

In this work, a novel AFRCNN scheme based on deep neural networks has been introduced for the efficient classification of surface faults in stainless steel plates. This methodology uses different algorithms like Region Proposal Network (RPN) and Fully Convolutional Neural Network (FCNN) to identify the fault that occurs during the manufacturing of stainless-steel plates. The main rationale behind this utilization of RPN is to learn appropriate features from the input plates since the license plates have a higher aspect ratio and different sizes. It also enhances the accuracy of plate identification in challenging circumstances that have non-uniform radiance and lesser resolution.

The vital objective of the proposed AFRCNN scheme is to exactly forecast fault types, determining useful information for detecting and subsequently rectifying contributory factors. Moreover, it implements both defect identification and number plate recognition model which significantly enlarges the recall and decreases the execution time of the proposed scheme.

The remaining part of this paper is formulated as follows. Section 2 summarizes several existing schemes. Section 3 describes the proposed system with the necessary stages in detail. The tested results of the detection process and the discussions are presented in Section 4. Conclusion and future directions are enumerated in Section 5.

2 Related work

The following section offers various existing schemes related to computer-aided detection and classification of stainless-steel plate faults. Table 1 depicts the comparison of various detection schemes. In [14], deep learning-based steel surface defect detection has been experimented with using a multilayer perceptron network. Primarily, a Differential Evolution Algorithm (DEA) is incorporated in the anticipated model to enlarge the classification accuracy of fault detection of the stainless-steel plate. Next, a new channel attention model has been implemented to eradicate information loss. The convolutional kernels extract the attributes and then the adaptive fusion part is employed to fusion the shallow and kernel features. Different category features would be achieved by combining low-level features. However, the fusion method causes higher computational complexity and a poor convergence rate.

Table 1
Comparison of various detection schemes

Models &Year Classifier Detection Accuracy Plate number recognition Merits Demerits

[14] &2021 DEA Medium Low Extracts different influential features Higher computational complexity and poor convergence rate.

[15] &2021 SVM Very High Medium Estimates various defects Lesser global optimal results

[16] &2021 RCNN High Very Low Higher accuracy and better precision Very complex to train a model

[17] &2020 PGA-Net Very Low Medium Better detection of various shapes of characters Lesser specificity and F1 score

[18] &2020 YOLOv3 High Very Low Identify different defects Lagged to recognize the serial number of the defective plate

[19] &2019 CNN Medium Low Requires lesser number of training inputs Failed to extract the tiny influential attributes

[20] &2020 CNN High Medium Provide accurate location of the defect Lagged to retrieved boundary portions of the steel plates

[21] &2019 CAE-SGAN Low Very Low Less misclassification errors Higher computational complexity

[24] &2018 TIA Very Low Medium Extracts various higher-level features Lesser specificity and Precision

[22] &2015 PECP Very Low Low Less complexity High misclassification errors

Models &Year	Classifier	Detection Accuracy	Plate number recognition	Merits	Demerits
[14] &2021	DEA	Medium	Low	Extracts different influential features	Higher computational complexity and poor convergence rate.
[15] &2021	SVM	Very High	Medium	Estimates various defects	Lesser global optimal results
[16] &2021	RCNN	High	Very Low	Higher accuracy and better precision	Very complex to train a model
[17] &2020	PGA-Net	Very Low	Medium	Better detection of various shapes of characters	Lesser specificity and F1 score
[18] &2020	YOLOv3	High	Very Low	Identify different defects	Lagged to recognize the serial number of the defective plate
[19] &2019	CNN	Medium	Low	Requires lesser number of training inputs	Failed to extract the tiny influential attributes
[20] &2020	CNN	High	Medium	Provide accurate location of the defect	Lagged to retrieved boundary portions of the steel plates
[21] &2019	CAE-SGAN	Low	Very Low	Less misclassification errors	Higher computational complexity
[24] &2018	TIA	Very Low	Medium	Extracts various higher-level features	Lesser specificity and Precision
[22] &2015	PECP	Very Low	Low	Less complexity	High misclassification errors

In order to detect the cracks present in a 304 SS plate, a weak magnetic technology is implemented using a multiclass Support Vector Machine (SVM) classifier [15]. The suggested model first utilizes the SVM model to obtain the position and contour of the fault and then employs a spreading mechanism to categorize the detected faults. The investigation illustrates that this technique is successful which correctly estimating the defect length, width, and depth. At the same time, the SVM scheme failed to obtain the global optimal results during the evaluation phase.

In order to preserve quality assurance during steel plates manufacturing, numerous deep learning schemes are exploited for the identification of faults. According to Wang et al. [16], the combined technique using ResNet50 and enhanced faster Region Convolutional Neural Networks (RCNN) has been established to improve the detection and classification of surface defects in steel plates. By implementing the deformable revolution network, the different shapes of faults are detected in the suggested scheme. This scheme shows higher accuracy and better precision. Nevertheless, it is very complex to train a steady and precise model based on RCNN and deformable revolution network.

In [17], a pyramid feature fusion and global context attention network (PGA-Net) has been established for pixel-wise recognition of surface fault. Initially, the multi-scale features are learned by executing Convolutional Neural Networks (CNN) model. Thereafter, the pyramid feature fusion model is utilized to combine these attained attributes into five motions via some effective intense skip relations. Finally, the global context attention model is invoked for mapping purposes that permit vital information broadcast from low-level feature to high-level feature. On the other hand, the PGA-Net scheme lagged to retrieved boundary portions of the steel plates that leads to acquiring lesser specificity and F1 score in the classification stage.

To acquire fast and effectual classification of the faults, a YOLOv3 network has been presented in the present research [18]. It exploits YOLOv3 as a fundamental network to execute priority classification and then swapped the two residual models by means of two dense models. This replaced network can obtain the result of the multi-layer attributes by the dense fusion block where it continuously performs the reuse and fusion process. The six types of faults were identified by the YOLOv3 scheme. But, the continuous fusion process creates more complexity in the network.

Fu et al. [19] have demonstrated a CNN-based model for rapid and accurate classification of the steel surface. The underlying features are trained and classified by combining the multiple receptive fields. It only necessitates a lesser number of defect-specific training inputs to obtain maximum classification accuracy. Moreover, it employed a diversity-improved testing dataset for the detection process that consist of non-uniform illumination, noise, and signal blur. Nonetheless, the CNN model does not concentrate to extract the tiny influential attributes of the steel plates.

Yu He et al. [20] have highlighted a steel plate defect inspection technique using deep learning. In this approach, a base features line CNN is employed to learn the significant attributes. Especially, multi-level features are combined with multiple hierarchical features which provide the details about the location of the defect. This approach produces high misclassification errors due to the involvement of maximal levels of layers in the CNN scheme. Besides, it failed to recognize the serial number of the defective plate.

Recently, a few deep learning algorithms are used in the fault detection of plates include Autoencoder, CNN, and generative adversarial networks. According to Di et al. [21] a new Convolutional Autoencoder and Semi-supervised Generative Adversarial Networks (CAE-SGAN) has been proposed to detect the faults in the stainless-steel plate. Once the CAE is trained, the features are forecasted by using the encoder network. After that, the SGAN is established to enlarge the generalization capability of the classifier. Owing to the high complexity of the CAE-SGAN scheme, it failed to learn tiny shaped attributes from the fault dataset.

Arjun et al. [22] have presented a design of Pulse Eddy Current Probe (PECP) for the detection of subsurface defects in AISI type 316 stainless steel plates. The probe configuration is enhanced using optimization technique and comparisons are made with the different types of probes such as ferrite cored send receive probe, outer shield ferrite cored send receive probe, and ferrite cored abs probe.

Ji et al. [23] have implemented a non-contact and non-destructive detection system to identify the delaminating defects. A finite element model is developed in stainless steel or carbon steel plates and the influence of ultrasonic waves and laser ultrasonic waves are used to recognize the delaminating defects.

Ali et al. [24] have proposed a classification technique in the identification of faults in steel plates using data mining. The detected portions are analyzed using a Thermal Intensity Analysis (TIA) procedure. Here, the models include the C5.0 decision tree, Ensemble model, Baseyan network, and Multi-perception Neural Network are applied. Among these, the C5.0 decision tree has secured the highest accuracy of 95.36% for training and 95.66% for the testing dataset. However, this technique lagged to offer higher specificity, precision, and recall because of the establishment of inappropriate feature learning strategies in the anticipated scheme.

From the literature discussed above, many traditional and machine learning-based schemes are applied and each scheme has its own strength and weakness. Some of the problems are identified in the existing schemes are: The methodologies are normally sensitive to noise and scales are easily affected. Moreover, in many cases, the algorithms find it difficult to achieve the accuracy of the automatic defect detection system. In some approaches, the features should be manually developed; therefore, the scope of the application is very less. Deep learning-based classification schemes can only classify images but fail to locate the spots and recognize the serial number of the defective plate. Training a model and reinforcement learning is very difficult that should be stable and accurate.

By considering the above problems, this paper proposes a new scheme to realize the automatic detection, location, and serial number of steel plate surface defects. Thus, three objectives are fulfilled by a single system. This method further improves the stability and classification accuracy of the fault detection system.

The several novel contributions of the current research are given below.

A new AFRCNN scheme has been proposed to identify the multi-scale surface faults in stainless steel plates. The RPN performs the extraction of candidate regions in the proposed scheme. Better detection of various shapes of characters is achieved with improved accuracy and robustness.

An effective FCNN algorithm has been integrated with the proposed scheme where the extracted region proposals contained in the candidate regions are fed to the FCNN to recognize the character of the stainless-steel plate. These combined features will offer accurate and fully automated results to guarantee the fault-free products of the stainless-steel plate.

The number corresponding to the particular plate is recognized using the Automated Plate Number Recognition (ALPR) technique with the support of the character recognition technique. Character recognition is applied after segmentation to identify the alphabets and numerical. These activities are successfully performed by a single automated system which makes the proposed scheme superior to the other existing schemes.

3 Proposed AFRCNN methodology

The following section offers the detailed process of stainless-steel plate fault detection system. The proposed scheme can be carried out in two stages: (1) Tracking the defective plate and (2) Recognition of the defective plate number. The overall block diagram of the proposed scheme is demonstrated in Fig. 1.

3.1 Defective plate tracking

Fig. 1

Sequential diagram of the proposed system.

According to the proposed system, the detection of faulted plates and recognition of plate numbers are the two key objectives. It is slightly difficult to attain accuracy in recognizing defects in stainless steel plates captured by the IR system. A deep learning-based technique called the AFRCNN scheme is implemented to perform the defective plate tracking function. This scheme is preferred to design the RPN network due to its characteristics like speed and accuracy. The purpose of the AFRCNN model is to generate the proposed regions with the assist of CNN.

The defective plate tracking model is composed of two modules: (i) RPN (ii) FCNN. Here, RPN focused on the Region of Interest (ROI) whereas the FCNN detector will classify the proposed regions. Initially, a network is designed for RPN and later an algorithm can be developed for training each module with the shared features. Both modules will work as a single combined network for fault detection. Figure 2 depicts the functional diagram of the proposed AFRCNN scheme. The two popular modules (RPN and FCNN) are discussed in the following subsections.

Fig. 2

Functional diagram of proposed AFRCNN scheme.

3.2 Proposed RPN model

RPN can predict both object scores and object bound simultaneously at each position of an image. It is trained throughout for achieving better quality region proposals. Further, it can efficiently bring up the region proposals with a broad range of aspect ratios and scales using the anchor boxes. The main rationale behind this utilization of RPN is to learn appropriate features from the input plates since the license plates have a higher aspect ratio and different sizes. It also enhances the accuracy of plate identification in challenging circumstances that have non-uniform radiance and lesser resolution.

RPN learns the influential features from the input plate datasets by applying proper convolutional layers in the identification phase. The proposed feature map model will extract the larger level attributes and thus is more useful for identifying the number plate and forecasting its bounding box. Empirically feature maps from various layers are analytically known to have dissimilar field dimensions. Furthermore, the proposed technique utilizes the feature maps from the inferior layers that can enhance the segmentation eminence because the inferior layers encapsulate more influential information of the input plates. It extracts the fine-grained and spatial features from the input dataset.

The working of the RPN is as follows: Primarily, the RPN takes an input image and generate a set of the rectangular object as output proposals. A spatial window of size n × n slides from the origin in the feature map which is taken from the previously shared convolutional layer. An individual sliding window is then mapped to a feature map with low dimension, which is further fed into two sibling full connected layers: regression layer a bounding-box L_R and classification layer bounding-box L_C. The working of the RPN is illustrated in Fig. 3.

Fig. 3

Working of proposed RPN model.

In this work, the k number of stainless-steel plate images in a frame is predicted simultaneously from each sliding-window location as shown in Fig. 3. In RPN, the k number of the proposal is referred to as anchors. Three sliding windows with the aspect ratios of 1 : 1, 1 : 2 and 2 : 1 yields a total of k = 7 anchors. The L_R layer has 4k outputs and the L_C layer has 2k scores. The total size of the convolutional feature map is (W×H)×k anchors.

For sharing the features, as well as addressing the scales, the design of multi-scale anchors is more essential. Hence, a novel approach is introduced to address the multiple scales as depicted in Fig. 4. It is clear that a multi-scale anchor is framed with a pyramid of anchors. This approach regresses and classifies the bounding boxes with respect to the anchor boxes of multi-scales and aspect ratios. It relies only on the feature maps and images of a single scale and a filter of a single size is utilized. The scheme for addressing multiple scale and sizes is enumerated in Table 2.

Fig. 4

Sample output of detecting the faulted plate.

Table 2

Different settings for Anchor scales and Aspect ratio

Setting types	Anchor scale	Aspect ratios
1-scale and 1 ratio	[128², 256²]	{1 : 1, 1 : 1}
1 scale and 3 ratio	[128², 256²]	{2 : 1, 1 : 1, 1 : 2} {2 : 1, 1 : 1, 1 : 2}
3 scale and 1 ratio	[128², 256², 512²]	{1 : 1}
3 scale and 3 ratio	[128², 256², 512²]	{2 : 1, 1 : 1, 1 : 2}

In order to check whether it’s a new stainless-steel plate or not it is necessary to assign each anchor with a binary class label. In general, each anchor is allotted such that, if the highest IoU overlap ratio is greater than 0.7 it is allotted with a positive label along with a ground-truth box. Simultaneously, a single ground-truth box is also assigned with a positive label to multiple anchors. Then, the first condition is applied in such cases where it fails to find the positive sample in the second condition.

Likewise, in all ground-truth boxes, the anchor box has an IoU ratio lesser than 0.3, it is assigned as a negative label. The loss function for an image is given by:

$\begin{matrix} L ({P_{i}}, {t_{i}}) = & \frac{1}{N_{cls}} \sum_{i} L_{cls} (p_{i}, p_{i}^{*}) \\ + λ \frac{1}{N_{reg}} \sum_{i} p_{i}^{*} L_{reg} (t_{i}, t_{i}^{*}) \end{matrix}$ (1) where, i represents the anchor index, p_i denotes the probability predicted by anchor i. $p_{i}^{*}$ implies the ground-truth label (1 for positive & 0 for the negative) anchor, t_i indicates the vector representing the coordinates of the predicted bounding box, $t_{i}^{*}$ symbolizes the coordinates of the ground-truth box with respect to positive anchor, L_cls is the log loss parameter over two classes for classification, L_reg represents the regression loss, $p_{i}^{*} L_{reg}$ intends the regression loss initiated exclusively for positive anchors.

The classification loss can be normalized by N_cls and the regression loss is normalized by the number of anchor locations N_reg. The regression loss is also weighted by a balancing parameter λ. For bounding-box regression, the 4 coordinates are calculated from the expressions (2)–(9). $t_{x} = (u - u_{a}) / w_{a}$ (2) $t_{y} = (v - v_{a}) / h_{a}$ (3) $t_{w} = \log (\frac{w}{w_{a}})$ (4) $t_{h} = \log (\frac{h}{h_{a}})$ (5) $t_{x}^{*} = (u^{*} - u_{a}) / w_{a}$ (6) $t_{y}^{*} = (v^{*} - v_{a}) / h_{a}$ (7) $t_{w}^{*} = \log (\frac{w^{*}}{w_{a}})$ (8) $t_{h}^{*} = \log (\frac{h^{*}}{h_{a}})$ (9) where, u, v, w, and h indicate the coordinates of the anchor such as centre, width, and height, etc. Likewise, u, u_a, and u^* intend the variables for the anchor box, ground-truth box, and predicted box respectively.

The RPN network is trained using the common stochastic gradient descent algorithm and backpropagation algorithm. To train the network, an image-centric sampling strategy is followed. While optimizing the loss function of all anchors, about 256 anchors per image are sampled randomly. The ratio with respect to positive and negative anchors is 1 : 1. In case, if the positive samples in the image are less than 128 the mini-batch will coincide with the negative anchors.

Algorithm 1: Training algorithm of the proposed AFRCNN
Begin
Step 1: Train the RPN model
Step 2: Initialize and tune the model of Image Net to achieve the region task.
Step3: The network is trained using FCNN through the generated proposals.
Step 4: Initialize the detection network by the Image Net pre-trained model.
Step 5: Using the detection network, start the RPN training.
Step 6: Keep the Shared convolutional layers as fixed and tune the layers unique to RPN using FCNN.
Step 7: Two networks will equally share the convolutional layer and a combined network is formed.
End

To obtain the zero-mean Gaussian distribution, the drawing weights are tuned randomly from 0.01. During Image Net classification all other layers are initialized by a pre-training model. Thus, tuning is needed for all layers of the ResNet to conserve the memory. The learning rate parameter will be selected such that, for 2k mini-batches it is 0.001 and for the next 0.5k mini-batches, the learning rate is 0.0001 on the public dataset. Moreover, the momentum rate and weight decay are fixed as 0.85 and 0.0006 respectively. After training the network for region proposal generation, FCNN is adopted to utilize these proposals for detection. In this paper, a training algorithm is developed to learn the shared features. The unified learning network constitutes RPN and FCNN with the shared convolutional layers. The training algorithm of the proposed AFRCNN scheme is shown in Algorithm 1.

3.3 Proposed character recognition using APNR

This section describes the proposed approach for Plate Number (PN) recognition and the stages of APNR are divided into two subsections: (a) Faulted Plate (FP) detection and (b) PN recognition. Figure 5 illustrates the pipeline of the proposed APNR approach. In each stage of the APNR approach, different CNN models are used. In this approach, the AFRCNN model is utilized for PN detection and character recognition. At each stage, the parameters are fine-tuned in order to perform the task without any error.

Fig. 5

Pipeline of the proposed APNR approach.

Stage 1: The FP detection can be carried out in the initial stage. In the different applications, the stainless-steel plates appear in all shapes and sizes, probably of circular or oval in shape. In some plates, the plate number is engraved by paints instead of having an impression. By considering all these factors, the appropriate way is to label a circular bounding box for all plates. Hence, the proposed scheme will be utilized in both ResNet-50 and VGG16 models to separately train this network.

Stage 2: Character segmentation will be performed in this stage which is more vital in vehicle recognition. Since the characters in the faulted plate are of non-uniform shape, font, even padding, and size. In this application, the standard method such as open APNR cannot be implemented. So, the plates are labelled such that the network is trained in order to identify the character locations as enumerated in Fig. 6.

Fig. 6

Character Recognition using APNR approach.

While identifying the faulted plate, the PN of the faulted plate is to be recognized. Henceforth, a character segmentation approach is developed in the proposed method. The characters (02 and 04) that are located in Fig. 6 are correctly identified with the aid of the character segmentation approach. It comprises three significant stages: Initially, annotations are developed to obtain the region of interest. Later, AFRCNN is employed to compute the deep features where this process is fulfilled with the support of DenseNet-41. In general, a DenseNet is a category of CNN that uses the dense links between layers by introducing dense blocks. The key advantage of Densenet-41 is that fewer parameters are necessitated when compared to the traditional DenseNet model. Moreover, the DenseNet-41 contains only 24 channels on the primary layer, instead of 64, and the size of the kernel is 3×3 instead of 7×7. This paves a way to attain lesser computational complexity. Finally, the classification layer is exploited to localize and classify the PN.

Stage 3: Character recognition has been addressed in this stage. A standard CNN is applied in order to recognize the letter/number from a cropped image. The cropped letters from plates are occupied to train the characters. Initially, around 1000 characters are extracted from various plates for training the model. Again, this model is utilized to extract 200 more characters for further training. In total, around 6000 characters are subjected for training with 100–200 characters per class. Finally, the recursive training will be executed to achieve superior results.

4 Simulation results and discussions

The performance of the proposed technique has been implemented using NVIDIA GTX TITAN GPU (with 12 G memory) platform on Ubuntu 16.04 Linux. The simulation settings are exposed in Table 3. The test outputs of the proposed AFRCNN scheme can be carried out based on the workflow. Accordingly, the results at different stages like FP detection and faulted plate detection are obtained.

Table 3
Simulation setup

Parameters Values

Maximum epoch 10

Mini-batch size 1

Initial learning rate 0.001

Positive overlap ranges [0–0.3]

Negative overlap ranges [0.6–1]

Anchor box pyramid scale factor 1.2

Optimizer Momentum_optimizer

Parameters	Values
Maximum epoch	10
Mini-batch size	1
Initial learning rate	0.001
Positive overlap ranges	[0–0.3]
Negative overlap ranges	[0.6–1]
Anchor box pyramid scale factor	1.2
Optimizer	Momentum_optimizer

The performance measures are estimated at each level which is presented in the following sub-sections. The classification results are evaluated in terms of Accuracy, Specificity, Precision, Recall, F1-score, and Area Under the Receiver Operating Characteristic (AUROC) curve. These measures are evaluated using the standard expressions [25 –28]. $Accuracy = \frac{(TP + TN)}{(TP + TN + FP + FN)}$ (10) $Specificity = \frac{FN}{(FN + FP)}$ (11) $Precision = \frac{TP}{(TP + FP)}$ (12) $Recall = \frac{TP}{(TP + FN)}$ (13) $F 1 score = 2 \times \frac{Precision \times Recall}{Precision + Recall}$ (14)

$\begin{matrix} AUROC (x^{+}, x^{-}) \\ = \frac{1}{c^{+} c^{-}} \sum_{m = 1}^{c} + \sum_{n = 1}^{c^{- 1}} 1_{[p (x_{m}^{+}) > p (x_{n}^{-})]} \end{matrix}$ (15) where, $1_{[p (x_{m}^{+}) > p (x_{n}^{-})]}$ is related to 1 when the factors $1_{[p (x_{m}^{+}) > p (x_{n}^{-})]}$ , for all m = 1, 2, … , + ; for all n = 1, 2, …, c^-1 and ^′0′ otherwise. The finest scheme is attained when AUROC is closer to or equal to 1.

To perform an analysis, the obtained results of the proposed AFRCNN scheme are compared with the recent related schemes like CNN [19], CAE-SGAN [21], YOLOv3 [18], PGA-Net [17], RCNN [16], SVM [15], and DEA [14].

4.1 Faulted stainless steel plate detection and classification results

The Severstal: Steel defect detection dataset can be considered for the analysis of stainless-steel plate detection [29]. The dataset consists of 12,568 grayscale images of steel sheets and it does not have any annotations. Therefore, the annotations are labelled manually for the stainless-steel plate bounding box on each image of the dataset. In this work, samples of 1000 images with one or two types of defects are prepared.

The faults are of four types (i) Pitted surface, (ii) Inclusion (iii) Scratch defects, and (iv) Patch defects. The total 1000 images are classified into train and valid images. Table 4 shows the classification of the sample data set along with the fault detection model dataset.

Table 4
Classification of the sample data set along with the fault detection model data set

Class Pitted defect Inclusion defect Scratch defect Patch defect Multiple defect Total

No of images 280 107 334 154 125 1000

Fault detection model data Testing Training Testing Training Testing Training Testing Training Testing Training Testing Training

50 230 19 88 52 282 24 130 5 5 145 855

Class	Pitted defect	Inclusion defect	Scratch defect	Patch defect	Multiple defect	Total
Fault detection model data	Testing	Training	Testing	Training	Testing	Training	Testing	Training	Testing	Training	Testing	Training
50	230	19	88	52	282	24	130	5	5	145	855

The training and validation accuracy of the proposed method is enumerated in Figs. 7, 8,. The confusion matrix of the proposed scheme is depicted in Fig. 9. Here, three classes (Class 1, 2, 3) of data are tested in the proposed system. Class 1 symbolizes the normal class employed for testing. The benign class is represented by class 2 whereas the abnormal class is intended by class 3. The green diagonal refers to accurate fault diagnosis and the light blue diagonal implies the false diagnosis. Normal and benign classes are identified appropriately but two images are incorrectly identified as a benign class instead of abnormal class from the selected 320 images. The confusion matrix is used to examine fundamental performance metrics. The performance result demonstrates that the detection accuracy of the confusion matrix is achieved as 99.36%.

Fig.7

Training accuracy and loss computation for 10 epochs.

Fig. 8

Validation accuracy and loss computation for 10 epochs.

Fig. 9

The confusion matrix of the proposed detection system.

The performance comparison of the proposed AFRCNN scheme over the recently stated schemes is exposed in Table 5. It is noticed from Table 5 that the proposed AFRCNN scheme exhibits superior performance than other schemes by learning the most prominent fault attributes with 99.36% detection accuracy and 99.24% specificity. This is owing to the execution of the optimal model for extracting the candidate regions in the proposed scheme. Furthermore, the integrating of RPN along with the FCNN algorithms as a classifier in the suggested scheme permits to exploration of the entire fault characteristics of the stainless-steel plate. This appropriate integrating strategy paves a way to extend the detection accuracy during the dense faults.

Table 5

Performance comparison of proposed model over existing models

Models &Year	Classifier	Accuracy (%)	Specificity (%)	Recognition Rate (%)	Error Rate
[19] &2019	CNN	87.6	85.2	80.4	1.92
[21] &2019	CAE-SGAN	89.2	88.7	86.5	1.24
[18] &2020	YOLOv3	93.1	92.4	90.3	1.11
[17] &2020	PGA-Net	86.0	89.5	84.2	1.62
[16] &2021	RCNN	94.2	93.6	95.3	0.86
[15] &2021	SVM	97.3	96.0	96.3	0.89
[14] &2021	DEA	97.8	98.6	97.2	0.88
Proposed	AFRCNN	99.36	99.24	98.36	0.83

Meanwhile, the AFRCNN deals with only a few parameters in the testing and training evaluation. It yields minimal error rates and lesser execution time in the proposed scheme. The proposed AFRCNN scheme exactly forecasts the fault types, determining useful information for detecting and subsequently rectifying contributory factors. The RPN performs the extraction of candidate regions in the proposed scheme. Better detection of various shapes of characters is achieved with improved recognition rate and robustness.

The detection accuracy of the existing schemes is minimal due to the continuation of the misclassification error rate in the classification stage. Especially, the DEA and SVM classifiers lagged to forecast the influential fault attributes of the stainless-steel plate. Consequently, it generates a lower detection accuracy of 97.8% and 97.3% respectively as compared to the proposed scheme. Similarly, the traditional CNN, CAE-SGAN, and RCNN classifiers obtain lesser accuracy of 87.6%, 89.2%, and 94.2% respectively because of combining more hidden layers to accomplish convolution and pre-processing processes to forecast the fault attributes.

Constructing of the numerous layers generates larger computational complexity which yields a lower accuracy and recognition rate while compared to the remaining stated classifiers. Finally, the PGA-Net and YOLOv3 classifiers neglected to search the external fault area of the stainless-steel plates and also lagged to formulate the optimal elucidation for fault attribute learning. This paves a way to acquire lesser accuracy and specificity than the proposed AFRCNN scheme.

Based on Fig. 10, it is noticed that the proposed AFRCNN scheme generates better precision of 98.64% and the recall of 99.82% when compared with existing schemes like CNN [19], CAE-SGAN [21], YOLOv3 [18], PGA-Net [17], RCNN [16], SVM [15], and DEA [14]. The major rationale behind this substantial improvement is that the finest prediction of stainless-steel plate defects from the lower region to a higher region in the proposed scheme. In addition, the input faulted plate dataset is chiefly assessed with an appropriate pre-processing technique to evade the noise and non-useful content.

Fig. 10

Comparative result analysis of proposed scheme over existing schemes.

These pre-processing results assist in achieving an ultimate detection of the stainless-steel plate defects in the next stages. Thereafter, the effective feature forecasting mechanism is integrated with the proposed AFRCNN scheme guarantee to preserve the outer along with miniature shapes of the faults. It provides minimal forecasting errors in the feature learning phase and supports for larger fault plate dataset.

Conversely, the existing classifiers do not deliberate a proper scheme for forecasting the different fault shapes in the extraction stage. The larger resolution fault images are not estimated in the existing CNN, CAE-SGAN, and RCNN schemes. Further, they lagged to eradicate the redundant noise content in the larger faulted plate dataset. As a result, the forecasting errors are increased in the evaluation model that causing a minimum precision value of 84.26%, 90.04%, and 87% respectively as compared with the proposed AFRCNN scheme. Likewise, the PGA-Net and YOLOv3 schemes generate more misclassification errors due to the involvement of maximal levels of layers in the CNN scheme. They also failed to recognize the serial number of the defective plate.

It is observed from Fig. 11 that the proposed AFRCNN scheme retains a better F1-score of 99.18% and AUROC of 98.22% than the conventional schemes. These better results are because of implementing an efficient detection algorithm in the proposed AFRCNN scheme. In general, character recognition plays a major challenging process in stainless steel plate fault detection. But, the proposed AFRCNN can accurately utilize the ALPR technique for identifying the faulted plate and recognizing PN with a lesser error rate. Moreover, the usage of the proper layer in the AFRCNN reduces the input dimensionality and the evaluation complexity. It also formulates it more easily to convey the preferred layers to the subsequent levels by mitigating unnecessary layers.

Fig. 11

Comparison of F1-score and AUROC for different schemes.

Alternatively, the improper layer activities of the CNN, CAE-SGAN, and RCNN schemes produce intractable PN recognition in the stainless-steel plate. This paves a way to accomplish a lesser F1-score of 85.3%, 91.35%, and 90% respectively. On the other hand, training a model and reinforcement learning is very difficult in the PGA-Net and YOLOv3 schemes where they would not be stable and accurate. These two schemes lagged to learn tiny shaped attributes from the fault dataset owing to the high complexity of the training model. Hence, they produce lower F1-score and AUROC values than the proposed AFRCNN scheme.

4.3 Proposed ALPR results

The evaluation of FP detection and PN recognition has been performed in terms of Precision and Recall. In the FP detection and PN recognition system, both ResNet50 and VGG16 are chosen as the Base CNN for proposed the AFRCNN scheme. The experimental results are illustrated in Table 6. The character recognition accuracy is defined as the number of characters recognized correctly divided by the total number of ground truth. Table 6 reveals that the ResNet50 is working better than VGG16 as a base CNN for the AFRCNN scheme. Further, the proposed scheme achieves better FP detection and PN recognition by using the ALPR model.

Table 6
Results of FP Detection and PN Recognition for Base CNN with RESNET50 and VGG16

Method FP Detection PN Recognition

ResNet-50 VGG16 ResNet-50 VGG16

CNN [19] 89.03% 84.90% 96.92% 90.76%

CAE-SGAN [21] 67.46% 82.82% 86.53% 84.74%

YOLOv3 [18] 76.43% 78.62% 88.45% 82.82%

PGA-Net [17] 87.48% 82.72% 92.42% 92.79%

RCNN [16] 88.06% 92.07% 87.89% 92.90%

SVM [15] 87.28% 92.83% 92.36% 94.85%

DEA [14] 95.67% 95.38% 97.26% 97.42%

Proposed 97.24% 96.78% 97.90% 97.62%

Method	FP Detection	PN Recognition
CNN [19]	89.03%	84.90%	96.92%	90.76%
CAE-SGAN [21]	67.46%	82.82%	86.53%	84.74%
YOLOv3 [18]	76.43%	78.62%	88.45%	82.82%
PGA-Net [17]	87.48%	82.72%	92.42%	92.79%
RCNN [16]	88.06%	92.07%	87.89%	92.90%
SVM [15]	87.28%	92.83%	92.36%	94.85%
DEA [14]	95.67%	95.38%	97.26%	97.42%
Proposed	97.24%	96.78%	97.90%	97.62%

Here, character recognition is applied after segmentation to identify the alphabets and numerical. These activities are successfully performed by a single automated system which makes the proposed scheme superior over the other existing schemes. In ALPR, the parameters are fine-tuned in order to perform the task without any error. It also implements both fault identification and number plate recognition model which significantly enlarges the recall and precision of the proposed scheme. On the contrary, the existing schemes produce a large number of misclassification errors causes lesser performance than the proposed scheme. They only classify the fault images but fail to locate the spots and recognize the serial number of the defective plate.

5 Conclusion and future work

In this work, a deep learning-based automated AFRCNN scheme was proposed in which the RPN algorithm has been integrated with the FCNN for identifying the faulted stainless-steel plate. Initially, the RPN performs the extraction of candidate regions whereas the extracted region proposals contained in the candidate regions are fed to the FCNN to recognize the character of the stainless-steel plate. The number corresponding to the particular plate is recognized through the ALPR technique along with the character recognition technique. The proposed AFRCNN scheme provides valuable information about the location, category, and complexity. These combined features will offer accurate and fully automated results to guarantee the fault-free products of the stainless-steel plate. This automatic fault detection scheme can significantly enlarge product quality and production efficiency.

The effectiveness of the proposed AFRCNN scheme has been analyzed with the aid of the public dataset. The numerical results prove that the proposed scheme outperforms well when compared with the conventional fault detection schemes. In particular, the proposed AFRCNN scheme retains a superior F1-score of 99.18% and AUROC of 98.22%. The AFRCNN deals with only a few parameters in the testing and training evaluation. As a result, it yields lower error rates of 0.83 and lesser execution time. Better detection of various shapes of characters is achieved with an improved recognition rate of 98.36%. The aforesaid results guarantee that the proposed AFRCNN scheme will significantly support the industry during the manufacturing process for stainless steel plate fault detection.

In future research, the meta-heuristic optimization algorithms are combined with the proposed scheme to further enhance the detection performance and optimize the fault detection system.

References

Song

, Sudden-Discharge Cycling Characteristics and Millisecond Dynamic Behaviors of a HTS Stainless-Steel Insulated Double-Pancake Coil With Thin Copper Plates, IEEE Transactions on Applied Superconductivity 30 (2020), 1–10.

Yue

, Ma

and Liu

, Research on image classification method of strip steel surface defects based on improved Bat algorithm optimized BP neural network, & }, Fuzzy Systems 41 (2021), 1509–1521.

Merot

, Morel

and Mayorga

, Observations on the influence of process and corrosion related defects on the fatigue strength of 316L stainless steel manufactured by Laser Powder Bed Fusion (L-PBF), International Journal of Fatigue 155 (2022), 106552:1–16.

Narendiranath

, Babu, N. Senthilnathan and S. Pancholi, Fault analysis on continuous variable transmission using DB-06 wavelet decomposition and fault classification using ANN, Journal of Intelligent & Fuzzy Systems 41 (2021), 1297–1307.

Tian

and Jia

, DCC-CenterNet: A rapid detection method for steel surface defects, Measurement 187 (2022), 110211:1–18.

Zhang

and Wang

, An accurate fuzzy measure-based detection method for various types of defects on strip steel surfaces, Computers in Industry 122 (2020), 103231:1–18.

Liu

, Luo

and Li

, A New Self-Reference Image Decomposition Algorithm for Strip Steel Surface Defect Detection, , IEEE Transactions on Instrumentation and Measurement 69 (2020), 4732–4741.

Luo

, Fang

and Liu

, Automated Visual Defect Detection for Flat Steel Surface: A Survey, , IEEE Transactions on Instrumentation and Measurement 69 (2020), 626–644.

Kim

, Choi

K.N.

and Kang

, Infrared Thermal Image-Based Sustainable Fault Detection for Electrical Facilities, Sustainability 13 (2021), 557:1–23.

10.

Lavanya

, Prasanth

and Jayachitra

, A Tuned classification approach for efficient heterogeneous fault diagnosis in IoT-enabled WSN applications, Measurement 183 (2021), 109771:1–16.

11.

Wang

, Zhang

and Tian

, A simple guidance template-based defect detection method for strip steel surfaces, , IEEE Trans Ind Informat 15 (2019), 2798–2809.

12.

Luo

, Sun

and Li

, Generalized completed local binary patterns for time-efficient steel surface defect classification, , IEEE Trans Instrum Meas 68 (2019), 667–679.

13.

Liu

and Yao

, Development of a Bidirectional-Excitation Eddy-Current Sensor With Magnetic Shielding: Detection of Subsurface Defects in Stainless Steel, , IEEE Sensors Journal 18 (2018), 6203–6216.

14.

Cheng

and Yu

, RetinaNetWith Difference Channel Attention and Adaptively Spatial Feature Fusion for Steel Surface Defect Detection, IEEE Transactions on Instrumentation and Measurement 70 (2021), 1–10.

15.

, Hu

and Lan

, Simulation and quantitative study of cracks in 304 Stainless steel under natural magnetic field, NDI & E International 119 (2021), 1–17.

16.

Wang

, Xia

and Ye

, Automatic Detection and Classification of Steel Surface Defect Using Deep Convolutional Neural Networks, Metals 11 (2021), 388:1–22.

17.

Dong

, Song

, He

and Xu

, PGAnet: pyramid feature fusion and global context attention network for automated surface defect detection, , IEEE Transactions on Industrial Informatics 16 (2020), 7448–7458.

18.

Zhang

, Kanga

, Ni

Surface defect detection of steel strips based on classification priority YOLOv3-dense network, Ironmaking & Steelmaking, (2020), 1–13.

19.

, Sun

and Zhu

, A deep-learning-based approach for fast and robust steel surface defects classification, Optics and Lasers in Engineering 121 (2019), 397–405.

20.

, Song

and Meng

, An End-to-end Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features, IEEE Transactions on Instrumentation and Measurement 69 (2020), 1493–1504.

21.

, X

and Peng

, Surface defect classification of steels with a new semi-supervised learning method, , Opt Laser Eng 117 (2019), 40–48.

22.

Arjun

, Sasi

and Rao

B.P.C.

, Optimization of pulsed eddy current probe for detection of subsurface defects in stainless steel plates, , Sensors and Actuators A: Physical 226 (2015), 69–75.

23.

and Zhang

, Non-contact detection of delamination in stainless steel/ carbon steel composites with lazer ultrasonic,, , Optic 226 (2021), 1–17.

24.

Ali

, Kazemi

and Hajianand

, Quality Control and Classification of Steel Plates Faults Using Data Mining, Applied Mathematics & Information Sciences Letters An International Journal 2 (2018), 59–67.

25.

Jayachitra

, Prasanth

An Efficient Clinical Support System For Heart Disease Prediction Using TANFIS Classifier, Computational Intelligence, (2021), 1–28.

26.

Prasanth

, Certain Investigations on Energy-Efficient Fault Detection and Recovery Management in Underwater Wireless Sensor Networks, Journal of Circuits, Systems and Computers 30 (2021), 2150137:1–20.

27.

Jayachitra

and Prasanth

, Multi-Feature Analysis for Automated Brain Stroke Classification Using Weighted Gaussian Naïve Baye’s Classifier, Journal of Circuits, Systems, and Computers 30 (2021), 2150178:1–26.

28.

Kalli

, Suresh