Abstract
The degree of sensitisation is a reliable measure of the susceptibility to intergranular corrosion for Al-Mg and Al-Mg-Mn marine-grade alloys (Al 5XXX series). Measuring the degree of sensitisation is an exhaustive and expensive operation, as it requires destructive sectioning and outsourcing the measurement to a laboratory to determine the necessary corrosion risk level. This paper proposes an alternative method to predict degree of sensitisation using a convolutional neural network trained on microstructural images of AA5456 alloy specimens etched with phosphoric acid to reveal β-phase precipitates along grain boundaries. Two convolutional neural network architectures are proposed to predict the degree of sensitisation of AA5456 specimens from 490 microstructural images at 200× and 500× magnification. An overall accuracy of 83% for a continuous prediction convolutional neural network model and 87% for a classification prediction convolutional neural network model. The high classification accuracy reflects the success of the augmentation process, which increased the dataset to 9000 images with an equal distribution of degree of sensitisation targets.
Introduction
The susceptibility to intergranular corrosion of Al-Mg and Al-Mg-Mn marine-grade alloys (Al 5XXX series), also known as the degree of sensitisation (DoS), remains a persistent concern for their use aboard Navy vessels.1–4 Alloys become sensitised by Mg diffusion to grain boundaries, forming an anodic Al3Mg2 intermetallic phase (β-phase),1,4,5,6 which preferentially corrodes and drastically increases the risk of failure through stress-corrosion cracking.1,2,4,5 The process of sensitisation has been documented to occur in alloys with a weight percentage greater than 3% Mg at elevated temperatures as low as 50 °C,4,6 posing a problem for any areas aboard Navy vessels that experience long-term solar radiation that can reach these temperatures, as well as welded areas that receive heat input that can cause locally sensitised regions.4,6
Accurate measurements of the DoS are obtained using the ASTM G67 standard, 7 which immerses a 2′′ × 0.5′′ × 0.5′′ specimen in 100 mL of nitric acid heated to 30 °C for 24 h, after which the mass loss is recorded and used to determine the DoS. This test requires destructive sectioning and outsourcing the measurement to a laboratory to determine the necessary risk level. An alternative method is desired to improve efficiency in assessing the DoS for weld repair.
The use of convolutional neural network (CNN) machine learning models (MLMs) to perform both classification and continuous predictions on image-based datasets has been proven effective for a wide variety of applications, including cancer diagnosis from histological tissue images, 8 human behaviour and facial recognition,9,10 defect detection in additively manufactured materials, 11 and handwriting digit recognition,12,13 to name a few. CNN algorithms are trained in the same framework as standard artificial neural networks. Still, they are especially suited for image-based analysis via convolutional operations that decompose image features into the neurons of the network's initial layers. 12
In this work, an alternative method for predicting DoS is presented using a CNN trained on microstructural images of AA5456 alloy specimens etched with a phosphoric-acid solution to reveal β-phase precipitates along grain boundaries. A metallographic approach would enable a quicker timeline for assessing the DoS, reducing costs and improving efficiency to facilitate weld repairs within the Navy. The investigation involved training CNNs for continuous DoS value prediction and for classifying DoS within the risk category for repair actions on Al 5XXX series alloys. The Navy classifies the risk of weld repair on Al 5XXX series alloys into three regions based on the magnitude of the DoS, expressed as mass loss (mg/cm2). Risk levels are categorised as low, medium and high for Al 5XXX series alloys based on DoS values: < 30 mg/cm2, 30–60 mg/cm2 and > 60 mg/cm2, respectively.
This paper is organised as follows. The methodology is elaborated in the following section. Simulation results and detailed discussions are discussed next. Following that, a summary and conclusions are drawn.
Methodology
In this paper, CNNs were developed using microstructural images for the assessment and classification of DoS within risk categories for weld repair.
Numerous iterations of CNN architectures were tested in the pursuit of optimising the accuracy based on key hyperparameters: (1) the number of neural network layers; (2) the types of layers used (i.e. convolutional, batch normalisation, max pooling, fully connected); (3) the dimensionality, or the number of features, between those layers; (4) the activation function or functions used to mimic neural ‘firing’; (5) the number of epochs, or training loops, used to adjust weights within the activation functions; (6) the batch size of sampling used within a single epoch; (7) the loss function used to quantify error during the training and testing processes, which informs adjustment of weights using stochastic gradient descent; and (8) the learning rate, which limits the increment at which weights in the activation functions are allowed to change during training.
Initial architectures for the CNNs were drawn from the literature,9,10,12,13,14 which served as helpful templates to establish a baseline for hyperparameter values to iterate on in pursuit of an optimal CNN model. This paper will briefly discuss the various considerations applied to each of the hyperparameters listed above for the development of the continuous and classification CNN models. Still, it will not cover all the different architectures tested before settling on a more optimised architecture.
Available dataset
A dataset of 980 microstructural images of AA5456 specimens, sectioned from naval vessels, was aggregated for use in training and validation of the CNN MLMs. Each specimen's DoS was determined using the ASTM G67 nitric-acid mass-loss test, and these measured mass-loss values (mg/cm2) served as the ground-truth labels for CNN training and evaluation. Specifically, each AA5456 specimen was tested by ASTM G67 (24-h immersion in heated nitric acid) to obtain the DoS value; the same specimen was then metallographically prepared and imaged at 200× and 500× magnifications for the dataset. All supervised training, regression analyses (R2, root-mean-square error (RMSE)) and classification metrics reported in this paper compare predicted outputs directly with the ASTM G67-measured DoS values. The raw image files were in RGB colour format. They had variable pixel dimensions, ranging from 600 to 800 pixels in width and 550 to 650 pixels in height, along with a scale bar to help distinguish between the two magnifications. A custom script in ImageJ was used to preprocess the raw files into a 300 × 300-pixel format, preserving the RGB values, to establish a consistent 300 × 300 × 3 dimensionality for use as tensor inputs to the CNN model. A set of processed images at 200× and 500× magnifications is given as an example in Figure 1, with DoS values superimposed.
Example images of AA5456 microstructures at 200× and 500× magnifications with degree of sensitisation (DoS) values superimposed.
DoS analysis and developed CNN models
Two efforts were conducted in parallel as part of a dual-pronged analysis to determine the optimal network architecture for accuracy. The first involved developing continuous-value prediction CNNs to output DoS from an image as mass loss (mg/cm2), which was then used to classify the DoS into risk levels (low, medium and high). This type of network yields two ways to assess accuracy: the continuous output predictions and the derivative classifications derived from them. The former was assessed using linear regression of actual and predicted values for a given trained model, yielding R2 and RMSE metrics. The R2 value indicates the correlation between the predicted and actual values, and the RMSE indicates the average deviation of the error between the predicted and actual mass loss. The derivative classifications were evaluated through confusion matrices to examine the individual precision for each classifier as well as overall accuracy based on averaging the precisions, given by:
The second effort involved developing CNNs to predict classification values, thereby directly outputting a risk classification. These models were evaluated using the same method as the derivative classifications from the first effort, allowing for a 1:1 comparison between the efforts to determine which type of CNN is optimal for DoS classifications.
For the continuous prediction MLMs, four separate datasets were used in developing the CNN models: (1) a random sampling of the 200× magnification images, (2) a curated sample of the 200× images as input with a more balanced distribution of DoS targets, (3) a sampling of the 500× magnification images indexed according to the same random assignment as dataset (1), and (4) a curated sample of the 500× images with the same balanced distribution and indexing of DoS targets as dataset (2).
This approach to sampling data in different ways addresses two key considerations. The data itself corresponds to pairs of 200× and 500× images from the same physical specimen, so by linking indexes for training files between datasets (1) and (3) and datasets (2) and (4), one can determine if the process of obtaining data at one magnification versus the other is a more efficient method of inspection when used as input to CNNs trained on the respective magnifications. In other words, if CNNs for 200× images were more accurate than CNNs for 500× images, or vice versa, one could recommend that inspectors only need to perform one practice of imaging to assess DoS in this manner. The rationale for sampling a more balanced distribution is illustrated in Figure 2, which shows the histogram of overall DoS values across the 490 images at a given magnification. The target DoS values of the used dataset ranged from 1.2 to 89.8 mg/cm2. From the histogram, it is clear that the data exhibit a bias towards lower sensitisation values: 56.74% fall below 30 mg/cm2, 34.69% between 30 and 60 mg/cm2 and 8.57% above 60 mg/cm2. As such, datasets (1) and (3) were randomly sampled using a conventional 80–20 split for training and test data. Datasets (2) and (4) were curated to create a more even distribution of DoS targets, but this was achieved through a 50–50 split of training and test data across the 490 images in a given magnification. Figure 3 illustrates the histogram of DoS targets in the curated datasets, showing a more balanced distribution of DoS than the overall histogram in Figure 2. However, the authors note that both approaches risk overfitting, either due to distribution bias or insufficient sample sizes to capture variance within similar cases.15,16
A histogram for DoS values used as output targets for the CNN models predicting DoS based on an AA5456 microstructural image dataset.
The histogram of DoS target values within a curated subset of the overall AA5456 microstructural image dataset is used to obtain a more even distribution for training CNN models.
All continuous prediction models were developed, and their hyperparameters were fine-tuned using the PyTorch package in Python. After numerous iterations across the eight parameters listed above, each model was trained and evaluated using the same architecture, as depicted schematically in Figure 4 and explicitly stated in Table 1. For training, batch sizes of 40 images, 150 epochs, a learning rate of 5 · 10 − 5 and the mean-squared error loss were used with the ‘Adam’ optimisation algorithm, which has been shown to demonstrate very efficient weight adjustments for the activation functions in the stochastic gradient descent approach for training CNNs. 17 The rectified linear unit (ReLU) function was used for activating nodes between layers.
Schematic of the CNN architecture used to train continuous DoS predictions from AA5456 microstructural images.
Architecture of the continuous prediction CNN models.
Total params: 10,572,809; Trainable params: 10,572,809; Non-trainable params: 0.
The efficacy of each model was evaluated by performing linear regression on the predictions made by the MLMs for DoS values of images in the testing datasets against the known DoS values from those images. The accuracy is measured by both the R2 value from this regression, or the correlation between the predictions and actual values, and the RMSE, which represents the average deviation of the predicted DoS from the target values. In addition to looking at the performance of models individually, test data predictions were averaged across models (1) and (3) and models (2) and (4) to examine whether the use of the set of 200× and 500× images taken on the same sample could lead to better predictive power.
For the classification prediction MLMs, three separate CNN models were created in Python: (1) one trained on both 200× and 500× magnification images with augmentation through the Keras package, (2) one trained on the 200× images with augmentation through the Scipy package and (3) one trained on the 500× images with augmentation through the Scipy package. Training and test data were split 80–20. In all models, images were pre-processed to normalise pixel values across the dataset and rescaled to 227 × 227 × 3 dimensions. Data augmentation techniques were used to increase the total amount of data available for both training and validation. This included random rotations within ±180°, horizontal and vertical shifts up to approximately 10% of the image width (±64 pixels for our 640 × 640-pixel micrographs), and horizontal/vertical flips. No zoom (scaling) transformation was applied in the SciPy-based augmentation, as preliminary tests showed interpolation during scaling could distort fine microstructural features (please refer to Figure 5). The SciPy rotate() function was implemented with mode = 'reflect’ and order = 0 (nearest-neighbour interpolation) to fill blank pixels created at image edges, thereby preventing the introduction of black borders. Figure 6 illustrates the nearest-pixel filling approach. The use of different Python packages for augmentation aimed to examine the effect of interpolation and its distortion on the base images. In Keras, augmentation algorithms modify pixels via interpolation to smooth over new regions introduced by the augmentation, whereas in Scipy, interpolation is not used at all. With augmentation, the overall dataset used for classification-prediction MLMs comprised 9000 images, 3000 per weldability category, as per NAVSEA guidance. This augmentation created a more balanced distribution of target output values, thereby overcoming the bias in the base dataset's distribution.
Example of the data augmentation methods used on images for the classification prediction CNN models.
Example of the nearest-pixel filling used to eliminate blank edges caused by rotation and translation during SciPy augmentation of images for the classification prediction CNN models. Rotations use reflect mode with nearest-neighbour filling (order = 0) to avoid black borders.
All classification prediction models were developed using the Keras package in Python. Hyperparameters were chosen to reflect the AlexNet CNN architecture, as originally detailed in 14 and introduced in the ImageNet Large Scale Visual Recognition Challenge. The architecture has demonstrated efficient, accurate performance on ImageNet, a massive dataset comprising 15 million images with 22,000 unique classification labels. 18 The AlexNet CNN architecture comprises eight layers, including five convolutional layers and three fully connected layers. Each convolutional layer consists of convolution, pooling and normalisation. In the original architecture, 14 the final fully connected layer in the network uses a softmax activation function, producing a vector representing a probability distribution over 1000 classes. For this work, the AlexNet CNN architecture has been modified slightly for both reducing computational time and reducing the number of outputs to three classes: (1) a low-risk class corresponding to weldability in NAVSEA guidance (i.e. DoS <30 mg/cm2), (2) a medium risk class corresponding to weldability with additional cold-working and (3) a high-risk class corresponding to not weldable. The explicit architecture is given in Table 2. For training, batch sizes of 120 images, a learning rate of 1 × 10−3 and the cross-entropy loss were used with the stochastic gradient descent optimisation algorithm. For the combined CNN trained on 200× and 500× images using data augmentation with the Keras package, 100 training epochs were used. For the CNNs trained separately on 200× and 500× images using data augmentation with the SciPy package, 200 epochs were used. In addition to this difference in training schedule, Keras-based augmentation interpolates newly created pixels, whereas our SciPy pipeline avoids interpolation entirely (nearest-neighbour for rotations; np.roll for shifts), which we adopt to preserve pixel-intensity statistics in etched micrographs. Model accuracy is measured as the ratio of correct predictions to the total number of predictions made during the test split fed into the models. The interpretation of their performance is further broken-down using confusion matrices.
Architecture of the classification prediction CNN models.
Total params: 12,341,891; Trainable params: 12,341,203; Non-trainable params: 688.
It is worth mentioning that the continuous-output CNN contains approximately 10.57 million trainable parameters, and the classification CNN has about 12.34 million trainable parameters (Tables 1 and 2). These values are within the typical range for deep CNNs (in the order of 106 parameters) used in comparable image classification tasks.
Simulation results and detailed discussion
Simulation results
The evolution of loss over epochs during the training of continuous prediction CNNs is shown in Figure 7. These plots serve as a check to determine whether the training converged and to reflect the model's optimisation during training. Figure 7(a) and (b), corresponding to models trained on the random 80–20 split of the training and testing datasets, show that both models converged to a training loss of approximately 1.5 and a testing loss of 2.5 in the mean-squared error. Figure 7(c) and (d), corresponding to models trained on the curated 50–50 split-square training and testing data, show that both models converged to a training loss of approximately 2.5 and a testing loss of 4.5 in the mean-squared error.
The evolution of loss during training of the various continuous prediction CNNs for DoS based on AA5456 microstructural images. (a) Loss for the CNN trained on a random 80% of the 200× magnification images. (b) Loss for the CNN trained on a random 80% of the 500× magnification images. (c) Loss for the CNN trained on a curated subset of 50% of the 200× magnification images. (d) Loss for the CNN trained on a curated 50% of the 500× magnification images.
Plots of predicted versus actual values for validating the individual and averaged CNN models using the test datasets are shown in Figures 8 and 9, respectively. The linear regression in these figures compares the CNN-predicted DoS with the actual values obtained from ASTM G67 nitric acid mass-loss measurements. All regression and accuracy metrics reported (R2, RMSE and classification accuracies) are computed by comparing model predictions to the ASTM G67 measured DoS values for each specimen. A summary of the R2 scores and RMSE values from linear regression is given in Table 3. These results clearly indicate that models trained on a random 80–20 split of training and test data perform better than those trained on a curated 50–50 split with a more balanced distribution of target DoS values, as evidenced by higher R2 scores and lower RMSE for the former. It thus implies that having more data to train from, even from an unbalanced distribution of target values, contributes more to accuracy than using less data with a more balanced distribution. For the 80–20 split models, averaging the predictions from the two yielded the highest accuracy, with an R2 score of 0.830 and an RMSE of 8.55 mg/cm2. However, averaging the predictions between the curated 50–50 split models worsened overall accuracy, and this is likely due to the inferior performance of the 50–50 split model trained on the 200× magnification images having a negative R2 score, which essentially means the model performs worse than using a model that predicts the exact same average DoS no matter the input.
Plots of predicted versus actual DoS values for the various continuous-output CNN models used for linear regression to determine model accuracy. The red line indicates a perfect prediction. Actual DoS values are from ASTM G67 nitric-acid mass-loss measurements. (a) Results for the CNN trained on a random 80% of the 200× magnification images. (b) Results for the CNN trained on a random 80% of the 500× magnification images. (c) Results for the CNN trained on a curated subset of 50% of the 200× magnification images. (d) Results for the CNN trained on a curated subset of 50% of the 500× magnification images.
Plots of predicted versus actual DoS values averaged across the CNN models. (a) Results for the average between CNNs trained on 80% of the 200× and 500× magnification images. (b) Results for the average across CNNs trained on a curated subset of 50% of the 200× and 500× magnification images. Actual DoS values are ASTM G67 measurements.
A summary of the linear regression fits, predicted versus actual DoS values, for the various CNNs trained on AA5456 microstructural images. Actual DoS values correspond to ASTM G67 nitric acid mass-loss measurements.
To directly compare the effect of magnification, we examined CNN performance for images captured at 200× and 500×. The classification CNN trained on 500× images attained approximately 87% testing accuracy, whereas the 200× model achieved about 84%. For continuous prediction, Table 3 shows R2 = 0.798 (RMSE = 9.32 mg/cm2) for 500× and R2 = 0.783 (RMSE = 9.66 mg/cm2) for 200×. These results indicate a modest but consistent advantage at higher magnification. We attribute this improvement to finer microstructural resolution at 500×, which enhances the CNN's ability to identify β-phase precipitate networks along grain boundaries. Conversely, 200× images encompass a larger field of view, averaging over more grains and increasing heterogeneity, which may reduce sensitivity to localised sensitisation features. In practice, if a single magnification must be chosen, 500× offers slightly greater accuracy; however, averaging results across both magnifications provides the best overall performance (R2 = 0.830), suggesting complementary information across scales.
In the highest-accuracy model, the results in Figure 9(a) show that the model is more accurate for lower DoS values. For the continuous output model, one would first predict the DoS from an image and then use that value to categorise the weld repair risk as low, medium, or high. Confusion matrices, based on redefining the continuous predictions from the highest-accuracy model as categorical risk, are shown in Figure 10 to subdivide the model's accuracy in determining the appropriate repair action. Figure 10(a) shows the same plot from Figure 9(a) with coloured rectangles superimposed to visually demonstrate which data points fall into the correct classification of repair action. Figure 10(b) and (c) are the confusion matrices that quantify the classifications, with Figure 10(b) showing the absolute counts and Figure 10(c) showing the percentages summed over the rows of the actual risk category. The distribution of target test values shows a bias toward lower risk categories, as reflected in the overall data: 283 actual values for low risk, 172 for medium risk and 35 for high risk. In terms of proportions, the low-risk category had 91.5% accurate classifications, followed by 77.9% and 37.1% for the medium and high-risk categories, respectively. For actual high-risk values, 62.9% showed a false-negative prediction of medium risk, whereas true medium-risk values showed only 0.6% false-positive high-risk predictions. For true medium-risk values, 21.5% showed a false-negative low-risk prediction, whereas for true low-risk values, 8.5% showed a false-positive medium-risk prediction.
Redefining the accuracy in terms of weldability risk for the averaged predictions of the 200× and 500× continuous output CNNs trained on an 80–20 split of the microstructural image dataset. (a) Results of predicted versus actual values with coloured rectangles superimposed to show regions of correct risk assessment. (b) A confusion matrix quantifying the number of correct and incorrect classifications from the predictions. (c) A confusion matrix shows the proportion of correct and incorrect classifications from the projections.
However, there was no misclassification between true low and high-risk targets. The evolution of loss over epochs during the training of classification-prediction CNNs is shown in Figure 11. For the model that used the Keras package for augmentation (Figure 11(a), one can see that the test-time loss did not track the training loss and remained relatively unstable throughout the process. This is contrasted with the results in Figure 11(b) and (c), which demonstrate convergence in both the training and test losses with much greater stability.
The evolution of loss during training of the various classification prediction CNNs for DoS based on AA5456 microstructural images. (a) Loss for the CNN trained on both 200× and 500× images, with augmentation performed using the Keras package. (b) Loss for the CNN trained on 200× images with augmentation using the Scipy package. (c) Loss for the CNN trained on 500× images with augmentation using the Scipy package.
Plots of the accuracy for the three classification prediction models as a function of training epochs are shown in Figure 12. The training accuracy converges to nearly 100% in all models, indicating that the CNN has created an ideal fit to the training dataset. The ultimate accuracy of the models is judged by the test accuracy at the final epoch. For the CNN trained on augmented data using the Keras package (interpolation applied), the test accuracy was highly unstable, averaging around 70%. Still, the final accuracy dipped to about 66%. For the CNNs trained on augmented data using the Scipy package (without interpolation), the test accuracy converged to a stable value after approximately 25 epochs. The CNN trained on 500× images achieved the highest accuracy, around 87%, while the model trained on 200× images achieved 84%.
The evolution of accuracy during training of the various classification prediction CNNs for DoS based on AA5456 microstructural images. (a) Accuracy for the CNN trained on both 200× and 500× images with augmentation done by the Keras package. (b) Accuracy for the CNN trained on 200× images with augmentation using the Scipy package. (c) Accuracy for the CNN trained on 500× images with augmentation using the Scipy package.
To further assess the accuracy of the CNN models, confusion matrices are shown in Figures 13 to 15. For the model trained on both 200× and 500× images with augmentation using the Keras interpolation (Figure 13), the accuracy for the low- and high-risk categories is higher than the overall accuracy shown in Figure 12(a), at 83.5% and 87.1%, respectively. The model performed poorly at classifying medium-risk images, achieving an accuracy of only 28.0%. Both of the models trained on images augmented with the Scipy package (no interpolation) performed similarly, with the model trained on 200× images performing slightly better overall, as seen in Figure 14. In both cases, false positives and negatives between medium- and high-risk images were around 10%, and between low- and medium-risk images, around 8%. However, the model trained on 500× images showed significantly lower false-positive and false-negative rates between low- and high-risk cases, at around 1.5%, compared to 4.5% for the model trained on 200× images.
Confusion matrix of predictions made on testing data for the classification prediction CNN model trained on both 200× and 500× magnification images with augmentation by the Keras package. (a) Absolute quantity of predictions. (b) Proportional quantity of predictions.
Confusion matrix of predictions made on testing data for the classification prediction CNN model trained on 200× magnification images with augmentation by the Scipy package. (a) Absolute quantity of predictions. (b) Proportional quantity of predictions.
Confusion matrix of predictions made on testing data for the classification prediction CNN model trained on 500× magnification images with augmentation by the Scipy package. (a) Absolute quantity of predictions. (b) Proportional quantity of predictions.
Detailed discussions
Both types of prediction models, continuous versus classification, were able to predict DoS for the AA5456 microstructural images with an accuracy of 80%. Comparing the two types of models, it was observed that the highest accuracy from the continuous prediction CNNs was achieved when combining the predictions from both models trained on a random 80–20 split of the overall dataset. For the classification prediction model, the highest accuracy was achieved with the model trained solely on the 500× images, with data augmentation performed without interpolating the base images. While the classification model in its current state cannot average between 200× and 500× predictions, the improvement in accuracy from averaging continuous prediction models suggests that these models would be even more accurate if microstructural images were sampled at multiple locations across the surface.
To demonstrate this notion, consider the case of taking multiple images for a binary classification problem, that is, the output can be only 0 or 1. Each image will have only one output, and the final output will be the mode of the aggregated outputs. In the case where there are equal amounts of 0 s and 1 s, it will be counted as an incorrect prediction. The possible combinations of 0 and 1 predictions as a function of multiple images taken follow Pascal's triangle for binomial distributions, given by the expression:
The coefficients (n choose k) in Equation (3) are derived from Pascal's triangle and represent the binomial distribution of correct and incorrect outcomes. For this consideration, the probability that a prediction from aggregated samples is correct, P1, comes from collecting all terms from the sum where k < (n − k),
In this model, each coefficient
This means that when each image has an 80% probability of correct classification, aggregating predictions from five independent images yields over 95% confidence in the majority-vote result. This is represented visually in Figure 16, a depiction of Pascal's triangle with regions highlighted to show which coefficients fall under which probability calculation.
A depiction of Pascal's triangle to show which coefficients would correspond to a correct versus incorrect prediction result based on the number of samples in a binary choice problem and the probability of their occurrence.
Figure 17 shows how the probability of making a correct versus an incorrect prediction changes as the number of sampling events increases, with the likelihood of an accurate prediction set to 80% and that of an inaccurate prediction set to 20%. From this, it's clear that with 80% confidence in a single sample, the chance of making a correct prediction exceeds 95% with at least nine samples.
The calculation of making a correct prediction as a function of individual predictions, with the individual probability of being correct set to 80% and incorrect set to 20%.
Table 4 shows the results of correct predictions with increased sampling for the highest-accuracy CNN models. For the calculations, the accuracy values for true positives from the confusion matrices were used as the probability of correct predictions, and the aggregate of false positives was used as the probability of incorrect predictions.
Calculations of making a correct prediction based on the mode of aggregated individual predictions and the individual correction prediction probability for the three risk categories using accuracies from both the continuous and classification prediction CNNs.
These results demonstrate that averaging predictions from the classification CNN on only five microstructural images of an AA5456 specimen increases the probability of an accurate prediction to >95% across all risk categories, and this probability increases to >99% when up to 15 images are used. For the continuous CNN prediction, however, due to the low individual correct-prediction probability for the high-risk category, the likelihood of making a correct prediction averaged over multiple images drops even further, and this is the case for any individual accuracy <50% for an individual correct prediction.
Beyond an end-use method to further boost confidence in making DoS and weld-repair action assessments on AA5456 microstructures using CNNs, there is still room for improvement in individual prediction accuracy. The first step would be to aggregate more input data to increase the overall size of the training dataset. While neural networks have been shown to generalise well even on small datasets, 19 it has been demonstrated that neural network accuracy tends to improve proportionally to the logarithm of the training dataset size. 20 Larger amounts of data are also necessary to reduce overfitting and overcome input variance due to noise. 21 The initial dataset used in this study was considerably small, with only 490 total images per magnification, which required augmentation to generate a training dataset in the thousands. Still, some of the most successful neural networks trained on the MNIST handwritten digit database or ImageNet for object classification have training datasets in the tens of thousands to several millions without using any data augmentation.22,23 Unfortunately, there is no comprehensive repository of AA5XXX series microstructures with corresponding ASTM G67 DoS measurements, so generating additional input data will require considerable metallographic work. Another focus for generating more data would be to capture a more even distribution of DoS target values, so that data curation or augmentation is not necessary to remove this bias from the original dataset. The second step to improve accuracy is to reduce noise in the input data, which, for the AA5456 micrographs, involves developing a standardised approach to capture these images. That is, the accuracy of the CNNs would likely improve if each micrograph were taken under standard lighting, focus and white-balancing conditions, thereby reducing the variance in colour, brightness and feature sharpness across images with similar DoS values. However, this approach strikes a somewhat difficult balance, requiring end-users to adhere to a set of microscopy standards while avoiding the risk of creating models that cannot generalise predictions when conditions deviate from the norm.
With the success of these CNN MLMs in predicting DoS in AA5456 microstructures, there is also the question of how well they may perform on other naval alloys, such as AA5083, which shares the same risks of sensitisation and corrosion susceptibility. Currently, there is no existing dataset within the Navy that would allow validation of these CNNs for a separate aluminium alloy series, which represents an opportunity to expand the use of these models or the use of the model architectures to generate new CNNs that would be able to diagnose DoS based on microstructures of different alloys accurately.
Limitations and robustness
Our current approach did not include explicit preprocessing for low-contrast or noisy micrographs. All training images had relatively consistent quality, and we relied primarily on data augmentation (rotations, shifts and scaling) to improve generalisation. However, in practical field scenarios, such as varying illumination, etching depth, or imaging noise, these factors may affect the consistency of predictions. Contrast enhancement methods, such as Contrast-Limited Adaptive Histogram Equalisation (CLAHE), have been shown to improve the visibility of metallurgical features, such as grain boundaries and β-phase precipitates, by locally increasing contrast and image entropy.23,24 Similarly, incorporating synthetic noise during training or applying denoising filters can improve the resilience of CNNs to real-world image noise. 25 Although these techniques were not implemented in the present study, they represent practical and computationally lightweight extensions for future work. For field-deployment applications such as Navy inspections, our achieved accuracy (>80%) already exceeds the operational baseline of ≥70%, but integrating adaptive contrast enhancement and noise-handling strategies could further improve robustness under variable imaging conditions.
Data and code availability
The Python code for training and evaluating the CNN models, as well as the AA5456 micrograph dataset with corresponding ASTM G67 DoS values, are available from the authors upon reasonable request and subject to prior approval from the U.S. Navy.
Summary and conclusions
The development of CNN architectures to predict the DoS of AA5456 specimens based on 490 microstructural images, each at 200× and 500× magnification, was successful, achieving overall accuracies of 83% for a continuous-prediction CNN model and 87% for a classification-prediction CNN model. By redefining the continuous prediction model output as risk, this model underperformed at diagnosing the high-risk category (DoS > 60 mg/cm2), with an accuracy of only 37.1% in these cases. This appears to reflect a bias in the original dataset, where DoS values were heavily distributed towards lower numbers, and was not corrected by attempting to curate a training dataset with a more even distribution of DoS targets.
The classification prediction model, augmented with 9000 images distributed equally across DoS targets, achieved an accuracy of more than 82% for each risk category. It was shown that having an individual prediction accuracy of >82% could lend itself to even more accurate predictions by making multiple predictions on images of a specimen at unique locations, where taking the mode of these various predictions can increase accuracy to >96% with the use of five sample images or >99% with the use of 15 sample images.
On 2-D optical micrographs versus nanoscale 3-D characterisation. We acknowledge that sensitisation (DoS) fundamentally arises from the 3-D distribution and continuity of Mg-rich β-phase (Al₃Mg₂) precipitates along grain boundaries, which can be directly imaged and quantified by high-resolution techniques such as TEM/STEM or 3-D atom-probe tomography. Such methods provide nanoscale and 3-D information about precipitate continuity and composition, but they require extensive sample preparation and specialised instrumentation and are time-consuming and costly. For routine or field applications (e.g. shipboard inspection), polished-and-etched 2-D optical micrographs are a practical alternative: chemical etching highlights grain-boundary β-phase and produces surface features that correlate with DoS, and that can be captured quickly at 200×–500× magnifications. Therefore, our CNN approach intentionally leverages the speed, low cost and practicality of 2-D etched optical microscopy for rapid DoS screening. We explicitly acknowledge the limitation that 2-D optical images cannot reveal subsurface or extremely fine nanoscale precipitates below optical resolution and may not capture full 3-D continuity.
Future work
Further work could improve the CNNs’ accuracy by either generating a larger dataset by collecting micrographs and DoS measurements from more AA5456 specimens or by standardising the procedure for taking these micrographs to ensure consistent lighting, focus and white balance, thereby reducing noise in the input data. Both approaches involve laborious metallographic preparation and numerous ASTM G67 DoS measurements. In addition, the model should be validated against microstructural images and DoS values of AA5083 to determine if it generally applies to AA5XXX series alloys or if the same CNN architecture could be used to create a separate model for other alloy systems.
Additionally, future efforts will focus on enhancing the robustness and generalisation of the CNN models under diverse imaging and operational conditions. Specifically, adaptive contrast enhancement techniques, such as CLAHE-based preprocessing pipelines,23,24 and noise-handling strategies, including denoising filters and noise-augmentation during training, 25 will be investigated to improve feature visibility and prediction stability under variable lighting and etching conditions. Expanding the dataset to include a broader range of imaging environments and alloy variants will further enable model validation and stress testing.
Beyond preprocessing, ensemble modelling offers a promising approach to improving predictive accuracy and reliability. Combining multiple independently trained CNNs or architectures through majority voting or weighted averaging can reduce model variance and enhance robustness.26–28 Our current averaged 200×/500× CNN already demonstrates this concept, achieving improved R2 and RMSE performance (Table 3). Future implementations could expand this into a complete deep-ensemble framework or employ multimodal data fusion – integrating 2-D optical images with limited 3-D or SEM/TEM datasets – to better capture microstructural variability and β-phase distributions.
These combined improvements will support reliable deployment of the model in real-world Navy inspection environments, where rapid, field-ready image analysis tools are required to assess sensitisation risk. The Python code and AA5456 micrograph dataset, including associated ASTM G67 DoS measurements, will be made available upon reasonable request from the authors, subject to prior approval from the U.S. Navy.
Finally, as future work, we propose targeted validation of the CNN using a subset of specimens characterised by higher-resolution (SEM/TEM) or 3-D methods to (a) identify which 2-D features most strongly correlate with 3-D precipitate morphology, and (b) refine the model (or develop multimodal models) that fuse 2-D optical and 3-D/SEM data for improved accuracy and physical interpretability.
Footnotes
Acknowledgements
The authors would like to acknowledge the support and resources provided by the U.S. Navy, Naval Surface Warfare Center, Carderock Division, and Southern University and A&M College (SUBR). Their guidance, technical assistance and institutional support were instrumental in completing this work.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.