Integrated brain tumor segmentation and MGMT promoter methylation status classification from multimodal MRI data using deep learning

Segmentation phase

This section discusses the pre-processing steps applied to the BraTS2021 dataset followed by a detailed description of U-Net architecture and 3D ResU-Net. This section is concluded with the loss function applied on 3D ResU-Net.

Preprocessing

The challenge organizers have already put the dataset (BraTS2021) through several different pre-processing steps before making it available to the public. ³³ Each image has already been skull-stripped, co-registered, aligned in a single space, and has the same resolution of 240 × 240 × 155 for each modality. These numbers represent the image volume's dimensions along three axes: width (240 pixels), height (240 pixels), and the number of z-axis slices (155 slices). Each slice in the MRI volume represents a two-dimensional image of the brain captured at a particular depth or position. The width and height dimensions determine the spatial resolution of each 2D image slice, whereas the z-axis indicates the number of stacked segments that comprise the 3D volume.

Pre-processing is required before passing data into the model. The MRI scans are resized from 240 × 240 × 155 to 128 × 128 × 128. Resizing the MRI images to 128 × 128 × 128 does not significantly change the performance of the models. Resizing allows for more efficient processing and lowers the computational complexity of the models. The decision of 128 × 128 × 128 as the new size was made carefully to ensure that no important information or infected regions were discarded during the resizing process. By using suitable interpolation methods during resizing, the spatial information and important features within the images were retained. This ensures that the resized images still provide useful information for tasks such as tumor detection or classification, without compromising the accuracy or performance of the models and this makes the model less computationally expensive. The images were stacked together to maximize the value of the combined data from all four modalities. This downsizing technique meant that the information included in the four sequences (T1, T1ce, T2, and FLAIR) could be used properly. During the training phase, the model was fed training examples with dimensions of 128 × 128 × 128 × 4, where the fourth dimension represents the stacked of four modalities. This method allowed the model to access and process the integrated information from all four sequences simultaneously, optimizing available data to increase performance.

Network architecture for segmentation

The U-Net architecture is often used for jobs like image segmentation, where the goal is to segment the image into different categories. There is an encoder path and a decoding path in the U-Net. The encoder path grabs high-level features and makes the input image less detailed in space. It usually has several convolutional layers, which are followed by max-pooling processes. The convolutional layers take increasingly abstract features from the input image, while the max-pooling operations reduce the spatial dimensions, allowing the network to understand a wider context. The U-Net's decoding path does the opposite of what the encoder path does. ³⁵ It takes the features from the encoder path and slowly makes them bigger until they are the same size as the original image. This up-sampling process helps make a thick segmentation map pixel by pixel. U-Net has been demonstrated to be useful for various segmentation tasks; nevertheless, it has a few drawbacks. During the up-sampling process that takes place in the decoder path, there is a possibility that some of the spatial information will be lost. ³⁶ The network may have difficulty correctly recreating the fine details and borders of the segmented regions as the spatial resolution increases. Because of this, the segmentation results may be hazy or imprecise. In deep neural networks, the vanishing gradient problem happens when the gradients transmitted backward during training grow smaller as they go through several layers. This issue is most noticeable within the encoder part of the U-Net or other systems with comparable functionality. The network's capacity to learn meaningful and discriminative features from the input data decreases as the gradients decrease. This is because the network has less information to learn from. As a result, the segmentation performance is hampered due to this, particularly when capturing the fine details and borders of objects in volumetric data. Both segmentation performance and the vanishing gradient problem needed to be resolved. Hence the 3D ResU-Net architecture was developed. ³⁷

To effectively segment the volumetric data, the 3D ResU-Net architecture combines the concepts of 3D convolutions, residual blocks, skip connections and an encoder-decoder structure. By combining these components, the architecture is customized for the segmentation task. The encoder path is essential for extracting high-level features from the volumetric data input. It accomplishes this by employing 3D convolutional layers to extract pertinent data while simultaneously reducing the spatial dimensions to capture global context. However, it is important to note that the downsampling process in the encoder path may introduce the issue of vanishing gradients. Which is something that should be taken into consideration. To circumvent the issue of vanishing gradients, the architecture of the 3D ResU-Net has residual blocks included in it. The network can learn residual mappings because residual blocks introduce skip connections that circumvent certain convolutional layers. This allows the network to learn residual mappings. The network can collect and preserve significant features across multiple layers more efficiently when gradients are propagated over these skip connections. This makes learning more representative features easier for the network. ²² The 3D ResU-Net addresses the problem of the loss of spatial information that occurs during the up-sampling process by incorporating skip connections. These skip connections directly connect the corresponding encoder and decoder levels. This enables the decoder path to access and use low-level and high-resolution features in earlier layers. The network can reconstruct the fine details and borders of the segmentation maps thanks to the fusion of information at multiple resolutions. The decoder path of the 3D ResU-Net gradually upsamples the feature maps by utilizing either interpolation techniques or 3D transpose convolutions. Upsampled feature maps are concatenated with associated skip connections to facilitate the integration of high-level and low-level features. The objective of the decoder path is to recreate the initial volumetric data supplied by refining the features and providing accurate segmentation maps. A Softmax activation function is used in the third and final layer of the 3D ResU-Net. It produces a dense prediction volume, where multiple segmentation categories are assigned to each voxel individually. The segmentation map is represented by the output volume, which indicates the presence of distinct structures or regions inside the input volumetric data and the boundaries between them. The 3D ResU-Net model efficiently tackles the vanishing gradient problem and enables accurate and detailed segmentation of volumetric data for medical imaging and other applications by integrating 3D convolutions, residual blocks, skip connections, and the encoder–decoder structure. The 3D ResU-Net architecture is shown in Figure 3.

OPEN IN VIEWER

Figure 3.

3D residual U-net architecture. ³⁷

The 3D ResU-Net architecture is a well-established variant of the U-Net framework specifically designed to process three-dimensional (3D) data in medical image segmentation tasks. ³⁸ It employs an encoding-decoding structure consisting of a contracting path (encoder) and an expanding path (decoder) that are interconnected through skip connections. The contracting path comprises five levels, denoting the depth of the network, and operates on a voxel grid with dimensions of 128 × 128 × 128 × 4, incorporating four stacked modalities, namely T1, T2, T1ce, and Flair. At each level of the encoder, residual blocks (green blocks) are employed to extract relevant features, followed by 3D max pooling with a pool size of (2, 2, 2) and a stride of (2, 2, 2) to reduce the spatial dimensions. To prevent overfitting, dropout regularization is implemented at each step. The residual blocks utilize an identity mapping approach, which involves a convolutional layer and batch normalization for the first two components, while the third component does not employ ReLU activation as shown in Figure 4. By integrating the input with the shortcut link using the ReLU function, the output of the third component is obtained, allowing the network to capture residual mappings and extract hierarchical and informative features effectively.

OPEN IN VIEWER

Figure 4.

Shortcut identity connection of residual block. ¹⁸

On the other hand, lower levels show fewer features but have better spatial precision. ³⁷ The encoder path has a function called “downsampling,” which shrinks the size of the image and then a “bottleneck” layer. In the bottleneck and decoder, segmentation masks are predicted using plain convolutional blocks (purple blocks). Using up-sampling or the Conv3D Transpose function, the decoder path tries to get the image back to its original shape. The up-sampling path aims to get accurate segmentation by putting together data from the down-sampling path that matches. Skip connections are used to keep information from the contracting path in context. Using conv3D transpose layers, the up-sampling path and the matching down-sampling path are joined to make the expanding path look like the original U-Net model. The next step is using convolutional blocks with a dropout regularizer of 0.1. In the expanding path, each convolution block comprises two conv3D layers, batch normalization, and the ReLU activation function. The final segmentation is done with a (1 × 1×1) convolution layer and a Softmax activation function, which gives each target class a probability. The model's output from the decoder path is a 128 × 128 × 128 × 4 tensor that shows the segmentation masks expected for the input image i.e., background, WT, TC, and ET are the four groups.

Loss function

Selecting a loss function is crucial to a DL model's efficacy. It specifies the objective the model attempts to optimize during training. The loss function quantifies the model's performance and instructs it to modify its parameters to minimize error. The model learns to make more precise predictions by minimizing loss. SDL is commonly used in medical image segmentation problems to overcome the issue of class imbalance. This research uses SDL in the segmentation phase. ³⁹ The advantages of SDL include robustness to class imbalance, spatial awareness, smooth optimization, and differentiability. SDL can be calculated in equation (1).

Classification phase

This section discusses the preprocessing steps applied to the result of the segmentation phase followed by a detailed description of 3D ResNet10 architecture. This section is concluded with the loss function applied on 3D ResU-Net.

Preprocessing

In the classification phase, the resultant 3D multimodality segmented voxel from the segmentation phase of the proposed pipeline serves as the proposed ResNet10 classifier's input. Only 536 patients’ data in BraTS2021 datasets contain both segmentation labels and MGMT labels. ¹⁰ The resultant 3D multimodality segmented voxel is divided into two classes: methylated and unmethylated, provided by BraTS2021. When passing the segmented voxels to the classifier, their dimensions are 128 × 128 × 128 × 4, the same shape used in the segmentation phase.

Network architecture for classification

The 3D ResNet10 model was selected for the classification phase due to its efficiency and effectiveness in handling volumetric data while mitigating overfitting with our smaller dataset. Unlike deeper architectures such as ResNet34, ResNet50, VGG16, and VGG19, ResNet10 is computationally more efficient while still capturing complex features from 3D medical images. Although ResNet50 improves upon ResNet34 by introducing bottleneck blocks for enhanced accuracy, ⁴⁰ we opted for ResNet10 to prevent overfitting given our dataset's size. The model processes 3D segmented tumor voxels obtained from the segmentation phase, with input dimensions 16 × 4 × 128 × 128 × 128 (batch size 16, four imaging channels, and spatial resolution of 128³). ⁴¹ The network starts with a 3D convolutional layer, which applies spatial filtering to extract low-level features, followed by batch normalization and ReLU activation to introduce non-linearity and enhance feature learning. The Basic Block, a key component of ResNet10, consists of two 3D convolutional layers, batch normalization, and ReLU activation, with residual connections allowing the model to learn residual functions, improving gradient flow and making deeper networks easier to train. ⁴² The network consists of 10 residual blocks, stacked in a hierarchical structure. After the residual layers, an adaptive pooling layer (1 × 1 × 1) reduces spatial dimensions while preserving essential features. Finally, the extracted features pass through a fully connected layer (400 × 1), which helps learn complex patterns and interdependencies, followed by a sigmoid activation function, mapping the features to classification probabilities—predicting MGMT promoter status as methylated or unmethylated. The 3D ResNet10 architecture is illustrated in Figure 5.

OPEN IN VIEWER

Figure 5.

3D ResNet10 architecture.

Loss function

The classification phase model uses loss function BCE with logits. ⁴³ The equation for the BCE with logits is given in equation (2).

Results of phase 1 (segmentation)

In phase 1 a 3D ResU-Net was trained and validated using the BraTS2021 training dataset because the validation data's source data is not readily accessible. As a result, the segmentation model is trained using 80% data for training and 20% for validation. The segmentation model creates a 3D volume of the segmentation mask, composed of WT, ET, and TC in the tumorous regions, and uses it to segment those MRIs with the MGMT label provided by the BraTS2021 challenge. The segmentation results of the MRI image with the MGMT value in the axial plane and random slice no are shown in Figure 6 The visual results representation describes that the results are quite close to the ground truth values. In Table 5, details about the Dice coefficient score of WT, TC, and ET of training and validation achieved by the pipeline phase 1 segmentation model are given. A modified U-Net model incorporating the residual network has been developed to handle the problem of BTS.

OPEN IN VIEWER

Figure 6.

Visualized segmentation results having MGMT label.

OPEN IN VIEWER

Table 5.

Quantitative results of training and validation of the segmentation phase.

Results Segmentation	Specificity	Sensitivity	Dice Score	Tumor Classes Dice Score
				WT	TC	ET
Training	0.98	0.99	0.89	0.90	0.90	0.85
Validation	0.97	0.99	0.83	0.84	0.81	0.80

The initial attempt was made to substitute two levels of the U-Net model's convolutional layers with residual convolutional blocks during the trial of the proposed design. However, the results did not improve much due to this strategy. After that, residual blocks were used at all stages of the encoder path, significantly improving the outcomes. The model was shown to have an overfitting tendency, and after a few epochs, its loss began to increase. The goal of the bottleneck layer is to ensure that the compressed view of the model's bottleneck layer only contains the information necessary for reconstructing the input or segmentation map. Therefore, only the first four levels of the suggested model's final experiments utilized residual blocks, leaving the other levels unchanged from the U-Net model with batch normalization and ReLU activation function. Within the encoder portion of the U-Net, the residual convolutional blocks used adaptive skip connections, which helped to keep the low-level feature information (such as the boundary or edges of the tumorous zone) with high-level features. In this instance, skip connections in the encoder improved the outcomes.

Figure 6 presents segmentation results on MRI scans from the BraTS 2021 dataset, displayed in the 2D axial plane. The selected cases include only MRIs associated with MGMT methylation status. Each row illustrates the model's segmentation output, highlighting tumor subregions across different MGMT classifications.

The training and validation dice coefficient score per epoch is shown in Figure 7. The training and validation loss is shown in Figure 8, training epoch-wise dice score of each tumor class is shown in Figure 9. Additionally, Figure 10 shows the box plots of segmentation accuracy for the three subregions WT, TC, and ET according to the dice score metric. When compared to TC and ET dice scores, it was found that the model performs well in segmenting WT class. With a higher standard deviation, ET can sometimes be significantly more challenging to segment. Subregions of brain tumors are indicated on the x-axis, and the Dice score value, which ranges from 0 to 1, is indicated on the y-axis. ⁴⁵

OPEN IN VIEWER

Figure 7.

Dice score of training and validation.

OPEN IN VIEWER

Figure 8.

Training and validation loss.

OPEN IN VIEWER

Figure 9.

Epoch-wise dice score of tumor classes.

OPEN IN VIEWER

Figure 10.

Boxplot for dice score of WT, TC, and ET.

Discussion of segmentation results

The segmentation phase of the proposed pipeline was evaluated using extensive quantitative analysis, with a particular focus on the impact of employing the 3D ResU-Net architecture. Table 5 displays the performance attained during both the training and validation phases. These quantitative metrics, which include accuracy, precision, recall, and other pertinent measurements, provide valuable insight into the efficacy of the proposed segmentation model. Utilizing the 3D ResU-Net architecture enables the segmentation phase to overcome obstacles such as the vanishing gradient problem and train the network at a deeper level. The residual connections in the architecture enhance the accuracy of segmentation by facilitating the flow of gradients and preserving earlier network information. The model's ability to reliably predict tumor regions is enhanced by the 3D ResU-Net’s capacity to leverage low-level features. The obtained results demonstrate the efficiency of the 3D ResU-Net in achieving high segmentation performance, validating its use in the segmentation phase. Integral to the overall pipeline, the segmentation model plays a crucial role in providing precise tumor region localization, thereby enhancing the accuracy and reliability of subsequent classification tasks. These results highlight the significance of the 3D ResU-Net architecture in enhancing segmentation results and furthering the field of brain tumor analysis.

Table 5 presents Segmentation results, including specificity, sensitivity, and Dice Score. The first column under “Dice Score” represents the average Dice Score calculated across the three tumor classes: whole tumor (WT), TC, and enhancing tumor (ET). These metrics provide insights into the performance of the segmentation model in accurately identifying tumor regions and differentiating them from non-tumor regions. Specificity measures how well the segmentation model can identify areas that are not tumors. A lower number of false positive results is indicated by a higher level of specificity. The training sample has a high level of accuracy in identifying non-tumor regions because the average specificity for all tumor classes is 0.98. Similar to the validation sample, the model's ability to precisely identify non-tumor regions is confirmed by the high average specificity of 0.97. The model's capacity to identify tumor regions is measured by its sensitivity. A lower number of false negatives indicates a higher sensitivity. In the preparation test, the responsiveness is reliably high for all cancer types, with a typical worth of 0.99. This suggests that the training dataset accurately identifies tumor regions for the model. In a similar vein, the validation sample's average sensitivity remains high at 0.99, indicating that the model is capable of accurately identifying tumor regions and generalizing to unseen data. The Dice score is a metric that is used a lot to measure how similar the predicted segmentation and the ground truth are. It takes into account both true and false positives. Dice scores of 0.90 for WT, 0.90 for TC, and 0.85 for ET can be found across all tumor types in the training dataset. These scores indicate that the actual tumor regions and the predicted segmentation overlap significantly. The Dice scores are slightly lower in the validation sample, with values of 0.84 for WT, 0.81 for TC, and 0.80 for ET. The segmentation model achieves high specificity, sensitivity, and Dice scores for all tumor classes in both the training and validation datasets, indicating that it performs well on unseen data. However, these scores still demonstrate a significant similarity between the predicted and ground truth segmentations. The model's ability to accurately distinguish tumor regions from non-tumor regions is demonstrated by these findings. Overall, the segmentation results show that both the training and test datasets show that the model has a high level of specificity and sensitivity. The Dice scores also show that the predicted segmentation and the real segmentation are pretty close, especially for the WT and TC tumor groups. But the success of the segmentation of the ET sub-region is not as good, as shown by slightly lower Dice scores.

The Dice coefficient scores for the training and validation stages of a segmentation model are shown on the graph. The x-axis shows the schedule or the number of iterations, and the y-axis shows the Dice coefficient scores. The Dice coefficient scores for the training phase run from 0.4065 to 0.8941. The graph shows that the scores get better over time, which means that the model is getting better at segmenting as it learns from the training data. There are some changes in the numbers, but in general, they tend to go up slowly. The Dice coefficient scores for the validation phase run from 0.2703 to 0.8519. Scores tend to go up, just like in the training phase, but the numbers are usually lower than they were in the training phase. The graph illustrates variations in scores, indicating that the model's performance differs across validation examples. When comparing Dice coefficient scores in the training and validation phases, it is expected that training scores are higher since the model is optimized using training data. However, the increasing trend in validation scores indicates improved generalization to unseen data. This aligns with previously discussed patterns of model learning, where performance improves over successive epochs.

The training and validation loss values decrease over time, reflecting improved model performance. As discussed earlier, lower loss values indicate better alignment between predictions and actual outcomes. The consistent reduction in both training and validation loss suggests that the model effectively learns from data while maintaining generalization.

The Dice coefficient scores for tumor subregions (WT, TC, and ET) follow a progressive upward trend, demonstrating improved segmentation accuracy over epochs. This pattern has been previously described, reinforcing the model’s ability to refine predictions over time. The ET subregion remains more challenging to segment compared to WT and TC, as noted earlier.

The box plot gives a condensed visualization of the dice score distribution for each BTS. This makes it possible to easily compare and identify any changes in the model's performance across different tumor segment.

Comparison with the related studies

BTS on the BraTS2021 dataset was performed in Phase 1 of this research utilizing the 3D ResU-Net architecture. The findings were then compared to other state-of-the-art methodologies utilized in comparable studies. The comparison was carried out by comparing the Dice scores given by each model for the BraTS 2020 and 2021, as indicated in the results in Table 6.

OPEN IN VIEWER

Table 6.

Segmentation results of the related studies.

Ref. Study	Architecture	Dataset	Dice Score
			TC	WT	ET
46	3D U-Net	BraTS 2020	0.79	0.84	0.68
47	Modified 3D U-Net	BraTS 2020	0.75	0.80	0.69
48	2D-Deep Residual U-Net	BraTS 2020	0.79	0.86	0.75
49	3D Encoder Decoder-based model	BraTS 2020	0.75	0.84	0.62
37	Deep Residual U-Net	BraTS 2020	0.83	0.86	0.80
22	3D Residual U-Net	BraTS 2021	0.85	0.91	0.81
24	Attention U-Net	BraTS 2021	0.81	0.87	0.79
23	Modified U-Net	BraTS 2021	0.76	0.87	0.73
25	U-Net 3D with Attention Block	BraTS 2021	0.80	0.89	0.78
Phase 1 Segmentation Model	3D ResU-Net	BraTS 2021	0 . 81	0 . 84	0 . 80

As can be seen in Table 6, the outcomes of Phase 1 achieved a Dice score of 0.80 for the ET, 0.84 for the WT, and 0.81 for the TC. These scores are comparable to those achieved by several state-of-the-art architectures, such as the Deep Residual U-Net and the Attention U-Net, which were utilized in experiments that are relevant to this one. The vanishing gradient problem is one that is frequently seen in deep neural networks. One solution to this issue is the 3D ResU-Net architecture, which combines the advantages of the U-Net architecture and residual connections. The U-Net architecture has shown performance in medical image segmentation tasks that are both encouraging and promising. The BraTS presents challenges due to variations in tumor size, shape, and appearance, as well as the presence of healthy brain tissue that can resemble tumors in appearance. Based on the findings of Phase 1, it appears that the 3D ResU-Net architecture has the potential to accurately segment brain tumors, attaining performance that is comparable to that of state-of-the-art methods employed in studies that are similar to the proposed study. Moving forward, the primary objective of the study in the future might be to enhance the performance of models by experimenting with a variety of preprocessing methods and hyperparameters, or by investigating the potential synergies that could be achieved by integrating the 3D ResU-Net with other architectural styles or methodological approaches. These initiatives are intended to significantly improve the accuracy and resilience of the pipeline that is used to segment brain tumors.

Model complexity

The Phase 1 model has 47.30 M parameters, and the 3D ResU-Net model's complexity suggests that it is capable of learning complicated correlations between input data and output labels. Still, its performance on both training and validation data must be monitored to ensure that it is not overfitting. The 3D ResU-Net model is quite complex. Our proposed model's execution time was 70.75 min per epoch, and it was trained for 40 epochs in roughly 47 h. The model testing duration is only 12 s, which is a good time for pipeline-based work in which the second phase model receives the outcome of the first phase model as an input. Model performance can be improved by using more power full GPU and more processing units of GPU to run more epochs on a reasonable batch size.

Results of phase 2 (classification of MGMT)

Phase 2 of the study built on the results of Phase 1 which introduced MGMT labels into the segmentation result to predict MGMT status. Only 536 patient's data consists of both segmentation labels and MGMT labels in BraTS2021 datasets. ¹⁰ In phase 2, a 3D ResNet10 model was trained and internally validated using the BraTS2021 training dataset, as the external validation dataset did not contain MGMT labels. As a result, the classification model is trained using 90% data for training and 10% for validation. The MGMT gene is a DNA repair gene commonly methylated in GBM and is related to better treatment response. The 3D segmentation information (voxels) from the first phase and the MGMT labels were used in this phase to train a predictive model to classify the tumor as MGMT methylation or unmethylated. The study used the 3D segmentation results from Phase 1 to create a DL network, 3D ResNet10, to predict MGMT status. The MGMT labels were obtained from the same BraTS dataset used in Phase 1, which included each patient's ground truth MGMT status. The DL model was trained to identify the tumor as MGMT methylated or unmethylated based on the 3D segmentation voxel and the matching MGMT label for each patient.

The model contains batch normalization and dropout layers to boost generalization and prevent overfitting. The 3D ResNet10 model was optimized during training using binary cross-entropy with logistics loss function and the Adam optimizer. The dataset was divided into training and validation sets, and the model was trained for numerous epochs before being stopped early based on the validation loss. After training, the model was tested on a separate test set to see how well it predicted the MGMT status of new patients.

Ablation study

The ablation study starts with MGMT classification using a brain MRI slice-wise approach. Figure 11 shows the training and validation ROC–AUC graph.

OPEN IN VIEWER

Figure 11.

Training and validation results of MRI slice-wise approach.

The validation AUC values graph demonstrates that the slice-wise approach for MGMT classification using a 2D ResNet model yields suboptimal results. The AUC values range between 0.45 and 0.56, indicating a relatively low accuracy of classification. These findings underscore the limitations of relying solely on a slice-by-slice approach and emphasize the necessity of incorporating all four modalities into MGMT classification. Each modality, including T1, T1ce, T2, and FLAIR contributes unique and valuable information about the underlying tissue characteristics and abnormalities when working with brain MRI data. Using all four modalities collectively, as opposed to contemplating individual slices, the model is able to capture the comprehensive context and complex relationships present across multiple slices and modalities. This integration permits a more comprehensive comprehension of the brain's structure and pathology, thereby enabling the model to make more accurate and reliable predictions. The poor performance indicated by the validation AUC values suggests that the slice-wise approach fails to exploit the abundant contextual information available across multiple modalities. This limitation may cause the model to lack crucial patterns and characteristics for accurate MGMT classification.

In the ablation study, a 3D ResNet10 model incorporating all four modalities from the BraTS2021 dataset was implemented, and significant improvements of ROC–AUC of 6% over the slice-by-slice approach were observed. The previous method, which evaluated each MRI slice separately, did not completely exploit the three-dimensional nature of the data and the spatial relationship between adjacent slices. By adopting a 3D model that incorporates all modalities, classification results have been significantly improved. Utilizing the 3D ResNet10 model enabled more thorough analysis of the complete tumor volume by capturing the spatial context and interslice dependencies. As a consequence, the model was better able to distinguish between tumor regions and non-tumor regions, resulting in improved MGMT classification accuracy. Nonetheless, there is still potential for improvement. Noting that the MGMT gene status resides within the tumorous region is crucial for effective treatment planning. A pipeline of segmentation and classification is necessary for improved outcomes. The “segmentation” step would accurately define the area of the tumor so that the “classification” step could focus on the areas related to the MGMT gene. The implementation of a pipeline consisting of segmentation and classification yielded an additional 4% improvement in ROC–AUC results. This pipeline strategy relies on the accurate segmentation of the tumor region, allowing the classification stage to precisely target and analyze the relevant regions associated with the MGMT gene. By integrating this pipeline, it is anticipated that the classification performance will become even more robust, leading to more precise predictions of the MGMT gene status in brain tumor analysis.

Overall, the proposed pipeline had a higher ROC–AUC than the 3D classification model, which took in input from all four modalities of whole brain MRI.

Discussion of classification results

The classification phase of the proposed pipeline was analyzed quantitatively, taking into account the impact of employing a segmentation and classification pipeline. Table 7 displays the comprehensive results obtained during the training and validation phases, including metrics such as precision, ROC–AUC, recall, and others. In the context of the segmentation and classification pipeline, these metrics provide invaluable insight into the performance of the classification model. By incorporating segmentation results into the classification process, the proposed pipeline takes advantage of the precise localization of tumor regions, which contributes to an increase in classification precision. The segmentation phase provides crucial information regarding the sub-regions of the tumor, allowing the classification model to make more accurate predictions regarding the MGMT promoter status. This integration of segmentation and classification improves the overall accuracy of the pipeline's MGMT status prediction and provides a comprehensive approach to brain tumor analysis. By integrating segmentation into the classification process, the proposed pipeline benefits from precise tumor localization, enhancing MGMT classification accuracy. The segmentation phase provides crucial insights into tumor sub-regions, allowing for more informed classification decisions. As a result, the pipeline effectively utilizes segmented tumor regions, leading to improved MGMT promoter status prediction and contributing to advancements in brain tumor analysis.

OPEN IN VIEWER

Table 7.

Validation results of the ablation study of two classification approaches whole brain and proposed pipeline.

Approach	Model	Precision	Recall	F1-Score	Accuracy	ROC–AUC
Multimodality Whole Brain Data as Input	3D ResNet 10	0.61	0.60	0.59	0.61	0.62
Proposed Pipeline (Multi- Modality Segmentation Results of Phase 1 as Input)	3D ResNet 10	0.74	0.64	0.60	0.65	0.66

Using the segmentation results from phase 1 proves to be a better strategy for MGMT Promoter status categorization than using the entire brain. The segmentation result as input to the classifier produces higher precision (0.74) and accuracy (0.65) and ROC–AUC (0.66) values, showing a superior capacity to detect positive instances and accurately categorize cases correctly. Although recall and F1-Score are slightly higher than in the whole brain technique, the improved precision accuracy and ROC–AUC make segmentation-based Implementation a more effective strategy for this classification job. This emphasizes leveraging information from the initial segmentation step to improve MGMT Promoter status categorization accuracy. In, ¹⁰ the reported results are on validation split data (training and validation). This study follows that pattern because the public test set has no ground truth labels to validate results. In Figure 12, epoch wise model's ROC–AUC is Shown. Early stopping was implied at the 40^th epoch to avoid overfitting and to have a comparable result of all evaluation measures. The confusion matrix of the phase 2 validation set is shown in Figure 12, which explains that the model performed well on class 1(MGMT status methylation) predication and faced a bit of difficulty on class 0 (MGMT status unmethylation predication).

OPEN IN VIEWER

Figure 12.

Training and validation ROC–AUC for phase 2 MGMT methylation prediction.

Figure 12 shows the ROC–AUC scores for the training and validation sets across different epochs. In tasks that require binary classification, the ROC–AUC is a performance metric that is commonly used. It plots the TPR against the fake positive rate to see how well the model can tell the difference between the positive and negative classes. In this case, ROC–AUC scores run to 0.92 for the training set and from 0.66 for the validation set on the 40^th epoch. Since the amount of data fed into the network was small there was a chance of the model getting overfit on train data. Early stopping was done at the 40^th phase to avoid overfitting and make sure that all evaluation measures were the same. This helps choose the model that works well outside of its training data and keeps the training data from being given too much weight. A higher ROC–AUC number means that the model works better, and values closer to 1 mean that the model can tell the difference between the classes very well. The line shows that the model works pretty well on both the training set and the validation set. The ROC–AUC scores for the training set run to 0.92, which shows that the model is able to tell the difference between the positive and negative classes in the training data. This shows that the model has found patterns and connections in the training set that make sense. Also, the ROC–AUC scores for the validation set run to 0.66. Even though the scores are lower in the validation set than in the training set, it is important to remember that the validation set is usually made up of new data that the model has not been trained on directly. Even so, the model shows that it can still tell the difference between the classes in the validation set. Based on Figure 12, we can say that the model works well overall because it gets high ROC–AUC scores on both the training and validation sets. The results show that the model has learned generalizable patterns and can also classify well from data it has not seen before. But it is important to do more research and look at other performance metrics to get a full image of how well the model works generally and where it might have problems.

Figure 13 shows the confusion matrix of results achieved by 3D ResNet10 which shows the model does a good job of identifying class 1 (MGMT methylation status). It knows that 26 cases belong to class 1, which makes them TPs. This shows that the model is good at figuring out when MGMT methylation is present. But the model has trouble figuring out what will happen in class 0 (MGMT state unmethylation). It predicts 18 cases belong to class 0 when they really belong to class 1. These are called false negatives (FN). It also says that 1 instance belongs to class 1 when it belongs to class 0. This is called a false positive (FP). The misclassification of class 0 that was seen in the validation confusion matrix could be caused by a class imbalance, a well-documented issue in MGMT classification studies. ⁹ Previous works on MGMT methylation classification have reported similar challenges, where the underrepresentation of the unmethylated class (class 0) negatively impacts model performance, leading to a higher rate of false negatives. The term “class imbalance” refers to a situation in which the number of cases in different classes is disproportionately distributed, with one class having significantly fewer examples than the other. In this case, class 0 (MGMT status not methylated) seems to have fewer instances in the dataset. This suggests that the model may not have had enough representative samples for this class to learn and apply well. So, the model might have trouble correctly classifying examples from class 0. This could lead to a higher number of false negatives (FN) and a lower average performance for this class. Overall, the model does a good job of finding cases with MGMT methylation, but it has trouble figuring out how to group cases without MGMT methylation. This demonstrates that the model may require additional development to distinguish these two groups. A more comprehensive picture of how well the model generally works and where it could be improved can be provided by conducting additional analysis and evaluation of other performance metrics. During the training process, it may be advantageous to utilize a larger sample size of MGMT label data in order to enhance the outcomes and eliminate the misclassification that was observed in class 0 (MGMT status unmethylation) of the validation confusion matrix. By consolidating a bigger and more different dataset containing a significant number of models from both class 0 and class 1, the model would have a more prominent possibility of learning the distinctive examples and qualities related to each class. The model's comprehension and classification precision for instances belonging to this class would improve as a result of this increased exposure to data from class 0. Besides, a bigger amount of MGMT mark information can give a more adjusted portrayal of the classes, in this way relieving the impact of class irregularity. This would guarantee that the model receives adequate training on both class 0 and class 1, leading to predictions that are more reliable and accurate. It is anticipated that the model's ability to classify class 0 (MGMT status unmethylation) will significantly improve with the use of a diverse and comprehensive MGMT label dataset. This will lead to predictions that are more reliable and accurate for both classes.

OPEN IN VIEWER

Figure 13.

Validation set confusion matrix.

Comparison with state-of-the-art studies

Phase 2 of the pipeline performs the classification of MGMT promoter status using a 3D ResNet10, which takes the result of phase 1 as input. The comparison of MGMT promoter status classification based on the input data is shown in Table 8. The proposed study adopts a comprehensive strategy by incorporating multiple modalities, including T1, T1w, T2, and FLAIR MRI images. By integrating data from different imaging modalities, the model gains a more diverse and informative representation of the brain, leading to improved classification performance. Previous studies have typically focused on either segmentation or classification, with some researchers using only specific modalities. However, relying on a limited set of modalities may not provide a sufficiently comprehensive view of MGMT status. ⁵⁰ Since each modality captures unique information about the tumor, ²⁹ omitting any of them may result in critical features being overlooked. Therefore, utilizing all four modalities ensures a more robust and informative prediction of MGMT status. In this research, we extracted tumor regions (voxels) from whole-brain MRIs and used all four modalities provided in the BraTS2021 dataset. 3D MRI scans, incorporating multiple imaging sequences, offer a detailed view of the tumor and its surrounding tissue, contributing to a deeper understanding of the pathology. This multi-modal approach helps extract richer features for MGMT promoter status classification. ⁶ Most previous studies have used whole-brain MRI for MGMT classification, which may introduce irrelevant information into the learning process. In contrast, our approach specifically focuses on tumor regions, ensuring that classification is based on features directly associated with MGMT promoter status. Given that MGMT methylation is present in tumor tissue,^6,10 segmenting the tumor before classification enhances the model's ability to extract meaningful features. By integrating segmentation and classification into a unified framework, the proposed method allows the segmentation output to serve as a direct input for classification, ensuring a more targeted and effective analysis. Additionally, previous MGMT classification studies have reported challenges related to class imbalance, particularly in datasets where the unmethylated class (class 0) is underrepresented.^9,29 Addressing this imbalance is crucial, as models trained on imbalanced datasets may exhibit bias toward the dominant class. We acknowledge this limitation and suggest that future work explore techniques such as data augmentation, oversampling, or reweighting strategies to mitigate class imbalance and enhance classification accuracy.

OPEN IN VIEWER

Table 8.

Results of related studies of BraTS2021 MGMT classification.

Ref No.	BraTS2021		Input Data				Methodology			Results
	Task 1	Task 2	T1	T1w	T2	FLAIR	Ensemble Results of all four Modalities	Segmentation	Classification	ROC–AUC
7		✓	✓	✓		✓			✓	0.60
30		✓	✓	✓	✓	✓			✓	0.68
10	✓	✓			✓			✓	✓	0.61
31		✓	✓	✓	✓	✓			✓	0.63
5		✓				✓			✓	0.61
9		✓					✓		✓	0.57
29		✓					✓		✓	0.69
50					✓				✓	0.60
Proposed	✓	✓	✓	✓	✓	✓	–	✓	✓	0.66

The proposed pipeline achieved a ROC–AUC value of 0.66 on the validation set, demonstrating its effectiveness in accurately classifying MGMT promoter status.

The performance achieved by the proposed study is comparable with other existing studies in the field, as demonstrated in Table 8. It is important to note that this study utilized data from 536 patients, which is a slightly smaller dataset compared to studies involving 585 cases. However, this sample size was selected because it contained both segmentation data and MGMT labels. Despite the smaller dataset, our pipeline, incorporating all available modalities, achieved superior performance compared to certain other studies. In the future, employing a larger labeled dataset could further improve these results. The suggested approach ensures that an appropriate and relevant subset of data is selected for this study's specific objectives.

Model complexity

The proposed 3D ResNet10 model exhibits a relatively simple model complexity compared to other architectures. With fewer parameters than complex models like ResNet50 or VGG16, the 3D ResNet10 model strikes a balance between being lightweight and capable of learning the correlations between input data and output labels. It incorporates 14.3 million parameters and performs at 8.1 Giga-Floating Point Operations Per Second (G FLOPs). This reduced intricacy was necessary due to the limited availability of training instances (536) with MGMT promoter status in the BraTS 2021 dataset. Furthermore, the model demonstrates efficient training with an average execution time of 5 minutes per epoch and a total training time of approximately 8 hours. Additionally, the model's assessment duration per instance is only 5 seconds, making it suitable for real-time deployment in clinical systems. Table 9 provides a comparison of the model complexity and computational cost among the 3D ResNet50, 3D VGG16, and 3D ResNet10 3D CNN-based models.

OPEN IN VIEWER

Table 9.

Comparison of parameters and FLOPs of CNN-based models.

Model	Parameters	FLOPs
3D ResNet50	40 million	150G FLOPs
3D VGG16	140 million	250G FLOPs
3D ResNet10	14.3 million	8.1G FLOPs

The 3D ResNet50 model has about 40 million parameters and needs about 150 GFLOPs of processing power to run. The 3D VGG16 model, on the other hand, has about 140 million parameters and needs about 250 GFLOPs. With about 14.3 million parameters and about 8.1 GFLOPs, the proposed 3D ResNet10 model stands out as the simplest and lightest. This demonstrates the varying levels of complexity and computational demands among the models. Notably, the 3D ResNet10 model stands out as a more efficient option, consuming less computing power compared to the larger and more resource-intensive 3D ResNet50 and 3D VGG16 models.