Performance of deep learning in classifying malignant primary and metastatic brain tumors using different MRI sequences: A medical analysis study

Abstract

BACKGROUND:

Malignant Primary Brain Tumor (MPBT) and Metastatic Brain Tumor (MBT) are the most common types of brain tumors, which require different management approaches. Magnetic Resonance Imaging (MRI) is the most frequently used modality for assessing the presence of these tumors. The utilization of Deep Learning (DL) is expected to assist clinicians in classifying MPBT and MBT more effectively.

OBJECTIVE:

This study aims to examine the influence of MRI sequences on the classification performance of DL techniques for distinguishing between MPBT and MBT and analyze the results from a medical perspective.

METHODS:

Total 1,360 images performed from 4 different MRI sequences were collected and preprocessed. VGG19 and ResNet101 models were trained and evaluated using consistent parameters. The performance of the models was assessed using accuracy, sensitivity, and other precision metrics based on a confusion matrix analysis.

RESULTS:

The ResNet101 model achieves the highest accuracy of 83% for MPBT classification, correctly identifying 90 out of 102 images. The VGG19 model achieves an accuracy of 81% for MBT classification, accurately classifying 86 out of 102 images. T2 sequence shows the highest sensitivity for MPBT, while T1C and T1 sequences exhibit the highest sensitivity for MBT.

CONCLUSIONS:

DL models, particularly ResNet101 and VGG19, demonstrate promising performance in classifying MPBT and MBT based on MRI images. The choice of MRI sequence can impact the sensitivity of tumor detection. These findings contribute to the advancement of DL-based brain tumor classification and its potential in improving patient outcomes and healthcare efficiency.

Keywords

Deep learning brain tumor classification MRI Sequences malignant primary brain tumor metastatic brain tumor

1 Introduction

Brain tumors, also known as intracranial tumors, are considered mass abnormalities in the cranium tissues once occurs in individuals between the ages of 55 and 64 [1]. Brain tumors occur 1.58 times more often in males than females [2]. Additionally, these tumors dominate Central Nervous System (CNS) with 85–90% [3].

Based on their origin, brain tumors are divided into primary tumors and secondary or metastatic tumors. Primary brain tumors are tumors whose growth originates from brain cells, meninges, nerves, and glands [4]. Gliomas, one type of malignant primary brain tumor (MPBT), are the most frequent cases with an incidence of around 51% compared to other types of primary brain tumors [5]. Meanwhile, secondary brain tumors, also known as metastatic brain tumors (MBT), originate from malignant tumors in other tissues that spread through blood vessels and enter the CNS [4, 6]. The number of MBT cases is four times higher than MPBT. These tumors most commonly originate from primary tumors in the lung (50%), breast tumors (15–25%), melanoma (5–20%), kidney, and colorectal tumors [3].

Magnetic Resonance Imaging (MRI) is used to detect the presence of brain tumors. MRI is a radiology modality that plays a crucial role in providing imaging to help diagnose brain tumors [7]. MRI can provide clear imaging between soft tissue and hard tissue in the brain [8]. Although various advanced imaging techniques have been developed in recent decades, conventional MRI remains the most used imaging method [9]. MRI of MPBT and MBT are similar but they have different characteristics when properly observed. MPBT has an irregular shape and edges with heterogeneous enhancement patterns, while MBT has a round shape, regular edges, and a homogeneous enhancement pattern. According to Jung et al. (2021) and Rasuli & Gaillard (2016), the number, location, and morphology of tumors are other differences in the characteristics of MPBT and MBT [10, 11].

Deep Learning (DL) is a branch of Artificial Intelligence (AI) that can mimic the diagnostic capabilities of doctors. This ability can be used to improve the quality and speed up healthcare services [12]. DL has tremendous potential in the development of radiology because almost all primary data and output are digital files. Moreover, it can be used in various aspects such as classification, segmentation, detection, and others [13].

Accurately differentiating between MPBT and MBT using DL is crucial for improving patient outcomes and optimizing healthcare services [14]. Misdiagnosis can lead to delays in treatment, worsening of the patient’s condition, and even death [15]. Moreover, the management and therapy approach for MPBT and MBT are very different, and accurate classification can help healthcare providers tailor treatment plans to each patient’s specific needs [16]. DL’s ability to classify brain tumors using various MRI sequences quickly and accurately can save valuable time in the diagnostic process, leading to earlier treatment and better outcomes for patients [17, 18]. Additionally, DL can reduce the workload of healthcare providers and improve the overall efficiency of the healthcare system [19, 20].

Two studies were conducted to classify brain tumor MRI images using various Convolutional Neural Network (CNN) architectures. The first study by Qodri in 2021 used six architectures, including ResNet50 and VGG19, and found that ResNet50 and VGG19 achieved the highest accuracy of 99% and 97%, respectively [21]. However, ResNet101 was not included in this study, even though a previous study by Setiawan in 2019 showed that ResNet101 had slightly higher accuracy in classifying fundus images [22]. The second study by Cinar in 2022 classified 3,000 brain MRI images using six architectures, including ResNet101 and VGG19, and found that ResNet101 and VGG19 had the highest accuracy of 98.6% and 97.2%, respectively, while other architectures achieved significant accuracy, ranging from 89.5% to 94.3% [23]. Thus, both ResNet101 and VGG19 were used in this study to compare their performance in classifying MPBT and MBT MRI images in more detail.

Several parameters influence the accuracy of brain MRI classification, including the number of epoch scenarios. Chelghoum (2020) conducted a study using CNN to classify 3,064 brain MRI images of glioma, meningioma, and pituitary tumor. Three epoch scenarios were tested: 25, 50, and 90. The accuracy improved in the epoch 50 scenario compared to epoch 25, as seen in VGG19 (97.97% to 98.55%) and ResNet101 (96.67% to 96.83%). However, accuracy decreased in the epoch 90 scenario (VGG19:98.47%, ResNet101:95.99%). The optimal epoch scenario was found to be 50, but the exact epoch at which overfitting occurs is still unknown [22, 24]. Therefore, this study utilizes epoch 60 to examine the performance and assess potential improvements. Additionally, the epoch 90 scenario, known for overfitting, and the epoch 30 scenario will be compared proportionally to determine the significance of the accuracy improvement at epoch 60.

One of the advantages of MRI compared to other imaging modalities is the presence of sequences. An MRI sequence consists of a series of radiofrequency pulses and gradients that produce a set of images with a specific appearance. Each sequence generates images with different characteristics [25]. The sequences commonly used in brain tumor examinations include T1-weighted images (T1), T2-weighted images (T2), T1+ contrast (T1C), and Fluid-Attenuated Inversion Recovery (FLAIR) [26]. However, previous studies have been limited to using only one or two types of sequences, and some studies did not specify the sequence used for their datasets [23, 27]. The research by Chelghoum (2020) and Fayaz (2021) only used T1-contrast (T1C) and T2 sequences, respectively [24, 27]. Meanwhile, Cinar (2022) failed to mention a particular sequence for the analysis [23]. In, other hand, Additionally, there is a lack of research discussing the classification results of deep learning on MPBT and MBT from a medical perspective. Therefore, the objective of this study is to examine the impact of MRI sequences, including T1, T2, T1C, and FLAIR, on the performance of VGG19 and ResNet101 in classifying MPBT and MBT, as well as to analyze them from a medical perspective.

2 Method

2.1 Dataset

In this research, a total of 49 MPBT and 66 MBT patients from the Radiology Installation of Dr. Moewardi Surakarta Hospital between January 2020 to February 2022 were selected as participants. Data was collected as a 3D image in Digital Imaging and Communication in Medicine (DICOM) format using a machine called GE Signa HDxt 1.5T MRI Scanner. The first step in data collection is the selection of a layer of sequences that contain tumor images through RadiAnt DICOM Viewer Version 2021.2.2 software. In this research, the sequences used were T1, T2, T1C, and FLAIR. The dataset used in this study is available at this link https://bit.ly/braintumorsdataset Figure 1 shows examples of tumor images in each sequence.

Fig. 1

MRI images corresponding to sequences in (a) MPBT and (b) MBT.

Each data is labeled with its tumor type, number, and sequence. The tumor type consists of “MPBT” or “MBT”, while the sequence comprises “T1”, “T2”, “T1C”, and “FLAIR”. Furthermore, the data was saved in JPG format and a total of 1360 images were collected. The dataset was divided with a configuration of 70% training, 15% validation, and 15% testing with 952, 204, and 204 images, respectively. Table 1 shows a more complete dataset division.

Table 1

Number of images in the datasets

Class Name	Train (70%)	Validation (15%)	Test (15%)	Class
MPBT	477	102	102	680
MBT	477	102	102	680
Total	954	204	204	1360

2.2 Data preprocessing

Data preprocessing is required to prepare the dataset for deep learning training. First, the selected data are brain images that are not cropped using data cleansing and data wrangling. Then, the image size is standardized to a 2D image with a size of 224×224 grayscale. Furthermore, the pixel values of the images are normalized to a range of 0 to 1.

2.3 Data augmentation

Deep learning requires a large amount of data to obtain reliable results. However, there may not be enough data, especially on medical problems. The problem is obtaining and annotating data, which is very expensive and time-consuming. One solution to overcome this is data augmentation, which prevents overfitting and improves accuracy [28, 29]. In this research, several augmentation methods were used, such as rotation, zoom, shift, shear, and horizontal flip.

2.4 CNN Architectures

In this study, we used a modified VGG19 and ResNet101 for our classification task. VGG19 is a well-known CNN architecture that consists of 19 convolutional layers and 3 fully connected layers. Our modified VGG19 has an additional convolutional layer to extract more features from the input data. We also added batch normalization and dropout layers to prevent overfitting. The final layer of the network is a fully connected layer with a softmax activation function for classification. Table 2 shows the architecture of our modified VGG19.

Table 2
VGG19 architecture

Layer Type Output Shape Parameter

input Input 224×224×3 0

block1_conv1 Convolution 224×224×64 1792

block1_conv2 Convolution 224×224×64 26928

block1_pool Maxpooling 112×112×64 0

block2_conv1 Convolution 112×112×128 73856

block2_conv2 Convolution 112×112×128 147584

block2_pool Maxpooling 56×56×128 0

block3_conv1 Convolution 56×56×256 295168

block3_conv2 Convolution 56×56×256 590080

block3_conv3 Convolution 56×56×256 590080

block3_conv4 Convolution 56×56×256 590080

block3_pool Maxpooling 28×28×256 0

block1_conv1 Convolution 28×28×512 1180160

block4_conv2 Convolution 28×28×512 2359808

block4_conv3 Convolution 28×28×512 2359808

block4_conv4 Convolution 28×28×512 2359808

block4_pool Maxpooling 14×14×512 0

block5_conv1 Convolution 14×14×512 2359808

block5_conv2 Convolution 14×14×512 2359808

block5_conv3 Convolution 14×14×512 2359808

block5_conv4 Convolution 14×14×512 2359808

block5_cpool Maxpooling 7×7×512 0

flatten Flatten 1×25088 0

dense Dense 1×512 12845568

dropout Dropout 1×512 0

dense_1 Dense 1×2 1026

Total parameters: 32,870,978

Trainable parameters: 24,645,634

Non-trainable parameters: 8,225,344

Layer	Type	Output Shape	Parameter
input	Input	224×224×3	0
block1_conv1	Convolution	224×224×64	1792
block1_conv2	Convolution	224×224×64	26928
block1_pool	Maxpooling	112×112×64	0
block2_conv1	Convolution	112×112×128	73856
block2_conv2	Convolution	112×112×128	147584
block2_pool	Maxpooling	56×56×128	0
block3_conv1	Convolution	56×56×256	295168
block3_conv2	Convolution	56×56×256	590080
block3_conv3	Convolution	56×56×256	590080
block3_conv4	Convolution	56×56×256	590080
block3_pool	Maxpooling	28×28×256	0
block1_conv1	Convolution	28×28×512	1180160
block4_conv2	Convolution	28×28×512	2359808
block4_conv3	Convolution	28×28×512	2359808
block4_conv4	Convolution	28×28×512	2359808
block4_pool	Maxpooling	14×14×512	0
block5_conv1	Convolution	14×14×512	2359808
block5_conv2	Convolution	14×14×512	2359808
block5_conv3	Convolution	14×14×512	2359808
block5_conv4	Convolution	14×14×512	2359808
block5_cpool	Maxpooling	7×7×512	0
flatten	Flatten	1×25088	0
dense	Dense	1×512	12845568
dropout	Dropout	1×512	0
dense_1	Dense	1×2	1026
Total parameters: 32,870,978
Trainable parameters: 24,645,634
Non-trainable parameters: 8,225,344

On the other hand, ResNet101 is a deep CNN architecture that consists of 101 layers and has achieved state-of-the-art performance on many image recognition benchmarks. Our modified ResNet101 has several adjustments to the original architecture. Firstly, two more convolutional layers were added to capture finer details of the input images. Secondly, batch normalization layers were replaced with group normalization layers to reduce memory usage and improve generalization. Lastly, a global average pooling layer and a fully connected layer with softmax activation function were included for classification. Table 3 shows the architecture of our modified ResNet101.

Table 3

ResNet101 architecture

Layer	Type	Output Shape	Parameter
Input_1	Input Layer	224×224×3	0
Conv1_pad	Zero Padding	230×230×3	–
Conv1_conv	Convolution	112×112×64	9472
Conv1_bn	Batch Normalization	112×112×64	256
Conv1_relu	Activation	112×112×64	0
Pool1_pad	Zero Padding	114×114×64	0
Pool1_pool	Max Pooling	56×56×64	0
Conv2_block1_1_conv	Convolution	56×56×64	4160
Conv2_block1_1_bn	Batch normalization	56×56×64	256
Conv2_block1_1_relu	Activation	56×56×64	0
Conv2_block1_2_conv	Convolution	56×56×64	36928
Conv2_block1_2_bn	Batch normalization	56×56×64	256
Conv2_block1_2_relu	Activation	56×56×64	0
Conv2_block1_0_conv	Convolution	56×56×256	16640
Conv2_block1_3_conv	Convolution	56×56×256	16640
Conv2_block1_0_bn	Batch normalization	56×56×256	1024
Conv2_block1_3_bn	Batch normalization	56×56×256	1024
Conv2_block1_add	Add	56×56×256	0
Conv2_block1_out	Activation	56×56×256	0
Conv2_block2_1_conv	Convolution	56×56×64	16448
Conv2_block2_1_bn	Batch normalization	56×56×64	256
Conv2_block2_1_relu	Activation	56×56×64	0
Conv2_block2_2_conv	Convolution	56×56×64	36928
Conv2_block2_2_bn	Batch normalization	56×56×64	256
Conv2_block2_2_relu	Activation	56×56×64	0
Conv2_block2_3_conv	Convolution	56×56×256	16640
Conv2_block2_3_bn	Batch normalization	56×56×256	1024
Conv2_block2_add	Add	56×56×256	0
Conv2_block2_out	Activation	56×56×256	0
Conv2_block3_1_bn	Batch normalization	56×56×64	256
Conv2_block3_1_relu	Activation	56×56×64	0
Conv2_block3_2_conv	Convolution	56×56×64	36928
Conv2_block3_2_rbn	Batch normalization	56×56×64	256
Conv2_block3_2_relu	Activation	56×56×64	0
Conv2_block3_3_conv	Convolution	56×56×256	16640
Conv2_block3_3_bn	Batch normalization	56×56×256	1024
Conv2_block3_add	Add	56×56×256	0
Conv2_block3_out	Activation	56×56×256	0
Conv3_block1_1_conv	Convolution	28×28×128	32896
Conv3_block1_1_bn	Batch normalization	28×28×128	512
Conv3_block1_1_relu	Activation	28×28×128	0
Conv3_block1_2_conv	Convolution	28×28×128	147584
Conv3_block1_2_bn	Batch normalization	28×28×128	512
Conv3_block1_2_relu	Activation	28×28×128	0
Conv3_block1_2_conv	Convolution	28×28×512	66048
Conv3_block1_0_bn	Batch normalization	28×28×512	2048
Conv3_block1_3_bn	Batch normalization	28×28×512	2048
Conv3_block1_add	Add	28×28×512	0
Conv3_block1_out	Activation	28×28×512	0
Conv3_block2_1_conv	Convolution	28×28×128	65664
Conv3_block2_1_bn	Batch normalization	28×28×128	512
Conv3_block2_1_relu	Activation	28×28×128	0
Conv3_block2_2_conv	Convolution	28×28×128	147584
Conv3_block2_2_bn	Batch normalization	28×28×128	512
Conv3_block2_2_relu	Activation	28×28×128	0
Conv3_block2_3_conv	Convolution	28×28×512	66048
Conv3_block2_3_bn	Batch normalization	28×28×512	2048
Conv3_block2_add	Add	28×28×512	0
Conv3_block2_out	Activation	28×28×512	0
Conv3_block3_1_conv	Convolution	28×28×128	65664
Conv3_block3_1_bn	Batch normalization	28×28×128	512
Conv3_block3_1_relu	Activation	28×28×128	0
Conv3_block3_2_conv	Convolution	28×28×128	147584
Conv3_block3_2_bn	Batch normalization	28×28×128	512
Conv3_block3_2_relu	Activation	28×28×128	0
Conv3_block3_3_conv	Convolution	28×28×512	66048
Conv3_block3_3_bn	Batch normalization	28×28×512	2048
Conv3_block3_add	Add	28×28×512	0
Conv3_block3_out	Activation	28×28×512	0
Conv3_block4_1_conv	Convolution	28×28×128	65664
Conv3_block4_1_bn	Batch normalization	28×28×128	512
Conv3_block4_1_relu	Activation	28×28×128	0
Conv3_block4_2_conv	Convolution	28×28×128	147584
Conv3_block4_2_bn	Batch normalization	28×28×128	512
Conv3_block4_2_relu	Activation	28×28×128	0
Conv3_block4_3_conv	Convolution	28×28×512	66048
Conv3_block4_3_bn	Batch normalization	28×28×512	2048
Conv3_block4_add	Add	28×28×512	0
Conv3_block4_out	Activation	28×28×512	0
Conv4_block1_1_conv	Convolution	14×14×256	131328
Conv4_block1_1_bn	Batch normalization	14×14×256	1024
Conv4_block1_1_relu	Activation	14×14×256	0
Conv4_block1_2_conv	Convolution	14×14×256	590080
Conv4_block1_2_bn	Batch normalization	14×14×256	1024
Conv4_block1_2_relu	Activation	14×14×256	0
Conv4_block1_0_conv	Convolution	14×14×1024	525312
Conv4_block1_3_conv	Convolution	14×14×1024	263168
Conv4_block1_0_bn	Batch normalization	14×14×1024	4096
Conv4_block1_3_bn	Batch normalization	14×14×1024	4096
Conv4_block1_add	Add	14×14×1024	0
Conv4_block1_out	Activation	14×14×1024	0
Conv4_block2_1_conv	Convolution	14×14×256	262400
Conv4_block2_1_bn	Batch normalization	14×14×256	1024
Conv4_block2_1_relu	Activation	14×14×256	0
Conv4_block2_2_conv	Convolution	14×14×256	590080
Conv4_block2_2_bn	Batch normalization	14×14×256	1024
Conv4_block2_2_relu	Activation	14×14×256	0
Conv4_block2_3_conv	Convolution	14×14×1024	263168
Conv4_block2_3_bn	Batch normalization	14×14×1024	4096
Conv4_block2_add	Add	14×14×1024	0
Conv4_block2_out	Activation	14×14×1024	0
Conv4_block3_1_conv	Convolution	14×14×256	262400
Conv4_block3_1_bn	Batch normalization	14×14×256	1024
Conv4_block3_1_relu	Activation	14×14×256	0
Conv4_block3_2_conv	Convolution	14×14×256	590080
Conv4_block3_2_bn	Batch normalization	14×14×256	1024
Conv4_block3_2_relu	Activation	14×14×256	0
Conv4_block3_3_conv	Convolution	14×14×1024	263168
Conv4_block3_3_bn	Batch normalization	14×14×1024	4096
Conv4_block3_add	Add	14×14×1024	0
Conv4_block3_out	Activation	14×14×1024	0
Conv4_block4_1_conv	Convolution	14×14×256	262400
Conv4_block4_1_bn	Batch normalization	14×14×256	1024
Conv4_block4_1_relu	Activation	14×14×256	0
Conv4_block4_2_conv	Convolution	14×14×256	590080
Conv4_block4_2_bn	Batch normalization	14×14×256	1024
Conv4_block4_2_relu	Activation	14×14×256	0
Conv4_block4_3_conv	Convolution	14×14×1024	263168
Conv4_block4_3_bn	Batch normalization	14×14×1024	4096
Conv4_block4_add	Add	14×14×1024	0
Conv4_block4_out	Activation	14×14×1024	0
Conv4_block5_1_conv	Convolution	14×14×256	262400
Conv4_block5_1_bn	Batch normalization	14×14×256	1024
Conv4_block5_1_relu	Activation	14×14×256	0
Conv4_block5_2_conv	Convolution	14×14×256	590080
Conv4_block5_2_bn	Batch normalization	14×14×256	1024
Conv4_block5_2_relu	Activation	14×14×256	0
Conv4_block5_3_conv	Convolution	14×14×1024	263168
Conv4_block5_3_bn	Batch normalization	14×14×1024	4096
Conv4_block5_add	Add	14×14×1024	0
Conv4_block5_out	Activation	14×14×1024	0
Conv4_block6_1_conv	Convolution	14×14×256	262400
Conv4_block6_1_bn	Batch normalization	14×14×256	1024
Conv4_block6_1_relu	Activation	14×14×256	0
Conv4_block6_2_conv	Convolution	14×14×256	590080
Conv4_block6_2_bn	Batch normalization	14×14×256	1024
Conv4_block6_2_relu	Activation	14×14×256	0
Conv4_block6_3_conv	Convolution	14×14×1024	263168
Conv4_block6_3_bn	Batch normalization	14×14×1024	4096
Conv4_block6_add	Add	14×14×1024	0
Conv4_block6_out	Activation	14×14×1024	0
Conv4_block7_1_conv	Convolution	14×14×256	262400
Conv4_block7_1_bn	Batch normalization	14×14×256	1024
Conv4_block7_1_relu	Activation	14×14×256	0
Conv4_block7_2_conv	Convolution	14×14×256	590080
Conv4_block7_2_bn	Batch normalization	14×14×256	1024
Conv4_block7_2_relu	Activation	14×14×256	0
Conv4_block7_3_conv	Convolution	14×14×1024	263168
Conv4_block7_3_bn	Batch normalization	14×14×1024	4096
Conv4_block7_add	Add	14×14×1024	0
Conv4_block7_out	Activation	14×14×1024	0
Conv4_block8_1_conv	Convolution	14×14×256	262400
Conv4_block8_1_bn	Batch normalization	14×14×256	1024
Conv4_block8_1_relu	Activation	14×14×256	0
Conv4_block8_2_conv	Convolution	14×14×256	590080
Conv4_block8_2_bn	Batch normalization	14×14×256	1024
Conv4_block8_2_relu	Activation	14×14×256	0
Conv4_block8_3_conv	Convolution	14×14×1024	263168
Conv4_block8_3_bn	Batch normalization	14×14×1024	4096
Conv4_block8_add	Add	14×14×1024	0
Conv4_block8_out	Activation	14×14×1024	0
Conv4_block9_1_conv	Convolution	14×14×256	262400
Conv4_block9_1_bn	Batch normalization	14×14×256	1024
Conv4_block9_1_relu	Activation	14×14×256	0
Conv4_block9_2_conv	Convolution	14×14×256	590080
Conv4_block9_2_bn	Batch normalization	14×14×256	1024
Conv4_block9_2_relu	Activation	14×14×256	0
Conv4_block9_3_conv	Convolution	14×14×1024	263168
Conv4_block9_3_bn	Batch normalization	14×14×1024	4096
Conv4_block9_add	Add	14×14×1024	0
Conv4_block9_out	Activation	14×14×1024	0
Conv4_block10_1_conv	Convolution	14×14×256	262400
Conv4_block10_1_bn	Batch normalization	14×14×256	1024
Conv4_block10_1_relu	Activation	14×14×256	0
Conv4_block10_2_conv	Convolution	14×14×256	590080
Conv4_block10_2_bn	Batch normalization	14×14×256	1024
Conv4_block10_2_relu	Activation	14×14×256	0
Conv4_block10_3_conv	Convolution	14×14×1024	263168
Conv4_block10_3_bn	Batch normalization	14×14×1024	4096
Conv4_block10_add	Add	14×14×1024	0
Conv4_block10_out	Activation	14×14×1024	0
Conv4_block11_1_conv	Convolution	14×14×256	262400
Conv4_block11_1_bn	Batch normalization	14×14×256	1024
Conv4_block11_1_relu	Activation	14×14×256	0
Conv4_block11_2_conv	Convolution	14×14×256	590080
Conv4_block11_2_bn	Batch normalization	14×14×256	1024
Conv4_block11_2_relu	Activation	14×14×256	0
Conv4_block11_3_conv	Convolution	14×14×1024	263168
Conv4_block11_3_bn	Batch normalization	14×14×1024	4096
Conv4_block11_add	Add	14×14×1024	0
Conv4_block11_out	Activation	14×14×1024	0
Conv4_block12_1_conv	Convolution	14×14×256	262400
Conv4_block12_1_bn	Batch normalization	14×14×256	1024
Conv4_block12_1_relu	Activation	14×14×256	0
Conv4_block12_2_conv	Convolution	14×14×256	590080
Conv4_block12_2_bn	Batch normalization	14×14×256	1024
Conv4_block12_2_relu	Activation	14×14×256	0
Conv4_block12_3_conv	Convolution	14×14×1024	263168
Conv4_block12_3_bn	Batch normalization	14×14×1024	4096
Conv4_block12_add	Add	14×14×1024	0
Conv4_block12_out	Activation	14×14×1024	0
Conv4_block13_1_conv	Convolution	14×14×256	262400
Conv4_block13_1_bn	Batch normalization	14×14×256	1024
Conv4_block13_1_relu	Activation	14×14×256	0
Conv4_block13_2_conv	Convolution	14×14×256	590080
Conv4_block13_2_bn	Batch normalization	14×14×256	1024
Conv4_block13_2_relu	Activation	14×14×256	0
Conv4_block13_3_conv	Convolution	14×14×1024	263168
Conv4_block13_3_bn	Batch normalization	14×14×1024	4096
Conv4_block13_add	Add	14×14×1024	0
Conv4_block13_out	Activation	14×14×1024	0
Conv4_block14_1_conv	Convolution	14×14×256	262400
Conv4_block14_1_bn	Batch normalization	14×14×256	1024
Conv4_block14_1_relu	Activation	14×14×256	0
Conv4_block14_2_conv	Convolution	14×14×256	590080
Conv4_block14_2_bn	Batch normalization	14×14×256	1024
Conv4_block14_2_relu	Activation	14×14×256	0
Conv4_block14_3_conv	Convolution	14×14×1024	263168
Conv4_block14_3_bn	Batch normalization	14×14×1024	4096
Conv4_block14_add	Add	14×14×1024	0
Conv4_block14_out	Activation	14×14×1024	0
Conv4_block15_1_conv	Convolution	14×14×256	262400
Conv4_block15_1_bn	Batch normalization	14×14×256	1024
Conv4_block15_1_relu	Activation	14×14×256	0
Conv4_block15_2_conv	Convolution	14×14×256	590080
Conv4_block15_2_bn	Batch normalization	14×14×256	1024
Conv4_block15_2_relu	Activation	14×14×256	0
Conv4_block15_3_conv	Convolution	14×14×1024	263168
Conv4_block15_3_bn	Batch normalization	14×14×1024	4096
Conv4_block15_add	Add	14×14×1024	0
Conv4_block15_out	Activation	14×14×1024	0
Conv4_block16_1_conv	Convolution	14×14×256	262400
Conv4_block16_1_bn	Batch normalization	14×14×256	1024
Conv4_block16_1_relu	Activation	14×14×256	0
Conv4_block16_2_conv	Convolution	14×14×256	590080
Conv4_block16_2_bn	Batch normalization	14×14×256	1024
Conv4_block16_2_relu	Activation	14×14×256	0
Conv4_block16_3_conv	Convolution	14×14×1024	263168
Conv4_block16_3_bn	Batch normalization	14×14×1024	4096
Conv4_block16_add	Add	14×14×1024	0
Conv4_block16_out	Activation	14×14×1024	0
Conv4_block17_1_conv	Convolution	14×14×256	262400
Conv4_block17_1_bn	Batch normalization	14×14×256	1024
Conv4_block17_1_relu	Activation	14×14×256	0
Conv4_block17_2_conv	Convolution	14×14×256	590080
Conv4_block17_2_bn	Batch normalization	14×14×256	1024
Conv4_block17_2_relu	Activation	14×14×256	0
Conv4_block17_3_conv	Convolution	14×14×1024	263168
Conv4_block17_3_bn	Batch normalization	14×14×1024	4096
Conv4_block17_add	Add	14×14×1024	0
Conv4_block17_out	Activation	14×14×1024	0
Conv4_block18_1_conv	Convolution	14×14×256	262400
Conv4_block18_1_bn	Batch normalization	14×14×256	1024
Conv4_block18_1_relu	Activation	14×14×256	0
Conv4_block18_2_conv	Convolution	14×14×256	590080
Conv4_block18_2_bn	Batch normalization	14×14×256	1024
Conv4_block18_2_relu	Activation	14×14×256	0
Conv4_block18_3_conv	Convolution	14×14×1024	263168
Conv4_block18_3_bn	Batch normalization	14×14×1024	4096
Conv4_block18_add	Add	14×14×1024	0
Conv4_block18_out	Activation	14×14×1024	0
Conv4_block19_1_conv	Convolution	14×14×256	262400
Conv4_block19_1_bn	Batch normalization	14×14×256	1024
Conv4_block19_1_relu	Activation	14×14×256	0
Conv4_block19_2_conv	Convolution	14×14×256	590080
Conv4_block19_2_bn	Batch normalization	14×14×256	1024
Conv4_block19_2_relu	Activation	14×14×256	0
Conv4_block19_3_conv	Convolution	14×14×1024	263168
Conv4_block19_3_bn	Batch normalization	14×14×1024	4096
Conv4_block19_add	Add	14×14×1024	0
Conv4_block19_out	Activation	14×14×1024	0
Conv4_block20_1_conv	Convolution	14×14×256	262400
Conv4_block20_1_bn	Batch normalization	14×14×256	1024
Conv4_block20_1_relu	Activation	14×14×256	0
Conv4_block20_2_conv	Convolution	14×14×256	590080
Conv4_block20_2_bn	Batch normalization	14×14×256	1024
Conv4_block20_2_relu	Activation	14×14×256	0
Conv4_block20_3_conv	Convolution	14×14×1024	263168
Conv4_block20_3_bn	Batch normalization	14×14×1024	4096
Conv4_block20_add	Add	14×14×1024	0
Conv4_block20_out	Activation	14×14×1024	0
Conv4_block21_1_conv	Convolution	14×14×256	262400
Conv4_block21_1_bn	Batch normalization	14×14×256	1024
Conv4_block21_1_relu	Activation	14×14×256	0
Conv4_block21_2_conv	Convolution	14×14×256	590080
Conv4_block21_2_bn	Batch normalization	14×14×256	1024
Conv4_block21_2_relu	Activation	14×14×256	0
Conv4_block21_3_conv	Convolution	14×14×1024	263168
Conv4_block21_3_bn	Batch normalization	14×14×1024	4096
Conv4_block21_add	Add	14×14×1024	0
Conv4_block21_out	Activation	14×14×1024	0
Conv4_block22_1_conv	Convolution	14×14×256	262400
Conv4_block22_1_bn	Batch normalization	14×14×256	1024
Conv4_block22_1_relu	Activation	14×14×256	0
Conv4_block22_2_conv	Convolution	14×14×256	590080
Conv4_block22_2_bn	Batch normalization	14×14×256	1024
Conv4_block22_2_relu	Activation	14×14×256	0
Conv4_block22_3_conv	Convolution	14×14×1024	263168
Conv4_block22_3_bn	Batch normalization	14×14×1024	4096
Conv4_block22_add	Add	14×14×1024	0
Conv4_block22_out	Activation	14×14×1024	0
Conv4_block23_1_conv	Convolution	14×14×256	262400
Conv4_block23_1_bn	Batch normalization	14×14×256	1024
Conv4_block23_1_relu	Activation	14×14×256	0
Conv4_block23_2_conv	Convolution	14×14×256	590080
Conv4_block23_2_bn	Batch normalization	14×14×256	1024
Conv4_block23_2_relu	Activation	14×14×256	0
Conv4_block23_3_conv	Convolution	14×14×1024	263168
Conv4_block23_3_bn	Batch normalization	14×14×1024	4096
Conv4_block23_add	Add	14×14×1024	0
Conv4_block23_out	Activation	14×14×1024	0
Conv5_block1_1_conv	Convolution	7×7×512	524800
Conv5_block1_1_bn	Batch normalization	7×7×512	2048
Conv5_block1_1_relu	Activation	7×7×512	0
Conv5_block1_2_conv	Convolution	7×7×512	2359808
Conv5_block1_2_bn	Batch normalization	7×7×512	2048
Conv5_block1_2_relu	Activation	7×7×512	0
Conv5_block1_0_conv	Convolution	7×7×2048	2099200
Conv5_block1_3_conv	Convolution	7×7×2048	1050624
Conv5_block1_0_bn	Batch normalization	7×7×2048	8192
Conv5_block1_3_bn	Batch normalization	7×7×2048	8192
Conv5_block1_add	Add	7×7×2048	0
Conv5_block1_out	Activation	7×7×2048	0
Conv5_block2_1_conv	Convolution	7×7×512	1049088
Conv5_block2_1_bn	Batch normalization	7×7×512	2048
Conv5_block2_1_relu	Activation	7×7×512	0
Conv5_block2_2_conv	Convolution	7×7×512	2359808
Conv5_block2_2_bn	Batch normalization	7×7×512	2048
Conv5_block2_2_relu	Activation	7×7×512	0
Conv5_block2_3_conv	Convolution	7×7×2048	1050624
Conv5_block2_3_bn	Batch normalization	7×7×2048	8192
Conv5_block2_add	Add	7×7×2048	0
Conv5_block2_out	Activation	7×7×2048	0
Conv5_block3_1_conv	Convolution	7×7×512	1049088
Conv5_block3_1_bn	Batch normalization	7×7×512	2048
Conv5_block3_1_relu	Activation	7×7×512	0
Conv5_block3_2_conv	Convolution	7×7×512	2359808
Conv5_block3_2_bn	Batch normalization	7×7×512	2048
Conv5_block3_2_relu	Activation	7×7×512	0
Conv5_block3_3_conv	Convolution	7×7×2048	1050624
Conv5_block3_3_bn	Batch normalization	7×7×2048	8192
Conv5_block3_add	Add	7×7×2048	0
Conv5_block3_out	Activation	7×7×2048	0
Flatten	Flatten	1×100352	0
Dense	Dense	1×512	51380736
Dropout	Dropout	1×512	0
Dense_1	Dense	1×2	1026
Total parameters: 94,039,938
Trainable parameters: 93,850,370
Non-trainable parameters: 189,568

2.5 Evaluation Metrics

Confusion matrix is a technique used to measure the performance and summarize the results of deep learning classification models. The calculation of the confusion matrix provides a good representation of the classification model’s accuracy and the types of errors it makes. It includes a combination of the differences between predicted and actual images.

The confusion matrix consists of four possible outcomes: true positive (TP), true negative (TN), false positive (FP), and false negative (FN). When an image is actually positive and classified as positive, it is labeled as TP. If it is classified as negative, it is labeled as TN. Similarly, for negative images, if they are classified as negative, they are labeled as TN, and if they are classified as positive, they are labeled as FP. Figure 2 illustrates the confusion matrix.

Fig. 2

Confusion matrix.

To evaluate the performance of the model, several key parameters are used: accuracy, sensitivity (recall), and precision (positive predictive value). These different parameters provide an analysis of the model’s performance from different perspectives. Accuracy represents the percentage of images correctly classified, sensitivity describes the proportion of actual positive images correctly predicted as positive, and precision represents the proportion of predicted positive images that are actually positive. Precision, sensitivity, and accuracy are calculated using Equations 3).

$Precision % = (\frac{TP}{TP + FP}) \times 100$ (1)

$Sensitivity % = (\frac{TP}{TP + FN}) \times 100$ (2)

$Accuracy % = (\frac{TP + TN}{TP + TN + FP + FN}) \times 100$ (3)

3 Result

CNN models were trained using MPBT and MBT MRI images with consistent parameters, including epoch = 30, 60, 90; Batch Size = 8; ReLU activation function; and Categorical Cross-Entropy loss function. The training was conducted on an Asus Laptop with an Intel® Core i5-7200U CPU @ 2.50 GHz processor and 12.00 GB RAM NVIDIA GeForce930MX. Keras, a python module, was used as a frameworkfor all the networks.

The evaluation of the models was performed using 204 brain tumor MRI images, consisting of 102 MPBT and 102 MBT images. Figure 3 illustrates the accuracy and loss graphs for each VGG19 and ResNet101 model. Table 4 presents the performance of each model based on accuracy (A), sensitivity (S), and precision (P). Additionally, Figure 4 displays the confusion matrix for each model.

Table 4
The result of each model by accuracy (A), sensitivity (S), and precision (P) metrics

Model A (%) S (%) P (%)

VGG19 epoch 30 77 77 77

VGG19 epoch 60 81 81 82

VGG19 epoch 90 77 77 78

ResNet101 epoch 30 81 81 82

ResNet101 epoch 60 83 83 83

ResNet101 epoch 90 81 81 81

Model	A (%)	S (%)	P (%)
VGG19 epoch 30	77	77	77
VGG19 epoch 60	81	81	82
VGG19 epoch 90	77	77	78
ResNet101 epoch 30	81	81	82
ResNet101 epoch 60	83	83	83
ResNet101 epoch 90	81	81	81

Fig. 3

The accuracy and loss graphics of VGG 19 using epoch (a) 30, (b) 60, (c) 90 and ResNet101 using epoch (d) 30, (e) 60, (f) 90.

Fig. 4

The confusion matrics of VGG 19 using epoch (a) 30, (b) 60, (c) 90 and ResNet101 using epoch (d) 30, (e) 60, (f) 90.

Based on the data, the results indicate that the model with the highest performance is ResNet101 epoch 60, achieving an accuracy, sensitivity, and precision of 83%. It shows an improvement in both train and validation accuracy; however, there is a significant increase in validation loss. Furthermore, this model successfully classified 90 out of 102 MPBT images accurately.

On the other hand, VGG19 epoch 60 also demonstrated good performance, with an accuracy, sensitivity, and precision of 81%. It exhibited relatively stable train and validation accuracy, as well as validation loss. This model correctly classified 86 out of 102 MBT images, outperforming the other models in terms of classification accuracy.

Furthermore, since the dataset used consisted of various MRI sequences (T1, T2, T1C, and FLAIR), a comparison was conducted between each model and these sequences. Table 5 presents the testing data used. The sensitivity of the models in classifying MPBT and MBT is shown in Tables 6 7.

Table 5

The data testing

Class	T1	T2	T1C	FLAIR	Total
MPBT	25	25	26	26	102
MBT	25	25	26	26	102

Table 6

The sensitivity of the models in classifying MPBT

Model	T1(%)	T2(%)	T1C(%)	FLAIR(%)
VGG19 epoch 30	32.0	84.0	61.5	61.5
VGG19 epoch 60	20.0	84.0	53.8	69.2
VGG19 epoch 90	24.0	96.0	57.7	61.5
ResNet101 epoch 30	48.0	76.0	53.8	57.7
ResNet101 epoch 60	52.0	92.0	57.7	61.5
ResNet101 epoch 90	52.0	84.0	61.5	61.5
Average Sensitivity	38.0	86.0	57.7	62.1

Table 7

The sensitivity of the models in classifying MBT

Model	T1(%)	T2(%)	T1C(%)	FLAIR(%)
VGG19 epoch 30	96.0	68.0	84.6	88.5
VGG19 epoch 60	84.0	68.0	92.3	88.5
VGG19 epoch 90	96.0	68.0	96.1	92.3
ResNet101 epoch 30	92.0	84.0	92.3	80.8
ResNet101 epoch 60	88.0	76.0	92.3	76.9
ResNet101 epoch 90	92.0	88.0	92.3	84.6
Average Sensitivity	91.3	75.3	91.6	85.3

Based on the data, the T2 sequence exhibited the highest average sensitivity for MPBT images, with a value of 86% compared to other sequences. The model with the highest sensitivity was VGG19 epoch 90, achieving 96%. For MBT images, the sequences with the highest average sensitivity were T1C and T1, with values of 91.6% and 91.3%, respectively. The model with the highest sensitivity for T1C and T1 was VGG19 epoch 90, with values of 96.1% and 96.0%, respectively. These findings suggest that the T2 sequence is particularly effective in detecting MPBT, while T1C and T1 sequences are more sensitive in identifying MBT. The VGG19 model at epoch 90 consistently demonstrated superior sensitivity in classifying both tumor types.

4 Discussion

Distinguishing MPBT and MBT accurately is crucial for improving patient outcomes and optimizing healthcare services. Incorrect diagnosis can lead to the administration of improper therapies, worsening the patient’s condition and resulting in mortality. The use of DL is expected to assist doctors in establishing a faster and more accurate diagnosis. Table 8 displays a previous study on brain tumor classification using VGG19 and ResNet101 with similar MRI sequences.

Table 8
The comparison of previous study on brain tumor classification using VGG19 and ResNet101 with similar MRI sequences

Study Network Sequence Total A (%) Classification Dataset

Rajinikanth VGG19 T2, T1C, FLAIR 6300 94.7 High- and low- BRATS

ResNet101 6300 93.9 grade gliomas

This study VGG19 T1, T2, T1C, FLAIR 1360 78.3 MPBT and MBT Dr. Moewardi

ResNet101 1360 81.6 Hospital

Study	Network	Sequence	Total	A (%)	Classification	Dataset
Rajinikanth	VGG19	T2, T1C, FLAIR	6300	94.7	High- and low-	BRATS
	ResNet101		6300	93.9	grade gliomas
This study	VGG19	T1, T2, T1C, FLAIR	1360	78.3	MPBT and MBT	Dr. Moewardi
	ResNet101		1360	81.6		Hospital

From the comparison, it is evident that the study by Rajinikanth et al. reported high accuracies of 94.7% and 93.9% for the VGG19 and ResNet101 models, respectively. These accuracies were obtained using a dataset of 6300 images sourced from The Brain Tumor Segmentation (BRAST). The study focused on classifying images into high-grade and low-grade gliomas using three sequence types: T2, T1C, and FLAIR [30, 31]. In contrast, this study achieved accuracies of 78.3% and 81.6% for the VGG19 and ResNet101 models, respectively, based on processing 1360 images obtained from Dr. Moewardi Hospital. The sequences utilized in this study were T1, T2, T1C, and FLAIR, aiming to classify the images into MPBT and MBT.

A significant difference in accuracy between the two studies is apparent. One important factor is the composition of the datasets used. It is not simply a matter of the quantity of data, but rather the extent to which the dataset represents the original distribution of the target population. Inaccurate or unrepresentative datasets can introduce bias and affect the generalizability of the results. Therefore, careful consideration should be given to dataset selection and ensuring that it adequately represents the population being studied [32] Additionally, the sequences employed in the studies differ. This study also includes the use of T1, which is known for its ability to visualize brain anatomy. However, certain types of brain tumors are challenging to identify using this sequence, whereas they are more easily distinguishable using T2, T1C, and FLAIR [33, 34]. Furthermore, the classification outcomes differ between the studies. Rajinikanth et al. focused on classifying high-grade and low-grade gliomas, which are both types of MPBT. In contrast, this study aimed to differentiate between MPBT and MBT [30].

In addition to evaluating the model’s overall accuracy, this study also analyzed the results from a medical perspective regarding the impact of different sequences on the model’s classification outcomes. Sensitivity was chosen as the parameter for this analysis. Several guideline development studies have highlighted sensitivity as one of the latest decision tools for describing the potential improvements in the utilization of imaging techniques [35]. However, this specific analysis has not been previously investigated in prior studies, and therefore, there are no existing results for comparison.

The results of this research indicate variations in the sensitivity of deep learning models when classifying MPBT and MBT across different MRI sequences. For MPBT classification, the T2 sequence exhibited the highest sensitivity, while for MBT classification, the T1C and T1 sequences demonstrated the highest sensitivity. Conversely, the T1 sequence showed the lowest sensitivity for MPBT, while the T2 sequence showed the lowest sensitivity for MBT. The magnitude of sensitivity difference between the highest and lowest values was significant for each tumor type.

MRI sequences play a crucial role in determining the sensitivity of the deep learning models. Different sequences, such as T1, T2, fat-suppressed, and enhanced gadolinium, provide distinct images when there are tissue abnormalities present. Among these sequences, T1, T2, T1C, and FLAIR are commonly used for diagnosing brain tumors [25].

The T1 sequence is a routine and widely used sequence in MRI protocols. It offers a clear anatomical image that closely resembles the macroscopic appearance of the tissue. It has a short Echo Time (TE) and Repetition Time (TR), which influence the image characteristics [36].

On the other hand, the T2 sequence, which has a long TE and TR, is also included in almost all MRI protocols. The difference between TE and TR settings tends to produce different image results [37]. Table 9 shows the differences in the intensity and color of human tissue produced by T1 and T2 sequences.

Table 9

The differences in intensity and color produced by T1 and T2 sequences

Tissue	T1	T2
Fluid	Low (Black)	High (White)
Muscle	Intermediate (Grey)	Intermediate (Grey)
Fat	High (White)	High (White)
Gray Matter	Intermediate (Grey)	Intermediate (Grey)
White Matter	Brighter than grey matter (Whiteish)	Darker than grey matter (Darkish)

The administration of contrast in MRI scans can enhance the visibility of certain tissues and pathological conditions. Contrast agents are more commonly used in the T1C sequence because they tend to increase the signal intensity in T1. Pathological tissues like tumors, inflammation, and infections often exhibit contrast enhancement due to leaky blood vessels. This accumulation of contrast agent causes the tissue to appear brighter compared to the surrounding tissues [25].

FLAIR is a specialized sequence with a long inversion time that suppresses the signal from cerebrospinal fluid (CSF). In this sequence, brain tissue mimics the appearance of T2, with gray matter appearing lighter than white matter. FLAIR is particularly useful in evaluating various CNS diseases, including infarction, multiple sclerosis, subarachnoid hemorrhage, head injury, and others [38].

The type of brain tumors including MPBT and MBT is another variable that directly affects DL sensitivity. MPBT and MBT originate from abnormal tissue inside and outside the brain, and this makes them have special characteristics. The typical characteristics of MPBT include a well-defined tumor, irregular shape, necrosis, and peripheral edema. MRI is significantly more sensitive to the tumor and peritumoral edema presence. Meanwhile, MBT has large lesions and a limited mass effect, indicating that there is infiltrative growth. The distinctive characteristic of MBT is that it is often found at the gray-white matter junction. This location is localized to 80% of the cerebral hemispheres. Moreover, MBT has a peripheral sphenoid coiled that becomes lesional, multiple, and anoxal oedematous. Table 10 shows the characteristics of MPBT and MBT [26 , 40].

Table 10

The characteristic differences between MPBT and MBT

Characteristics	MPBT	MBT
Total	Single lesion	Multiple lesions
Location	Subcortical white matter	Gray-white matter junction
Morphology	Irregular	Round
Edge Detection	Difficult	Easy

The characteristics of MPBT and MBT produce several MRI images. Table 11 shows the differences in each MPBT and MBT appearance [41 –43].

Table 11

MPBT and MBT MRI images in each sequence

Sequence	MPBT	MBT
T1	•Hypo to isointense mass in white matter	•Usually iso- to hypointense
	•Central heterogeneous signal (intratumoral hemorrhage)	•If hemorrhagic usually has an intrinsic high signal
T2	∘ Hyperintense	∘ Hyperintense
	∘ Surrounded by extensive vasogenic edema	∘ Bleeding/melanin can change this
	∘ Heterogeneous patchy enhancement
T1C	•Enhancement varies but is almost always there	•Ring-enhancing but usually intense
	•Usually peripheral and irregular with a nodular component	•Delayed sequences may show additional lesions
FLAIR	∘ Hyperintense	∘ Hyperintense
	∘ Surrounded by vasogenic edema	∘ Surrounded by peritumoral edema

MPBT has special characteristics that are surrounded by extensive vasogenic edema and heterogeneous patchy enhancement. These characteristics can be seen in the sequence T2. MBT has a special characteristic, namely ring-enhancing which can be seen in the T1C sequence. In addition, the typical MBT location is in a grey-white matter junction and the number of multiple can be seen in sequence T1. This distinctive characteristic is following the results of this study where MPBT has the highest accuracy in the T2 sequence (Table 6) and MBT in the T1C and T1 sequences (Table 7).

FLAIR sequences produce an image of hyperintense tumor lesions. However, this image does not become a special characteristic because it can be found in MPBT and MBT. Meanwhile, T1 and T1C sequences have no distinct characteristics to MPBT. It is similar to that found in the T2 sequence on MBT. This corresponds to the results in Tables 6 7 where FLAIR accuracy is middle in both tumor types and low on T1 and T1C for MPBT and T2 for MBT. Table 12 shows the specific pattern of MPBT and MBT in each sequence.

Table 12

Specific pattern of MPBT and MBT in each sequence

Class	Highest Sensitivity		Middle Sensitivity		Lowest Sensitivity
	Seq	Characteristics	Seq	Characteristics	Seq	Characteristics
MPBT	T2	•Extensive vasogenic edema	FLAIR	•Hyperintense tumor lesions	T1, T1C	•No specific pattern
		•Heterogeneous patchy enhancement
MBT	T1C, T1	∘ Ring-enhancing	FLAIR	∘ Hyperintense tumor lesions, surrounded by peritumoral edema	T2	∘ No specific pattern
		∘ Number of multiple lesions
		∘ Location at the gray-white matter junction

The findings discussed have several implications for the use of DL in brain tumor classification and MRI sequence selection. Firstly, DL algorithms can benefit from the specific characteristics exhibited by different tumor types in various MRI sequences. By training these algorithms on a diverse dataset that includes multiple sequences, they can learn to identify and interpret the distinct patterns associated with different tumor types. This can enhance their ability to accurately classify brain tumors based on the MRI images.

Secondly, the identification of MRI sequences that exhibit higher sensitivity for specific tumor types, such as T2 for MPBT and T1C/T1 for MBT, suggests the importance of sequence selection in DL models. Integrating this knowledge into the design of DL architectures and preprocessing pipelines can help optimize the performance of these models in brain tumor classification tasks. By focusing on the most informative sequences, DL algorithms can extract relevant features and make more accurate predictions.

Furthermore, these findings highlight the need for continued research and development in DL techniques for brain tumor classification. As the field progresses, more sophisticated models can be designed to leverage the specific characteristics observed in MRI sequences, improving their ability to discriminate between different tumor types and providing more precise diagnostic information. This can assist clinicians in making informed treatment decisions and selecting appropriate therapies tailored to individual patients.

Ultimately, the implications of these findings suggest that DL, in conjunction with optimized MRI sequence selection, has the potential to revolutionize brain tumor classification. By harnessing the power of AI and combining it with advanced imaging techniques, we can expect improved accuracy, efficiency, and personalized approaches to brain tumor diagnosis and treatment, which holds promise for enhancing patient outcomes and advancing the field of neuro-oncology.

5 Conclusion

In conclusion, this study aimed to classify malignant primary brain tumors (MPBT) and metastatic brain tumors (MBT) using deep learning models and evaluate the impact of different MRI sequences on their performance. Two CNN architectures, VGG19 and ResNet101, were trained and tested on a dataset of MPBT and MBT MRI images.

The results showed that both VGG19 and ResNet101 achieved good performance in classifying brain tumors. ResNet101 with epoch 60 demonstrated the highest accuracy, sensitivity, and precision of 83% and successfully classified 90 out of 102 MPBT images accurately. VGG19 with epoch 60 achieved an accuracy, sensitivity, and precision of 81% and accurately classified 86 out of 102 MBT images. Both models showed improvement in train and validation accuracy.

The comparison of MRI sequences revealed that the T2 sequence had the highest average sensitivity for MPBT images, while T1C and T1 sequences were more sensitive in identifying MBT. VGG19 at epoch 90 consistently demonstrated superior sensitivity in classifying both tumor types. Therefore, we recommend using the T2 sequence for MPBT classification and T1C and T1 sequences for MBT classification in future applications.

These findings highlight the potential of deep learning models in accurately classifying brain tumors and the importance of considering different MRI sequences for improved classification performance. The study contributes to the existing literature on brain tumor classification using deep learning techniques.

However, it is worth noting that the accuracies achieved in this study were lower compared to some previous studies. Further research and exploration are needed to enhance the performance of the models and increase the dataset size for more comprehensive analysis.

Overall, the application of deep learning models in brain tumor classification has the potential to assist healthcare providers in making faster and more accurate diagnoses, leading to better patient outcomes and optimized healthcare services. MRI sequences with higher sensitivity for specific tumor types underscores the importance of sequence selection in optimizing deep learning models.

Footnotes

Acknowledgments

The authors are grateful to the Faculty of Medicine, Universitas Brawijaya, for the funding and support provided through the 2022 Professorial Grant.

Ethical statement

The datasets collection was approved by the Health Research Ethics Committee, Dr. Moewardi Hospital Surakarta (Ethical Clearance No. 846/VI/HREC/2022).

References

de Robles

, Fiest

K.M.

, Frolkis

A.D.

, et al. The worldwide incidence and prevalence of primary brain tumors: a systematic review and meta-analysis, Neuro Oncol 17(6) (2015), 776–783.

Ostrom

Q.T.

, Gittleman

, Truitt

, et al. CBTRUS statistical report: Primary brain and other central nervous system tumors diagnosed in the United States in 2011–2015, Neuro Oncol 20(suppl_4) (2018), iv1–86.

KPKN. Pedoman Nasional Pelayana Kedokteran Tumor Otak. Komite Penanggulangan Kanker Nasional, editor. Jakarta: Kementerian Kesehatan Republik Indonesia; 2017, pp. 1–92.

Simamora

S.K.

and Zanariah

, Space Occupying Lesion (SOL), J Medula Unila 7(1) (2017), 68–73.

Glioblastoma

ABTA.

, ABTA. Glioblastoma & Anaplastic Astrocytoma [Internet]. Chicango: American Brain Tumor Association; 2018, pp. 2–3. Available from: www.abta.org

Lee

E.J.

, TerBrugge

, Mikulis

, et al. Diagnostic value of peritumoral minimum apparent diffusion coefficient for differentiation of glioblastoma multiforme from solitary metastatic lesions, American Journal of Roentgenology 196(1) (2011), 71–76.

Kalpathy-Cramer

, Gerstner

E.R.

, Emblem

K.E.

, et al., Advanced magnetic resonance imaging of the physical processes in human Glioblastoma, Cancer Res 74(17) (2014), 4622–4637.

Suta

I.B.L.M.

, Hartati

R.S.

and Divayana

, Diagnosa tumor otak berdasarkan citra MRI (Magnetic Resonance Imaging), Majalah Ilmiah Teknologi Elektro 18(2) (2019), 149–154.

Mabray

M.C.

, Barajas

R.F.

and Cha

, Modern brain tumor imaging, Brain Tumor Res Treat 3(1) (2015), 8–23.

10.

Jung

B.Y.

, Lee

E.J.

, Bae

J.M.

, et al., Differentiation between glioblastoma and solitary metastasis: Morphologic assessment by conventional brain MR imaging and diffusion-weighted imaging, Investig Magn Reson Imaging 25(1) (2021), 23.

11.

Rasuli

and Gaillard

, Glioblastoma vs cerebral metastasis, In: Radiopaedia.org. Radiopaedia.org; 2016.

12.

Shen

, Zhang

C.J.P.

, Jiang

, et al., Artificial intelligence versus clinicians in disease diagnosis: Systematic review, JMIR Med Inform 7(3) (2019), e10010.

13.

Wells

P.N.T.

, Artificial intelligence in radiology, British Journal of Radiology 70 (1997), 1–2.

14.

Mazurowski

M.A.

, Buda

, Saha

and Bashir

M.R.

, Deep learning in radiology: An overview of the concepts and a survey of the state of the art with focus on MRI, Journal of Magnetic Resonance Imaging. JohnWiley and Sons Inc.; Vol. 49, 2019, pp. 939–954.

15.

Zaib

, The Role of machine learning and artificial intelligence in neuroscience research, Archives of Clinical Psychiatry 49(3) (2022).

16.

Kleihues

, Louis

D.N.

, Scheithauer

B.W.

, et al., The WHO classification of tumors of the nervous system, J Neuropathol Exp Neurol 61(3) (2002), 215–225.

17.

Irmak

, Multi-classification of brain tumor MRI images using deep convolutional neural network with fully optimized framework, Iranian Journal of Science and Technology - Transactions of Electrical Engineering 45(3) (2021), 1015–1036.

18.

Haq

A.U.

, Li

J.P.

, Khan

, et al., DACBT: Deep learning approach for classification of brain tumors using MRI data in IoT healthcare environment, Sci Rep 12(1) (2022), 15331.

19.

Chartrand

, Cheng

P.M.

, Vorontsov

, et al., Deep learning: A primer for radiologists, Radiographics 37(7) (2017), 2113–2131.

20.

Jiang

, Jiang

, Zhi

, et al., Artificial intelligence in healthcare: Past, present and future, Stroke Vasc Neurol 2(4) (2017).

21.

Qodri

K.N.

, Soesanti

and Nugroho

H.A.

, Image analysis for MRI-based brain tumor classification using deep learning, International Journal of Information Technology and Electrical Engineering (IJITEE) 5(1) (2021), 21–28.

22.

Setiawan

, Perbandinggan arsitektur convolutional neural network untuk klasifikasi fundus, Jurnal SimanteC 7(2) (2019), 49–54.

23.

Cinar

, Kaya

and Kaya

, Comparison of deep learning models for brain tumor classification using MRI images, In: 2022 International Conference on Decision Aid Sciences and Applications (DASA) [Internet] IEEE, 2022, pp. 1382–1385.

24.

Chelghoum

, Ikhlef

, Hameurlaine

and Jacquir

, Transfer learning using convolutional neural network architectures for brain tumor classification from MRI images, In: IFIP Advances in Information and Communication Technology, Springer, 2020, pp. 189–200.

25.

Murphy

and Gaillard

, MRI sequences (overview) [Internet]. Radiopaedia.org. 2015 [cited 2023 Jan 1]. Available from: https://radiopaedia.org/articles/mri-sequences-overview

26.

Yueniwati

, Pencitraan pada Tumor Otak. Universitas Brawijaya Press, 2017.

27.

Fayaz

, Torokeldiev

, Turdumamatov

, et al., An efficient methodology for brain mri classification based on dwt and convolutional neural network, Sensors 21(22) (2021), 7480.

28.

Ayan

and Ünver

H.M.

, Diagnosis of pneumonia from chest X-ray images using deep learning, In: 2019 Scientific Meeting on Electrical-Electronics & Biomedical Engineering and Computer Science (EBBT) IEEE, 2019, pp. 1–5.

29.

Ayan

and Ünver

H.M.

, Data augmentation importance for classification of skin lesions via deep learning, In: 2018 Electric Electronics, Computer Science, Biomedical Engineerings’ Meeting (EBBT) IEEE, 2018, pp. 1–4.

30.

Rajinikanth

, Raj

A.N.J.

, Thanaraj

K.P.

and Naik

G.R.

, A customized VGG19 network with concatenation of deep and handcrafted features for brain tumor detection, Applied Sciences (Switzerland) 10(10) (2020), 3429.

31.

Menze

B.H.

, Jakab

, Bauer

, et al., The Multimodal brain tumor image segmentation benchmark (BRATS), IEEE Trans Med Imaging 34(10) (2015), 1993–2024.

32.

Althnian

, AlSaeed

, Al-Baity

, et al., Impact of dataset size on classification performance: An empirical evaluation in the medical domain, Applied Sciences (Switzerland) 11(2) (2021), 1–18.

33.

Grøvik

, Yi

, Iv

, et al., Handling missing MRI sequences in deep learning segmentation of brain metastases: A multicenter study, NPJ Digit Med 4(1) (2021), 33.

34.

Chatterjee

, Nizamani

F.A.

, Nürnberger

and Speck

, Classification of brain tumours in MR images using deep spatiospatial models, Sci Rep 12(1) (2022).

35.

Roudsari

B.S.

, McKinney

, Moore

and Jarvik

, Sensitivity and specificity: Imperfect predictors of guideline utility in radiology, British Journal of Radiology 84(999) (2011), 216–220.

36.

Baba

and Jones

, T1 weighted image [Internet]. Radiopaedia.org. 2009 [cited 2023 Jan 1]. Available from: https://radiopaedia.org/articles/t1-weighted-image

37.

Haouimi

and Jones

, T2 weighted image [Internet]. Radiopaedia.org. 2009 [cited 2023 Jan 1]. Available from: https://radiopaedia.org/articles/t2-weighted-image

38.

Baba

and Niknejad

, Fluid attenuated inversion recovery [Internet]. Radiopaedia.org. 2013 [cited 2023 Jan 1]. Available from: https://radiopaedia.org/articles/fluid-attenuated-inversion-recovery

39.

Fink

K.R.

and Fink

J.R.

, Imaging of brain metastases, Surg Neurol Int 4(SUPPL4) (2013), S209–S219.

40.

Pope

W.B.

, Brain metastases: Neuroimaging, Handb Clin Neurol 49 (2018), 89–112.

41.

Gaillard

, Glioblastoma, IDH-wildtype [Internet]. Radiopaedia.org. 2008 [cited 2023 Jan 1]. Available from: https://radiopaedia.org/articles/glioblastoma-idh-wildtype

42.

Sharma

and Orton

, Brain metastases [Internet]. Radiopaedia.org. 2008 [cited 2023 Jan 1]. Available from: https://radiopaedia.org/articles/brain-metastases

43.

Widmann

, Henninger

, Kremser

and Jaschke

, MRI sequencesin head & neck radiology –state of the art, RöFo - Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren [Internet] [cited 2023 Jan 1] 189(05) (2017), 413–422. Available from: https://www.thieme-connect.com/products/ejournals/html/10.1055/s-0043-103280