Use of Artificial Intelligence Deep Learning to Determine the Malignant Potential of Pancreatic Cystic Neoplasms With Preoperative Computed Tomography Imaging

Abstract

Background

Society consensus guidelines are commonly used to guide management of pancreatic cystic neoplasms (PCNs). However, downsides of these guidelines include unnecessary surgery and missed malignancy. The aim of this study was to use computed tomography (CT)-guided deep learning techniques to predict malignancy of PCNs.

Materials and Methods

Patients with PCNs who underwent resection were retrospectively reviewed. Axial images of the mucinous cystic neoplasms were collected and based on final pathology were assigned a binary outcome of advanced neoplasia or benign. Advanced neoplasia was defined as adenocarcinoma or intraductal papillary mucinous neoplasm with high-grade dysplasia. A convolutional neural network (CNN) deep learning model was trained on 66% of images, and this trained model was used to test 33% of images. Predictions from the deep learning model were compared to Fukuoka guidelines.

Results

Twenty-seven patients met the inclusion criteria, with 18 used for training and 9 for model testing. The trained deep learning model correctly predicted 3 of 3 malignant lesions and 5 of 6 benign lesions. Fukuoka guidelines correctly classified 2 of 3 malignant lesions as high risk and 4 of 6 benign lesions as worrisome. Following deep learning model predictions would have avoided 1 missed malignancy and 1 unnecessary operation.

Discussion

In this pilot study, a deep learning model correctly classified 8 of 9 PCNs and performed better than consensus guidelines. Deep learning can be used to predict malignancy of PCNs; however, further model improvements are necessary before clinical use.

Keywords

pancreatic cystic neoplasms deep learning predictive modeling image analysis

Introduction

Pancreatic cystic neoplasms (PCNs) are rare tumors of the pancreas and encompass approximately .5% of all pancreatic neoplasms.¹ PCN is a broad term encompassing multiple histologically different subtypes including serous cystic tumors, mucinous cystic neoplasms (MCNs), intraductal papillary mucinous neoplasms (IPMNs), and solid pseudopapillary neoplasms. While most PCNs are benign, mucinous cystic neoplasm, IPMNs, and solid pseudopapillary tumors may harbor risk for malignancy.² With increasing identification of PCNs on incidental imaging^3,4 and inconsistent diagnostic accuracy,⁵ there has been interest in creation of schema to aid management of PCNs. In general, surveillance is recommended for serous cystic tumors and resection recommended for main-branch IPMNs and solid pseudopapillary tumors due to risk of malignancy or malignant transformation. However, for intermediate lesions such as MCNs or side-branch IPMNs, most clinicians follow consensus guidelines for recommendations regarding resection. The most prominent of these are the 2015 American Gastroenterological Association (AGA) guidelines⁶ (for asymptomatic pancreatic neoplastic cysts such as serous cystic tumors and MCNs) and the Fukuoka guidelines⁷ (for IPMNs). The guidelines describe high-risk and worrisome criteria of PCNs based on preoperative imaging and clinical symptoms. Retrospective studies have demonstrated high specificity; however, these guidelines have poor sensitivity, and using guidelines alone would have led to missing approximately 50% of advanced neoplasia in 1 series.^8,9 This results in nearly 20% of patients with a benign lesion⁵ undergoing operations with high morbidity.¹⁰ Given that existing guidelines are insufficient for identification of high-risk features of PCNs, an opportunity exists to leverage innovative new technologies to aid in diagnosis of potentially malignant lesions.

Deep learning has emerged as a promising technology for image analysis in medicine.^11-14 Deep learning is an artificial intelligence (AI), machine learning technique that uses a “neural network” to make predictions based on discrete inputs. The model uses an iterative process to take output predictions and reanalyze the model to further improve accuracy, continually improving with each new data point. The advantage of deep learning over other machine learning techniques is the use of “representation learning,” whereby the AI model identifies relationships within a data set that human operators cannot perceive.¹¹ Therefore, a deep learning model can have a pictorial input, broken down into individual pixels, and find relationships between the orientation and intensity of these pixels. There is growing interest in the use of deep learning in medicine with over 500 articles published since 2015.¹² The majority of these articles focus on the use of deep learning for classification and detection of objects in radiographic images. A study of the use of deep learning for identification of prostate cancer on histopathological slides demonstrated 99% accuracy for determination of malignancy.¹³ A deep learning model was used to identify breast cancer on mammography alone with over 85% accuracy.¹⁴ Deep learning in medicine is a growing field with boundless potential for clinical application in the right context.

Deep learning is a powerful and increasingly utilized tool that has previously been used to accurately identify malignancy from image analysis only. Considering the uncertainty in preoperative diagnosis of PCNs, a deep learning model may be able to find difficult-to-characterize patterns in images that can aid in diagnosis of PCNs. The aim of this study was to compare an AI deep learning model to existing guidelines in the diagnosis of high-grade dysplasia or malignancy in PCNs. Our hypothesis is that a deep learning model will have higher accuracy than the Fukuoka guidelines, which are the current standard of care.

Patients and Methods

Patient Selection and Image Analysis

Patients were identified using our institution’s Research Electronic Data Capture electronic database between March 2008 and March 2017. Institutional review board approval was received (IRB#04-18-14E), and as this is a retrospective study with deidentified data, informed consent was not required. The inclusion criteria included preoperative diagnosis of IPMN, mucinous cystic neoplasm, or serous cystic neoplasm. Diagnoses of pancreatic pseudocysts were excluded from analysis. Patients were excluded if there was no preoperative computed tomography (CT) scan available for review. Additionally, patients included had a pathology report available for review with pathologic diagnosis of their lesion. Characteristics of patients were determined through retrospective record review and included age, sex, operation performed, preoperative diagnosis, and postoperative pathologic diagnosis. Data are reported as either median with interquartile range (IQR) or counts with percentage.

The axial image with the largest cross-sectional lesion diameter on preoperative CT scan was collected for each patient and deidentified. All structures surrounding the lesion were removed from the image to isolate the lesion. The isolated lesion was then centered on a 150-pixel × 150-pixel white background for standardization (Figure 1). Using pathologic diagnosis, each image was assigned a binary outcome of advanced neoplasia or benign for purposes of model creation. Advanced neoplasia was defined as adenocarcinoma or IPMN with high-grade dysplasia, while benign was defined as IPMN with intermediate- or low-grade dysplasia or serous cystadenoma. A “training set” was created with 66% of randomly selected lesion images, and a “testing set” of 33% of images was created for model testing.

Figure 1.

Example of images used to train the deep learning model. The largest cross-sectional diameter of the lesion in the axial plane was isolated and placed on a 150-pixel × 150-pixel white background.

Deep Learning Model

The deep learning model utilized in the experiment was modeled after the LeNet architecture which included 3 convolutional layers: 1 flattened layer and 2 fully connected layers. Batch sizes for model analysis were set to 3. Input classes were designated during training as advanced neoplasia status labeled as “yes” or “no.” Overall, 20% of the data were utilized within the convolutional neural network (CNN) for internal validation purposes. Images for both testing and training were standardized by sizing to 128 × 128 with 3 color channels for analysis.

The neural network graph parameters were specified as follows: the first 2 convolutional layers were given a filter size of 3 with 32 filters. Convolutional layer 3 was given a filter size of 3 with 64 filters. The fully connected layer size was subsequently set to 128 based on these parameter specifications. The activation function utilized in the first fully connected layer is the rectified linear unit. The learning rate for the CNN was set to .0001 to ideally minimize potential overfitting errors associated with model construction. The model was trained for 4000 iterations. Further CNN training beyond the 4000 iterations was not performed due to a reported loss in accuracy and higher associated cost function associated with overtraining the model.

After completing CNN training and validation, external validation was accomplished with a separate sample of images completely unknown to the deep learning model. Classification percentages for these validation images were calculated by utilizing the trained weights from the CNN. These values were recorded with the corresponding classification label (advanced neoplasia or benign) and later evaluated for model discriminative ability through receiver operating characteristic value.

Model Validation and Comparison to Established Guidelines

The 9 “testing set” images were analyzed by the model, and each image was given a percentage chance of advanced neoplasia. If the model’s predicted chance of advanced neoplasia was greater than 50%, the model was considered to have predicted advanced neoplasia. If the model’s predicted chance of advanced neoplasia was less than 50%, the model was considered to have predicted benign pathology. These predictions were compared to the actual pathology report to determine model accuracy. Each lesion was then evaluated by an attending hepato-pancreato-biliary surgeon to determine whether it met “high risk” or “worrisome risk” criteria based on the Fukuoka guidelines.⁷ A guideline prediction of “high risk” was considered a prediction of advanced neoplasia, while a prediction of “worrisome” was considered a prediction of benign pathology. The model predictions were compared with the Fukuoka guideline predictions. Data were analyzed with STATA (StataCorp LLC, College Station, Texas, USA).

Results

In total, 27 patients met the inclusion criteria. All patients in this study underwent pancreatic resection, with 11 (40.7%) undergoing left pancreatectomy and 16 (59.3%) undergoing pancreaticoduodenectomy. Median age was 69 (IQR 57.8-73) years with 57.7% male and 42.3% female. Postoperative pathology was reviewed for all patients. In total, 9 (33.3%) patients had evidence of advanced neoplasia consisting of 7 (25.9%) patients with invasive pancreatic adenocarcinoma and 2 (7.4%) patients with IPMN with high-grade dysplasia. Eighteen patients had benign pathology including 5 (18.5%) serous cystadenoma and 13 (48.1%) IPMN with intermediate- or low-grade dysplasia. Surgery was indicated in ten patients (37.0%) deemed “high risk” and 15 (55.6%) deemed “worrisome” based on Fukuoka guidelines. Of the remaining 5 (18.5%) patients, surgery was indicated for symptoms of abdominal pain (4; 14.8%) or recurrent pancreatitis (1; 3.7%).

Model predictions for all tested lesions are listed in Table 1, and comparison to Fukuoka guidelines is demonstrated in Table 2. The trained deep learning model correctly identified the pathology of 8 of 9 lesions, while the Fukuoka guidelines correctly identified the pathology of 6 of 9 lesions. Of the 3 malignant lesions, the deep learning model correctly predicted advanced neoplasia in all 3, while the Fukuoka guidelines only classified 2 as advanced neoplasia. Of the 6 benign lesions, the deep learning model correctly predicted benign pathology in 5, while Fukuoka guidelines only classified 4 as benign. If these patients were treated according to consensus guidelines, 2 would have unneeded operations and 1 would not have been treated for advanced neoplasia. However, if the deep learning model was followed, there only would be 1 unneeded operation and no advanced neoplasia would have been missed.

Table 1.

Model Predictions for All Tested Lesions.

Advanced neoplasia	Model prediction	Fukuoka prediction^a
Lesion 1	55.5% advanced neoplasia	Advanced neoplasia
Lesion 2	96.0% advanced neoplasia	Benign
Lesion 3	86.7% advanced neoplasia	Advanced neoplasia
Total	3/3 correct	2/3 correct

Benign	Model prediction	Fukuoka prediction^a
Lesion 4	96.2% benign	Benign
Lesion 5	87.5% benign	Advanced neoplasia
Lesion 6	7.1% benign	Advanced neoplasia
Lesion 7	99.9% benign	Benign
Lesion 8	92.1% benign	Benign
Lesion 9	54.9% benign	Benign
Total	5/6 correct	4/6 correct

^aFukuoka classification as “high risk” was considered a prediction of advanced neoplasia and classification as “worrisome” was considered a prediction of benign.

Table 2.

Comparison of Deep Learning Model Predictions with Fukuoka Guidelines.

Test lesion	Model prediction	Fukuoka guidelines^a	Pathology^b
Lesion 1	Advanced neoplasia	Advanced neoplasia	Advanced neoplasia
Lesion 2	Advanced neoplasia	Benign	Advanced neoplasia
Lesion 3	Advanced neoplasia	Advanced neoplasia	Advanced neoplasia
Lesion 4	Benign	Benign	Benign
Lesion 5	Benign	Advanced neoplasia	Benign
Lesion 6	Advanced neoplasia	Advanced neoplasia	Benign
Lesion 7	Benign	Benign	Benign
Lesion 8	Benign	Benign	Benign
Lesion 9	Benign	Benign	Benign

^aFukuoka classification as “high risk” was considered a prediction of advanced neoplasia and classification as “worrisome” was considered a prediction of benign.

^bAdvanced neoplasia pathology included adenocarcinoma and intraductal papillary mucinous neoplasms (IPMNs) with high-grade dysplasia, while benign pathology included IPMN with intermediate- or low-grade dysplasia and serous cystadenoma.

Discussion

In this pilot study for prediction of advanced neoplasia from preoperative CT imaging, a deep learning model was superior to consensus guidelines. The deep learning model was able to correctly classify 8 of 9 high-risk PCNs, while consensus guidelines were correct 6 of 9 times. In terms of possible patient outcomes, following predictions from the deep learning model would have resulted in 1 unnecessary operation, while following consensus guidelines would have resulted in 2 unnecessary operations and 1 missed neoplasm.

Artificial intelligence and deep learning are exciting new technologies that have shown promise in clinical applications, with over 300 publications since 2015.¹² Many applications of deep learning thus far have been focused on oncology to predict malignant behavior of tissue samples,¹⁵ histology slides,¹³ or radiographic images.¹⁴ Litjens et al¹³ demonstrated that a deep learning model was able to identify specific areas of cancerous tissue in prostate cancer biopsies with an area under the curve (AUC) of .999. Further analysis demonstrated that a different deep learning model was able to identify areas of cancerous tissue in breast sentinel lymph node samples with an AUC of .90.¹³ Another study evaluated the use of a deep learning model for identification of suspicious microcalcifications in diagnostic mammography and compared it to other machine learning techniques.¹⁴ The authors demonstrated that the deep learning model was superior to other techniques and was able to identify microcalcifications with an accuracy of 82%.¹⁴ These studies demonstrate the power of deep learning techniques to aid clinicians in cancer diagnoses. For a disease such as PCN, where the current standard of diagnosis is insufficient,⁵ new approaches are needed.

This study is the first to use preoperative CT imaging to predict dysplastic and malignant biology of a PCN. Previous studies have used deep learning models incorporating pancreatic fine needle aspirate (FNA) analysis to classify PCNs.¹⁵ Kurita et al¹⁵ retrospectively reviewed data from patients undergoing endoscopic ultrasound (EUS) and FNA of PCNs over a 10-year period. Using carcinoembryonic antigen (CEA) value, cancer antigen 19-9, fluid amylase, patient sex, cyst characteristics, and cytology, the authors created a deep learning CNN model. The output prediction was malignant or benign, based on final pathology or from subsequent clinical follow-up. This study found that a deep learning model had an AUC of .966 and accuracy of 92.9% in high-risk lesions, which was more accurate than using CEA or cytology alone. While the deep learning model had high accuracy compared to other measures, the patient population was limited to patients undergoing diagnostic EUS-FNA, introducing a selection bias toward high-risk lesions. The study did not mention the indications for EUS-FNA for patients in the study; however, it is likely that the group of patients with high-risk features from consensus guidelines is overrepresented in this study. This model has limitations in broad applicability because it may not represent all patients with PCNs and would require patients to undergo EUS-FNA which is costly and may place patients at risk of procedure-related complications.

Another study has used preoperative magnetic resonance imaging (MRI) to create a deep learning model to classify malignant behavior of PCNs for patients undergoing eventual surgical resection.¹⁶ The authors obtained T2-weighted and postcontrast T1-weighted MRI images of the pancreatic region to use in a CNN. Images were used for both model creation and model testing, rather than using separate “training” and “testing” sets. The authors found that the AUC of the deep learning model (.78) was similar to that of AGA (.76) and Fukuoka (.77) consensus guidelines. While this study demonstrated the feasibility of deep learning to predict malignant biology from preoperative imaging, accuracy was no better than currently existing and simple consensus guidelines. Additionally, by using the image data set for both model creation and model validation, there is a risk of overfitting the model to the data set and limiting external validity to new data.

There are several important limitations of our study. First, this is a pilot study with a small sample size; therefore, we cannot make broad statements about the performance of our model. However, initial results from our model are quite promising, and use of techniques such as image modulation may be help increase the number of images for input into the model.¹⁷ A characteristic of deep learning models is that predictive ability improves through sequential learning; therefore, as the sample size grows, the model will improve. Second, images obtained in this study were from a single axial image of the largest diameter of the target lesion without surrounding pancreatic tissue or further images of the lesion. Therefore, our model lacks information regarding surrounding pancreatic structures, location in the pancreas, or other features of the lesion from other axial images. Further model construction with use of multiple images of the pancreas including the lesion may improve model accuracy. Third, the retrospective nature of this study adds an inherent selection bias, and there may be overrepresentation of patients with high-risk features. As this study includes only patients who had surgical resection and pathologic diagnosis, patients with low-risk features undergoing surveillance were excluded. Therefore, there is a risk that the model was biased toward lesions with high-risk features. Finally, we chose a deep learning prediction cutoff of 50%; however, this may not be reflective of choices made by surgeons. This was a pilot study that demonstrates the potential of deep learning image analysis for characterizing malignant potential of PCNs and may not be ready for clinical application without further refinement.

Future studies should be directed toward using techniques to increase the number of images for model creation¹⁷ and increasing the number of patients evaluated. Considering that a deep learning model increasingly improves with more input data, this will likely lead to improvement in the model’s predictive ability. As PCNs are uncommon, multi-institutional collaboration may help with creation of a larger cohort of patients to evaluate. Additionally, these further studies should investigate the use of more computationally complex models that can incorporate clinical information, more CT images including surrounding structures, and standardized CT protocols with higher resolution (ie 1.5-mm slices vs. 5-mm slices).

In conclusion, this pilot study with a small group of patients (n = 27) demonstrated that a deep learning model based only on preoperative CT imaging was better able to predict advanced neoplasia than generally accepted consensus guidelines. Further study should be directed toward creation of a larger, more sophisticated model and ultimately prospective validation of a model for prediction of malignant behavior of PCNs.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Limaiem

Khalfallah

Ben

Bouraoui

Lahmar

Mzabi

. Pancreatic cystic neoplasms. N Am J Med Sci. 2014;6(8):413-417. doi:10.4103/1947-2714.139298

Scheiman

Hwang

Moayyedi

. American gastroenterological association technical review on the diagnosis and management of asymptomatic neoplastic pancreatic cysts. Gastroenterology. 2015;148(4):824-848.e22. doi:10.1053/j.gastro.2015.01.014

de Jong

Nio

Hermans

, et al. High prevalence of pancreatic cysts detected by screening magnetic resonance imaging examinations. Clin Gastroenterol Hepatol. 2010;8(9):806-811. doi:10.1016/j.cgh.2010.05.017

Laffan

Horton

Klein

, et al. Prevalence of unsuspected pancreatic cysts on MDCT. AJR Am J Roentgenol. 2008;191(3):802-807. doi:10.2214/AJR.07.3340

Cho

Russ

Loeffler

, et al. Preoperative classification of pancreatic cystic neoplasms: The clinical significance of diagnostic inaccuracy. Ann Surg Oncol. 2013;20(9):3112-3119. doi:10.1245/s10434-013-2986-6

Vege

Ziring

Jain

Moayyedi

; Clinical Guidelines Committee; American Gastroenterology Association. American gastroenterological association institute guideline on the diagnosis and management of asymptomatic neoplastic pancreatic cysts. Gastroenterology. 2015;148(4):819-822. doi:10.1053/j.gastro.2015.01.015

Tanaka

Fernández-del Castillo

Kamisawa

, et al. Revisions of international consensus Fukuoka guidelines for the management of IPMN of the pancreas. Pancreatology. 2017;17(5):738-753. doi:10.1016/j.pan.2017.07.007

Goldberg

Thiruvengadam

, et al. Comparing american gastroenterological association pancreatic cyst management guidelines with Fukuoka consensus guidelines as predictors of advanced neoplasia in patients with suspected pancreatic cystic neoplasms. J Am Coll Surg. 2016;223(5):729-737.e1. doi:10.1016/j.jamcollsurg.2016.07.011

Kaimakliotis

Riff

Pourmand

, et al. Sendai and fukuoka consensus guidelines identify advanced neoplasia in patients with suspected mucinous cystic neoplasms of the pancreas. Clin Gastroenterol Hepatol. 2015;13(10):1808-1815. doi:10.1016/j.cgh.2015.03.017

10.

Lyman

Passeri

Cochran

, et al. Discrepancy in postoperative outcomes between auditing batabases: A NSQIP comparison. Am Surg. 2018;84(8):1294-1298.

11.

Lecun

Bengio

Hinton

. Deep learning. Nature. 2015;521(7553):436-444. doi:10.1038/nature14539

12.

Litjens

Kooi

Bejnordi

, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60-88. doi:10.1016/j.media.2017.07.005

13.

Litjens

Sánchez

Timofeeva

, et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep. 2016;23(6):26286. doi:10.1038/srep26286

14.

Wang

Yang

Cai

Tan

Jin

. Discrimination of breast cancer with microcalcifications on mammography by deep learning. Sci Rep. 2016;7(6):27327. doi:10.1038/srep27327

15.

Kurita

Kuwahara

Hara

, et al. Diagnostic ability of artificial intelligence using deep learning analysis of cyst fluid in differentiating malignant from benign pancreatic cystic lesions. Sci Rep. 2019;9(1):6893. doi:10.1038/s41598-019-43314-3

16.

Corral

Hussein

Kandel

Bolan

Bagci

Wallace

. Deep learning to classify intraductal papillary mucinous neoplasms using magnetic resonance imaging. Pancreas. 2019;48(6):805-810. doi:10.1097/MPA.0000000000001327

17.

Lee

Kim

Shin

. Feature image-based automatic modulation classification method using CNN algorithm. Paper presented at: 1st international conference on artificial intelligence in information and communication, ICAIIC 2019 February 11-13, 2019; Okinawa, Japan:560-563. doi:10.1109/ICAIIC.2019.8669002