Abstract
Objectives:
We compared validity of 2010 McDonald and newly proposed 2016 Magnetic Resonance Imaging in Multiple Sclerosis (MAGNIMS) criteria for dissemination in space (DIS) in predicting the conversion to clinically definite multiple sclerosis (CDMS) in patients with clinically isolated syndrome (CIS).
Methods:
Between 2006 and 2016, we enrolled 170 patients who had a first clinical event suggestive of multiple sclerosis (MS) from seven referral hospitals in Korea. Patients were classified into two groups based on the main outcome at the last follow-up: CDMS converters, who experienced a second attack, and non-converters.
Results:
Of 170 patients with mean follow-up duration of 54 months, 51% converted to CDMS. The sensitivity, specificity, accuracy, and positive and negative predictive values of 2010 McDonald criteria were 70.9%, 63.1%, 67.1%, 66.3%, and 67.9%, and those for 2016 MAGNIMS criteria were 88.4%, 46.4%, 67.7%, 62.8%, and 79.6%, respectively. When we excluded 80 patients who underwent disease-modifying therapy before the second clinical event, the specificity increased to 92.3% and 84.6%, but the sensitivity decreased to 58.8% and 82.4% for 2010 McDonald and 2016 MAGNIMS criteria, respectively.
Conclusion:
2016 MAGNIMS magnetic resonance imaging (MRI) criteria for DIS showed higher sensitivity but lower specificity than 2010 McDonald criteria in predicting conversion to CDMS in CIS patients.
Keywords
Introduction
Since 2001, magnetic resonance imaging (MRI) has been able to support a diagnosis of multiple sclerosis (MS) in patients with a clinically isolated syndrome (CIS), enabling early diagnosis and treatment. The Magnetic Resonance Imaging in Multiple Sclerosis (MAGNIMS) group, a European collaborative research network, recently proposed modified MRI criteria for the diagnosis of MS. 1 In the 2016 MAGNIMS MRI criteria for dissemination in space (DIS), optic nerve was newly included and the distinction between symptomatic and asymptomatic lesions is no longer necessary for DIS. Additionally, the 2016 MAGNIMS MRI criteria for DIS raised the number of periventricular lesions from 1 to 3 and merging the cortical and juxtacortical lesions into a single topography.
In this study, we compared the diagnostic performance of the 2010 McDonald and 2016 MAGNIMS MRI criteria for DIS in patients with CIS.1,2
Methods
From 2006 to 2016, we identified patients who visited seven referral hospitals in Korea and experienced clinical symptoms and signs suggestive of an inflammatory demyelinating disorder of the central nervous system (CNS). We enrolled patients who fulfilled the following inclusion criteria: (1) patients presenting with CIS suggestive of MS between the ages of 15 and 65 years: CIS suggestive of MS included partial myelitis, unilateral optic neuritis, brainstem, cerebellar and cerebral hemispheric syndromes without any red flags for MS;3,4 (2) neurological evaluation and initial brain MRI performed within 3 months of onset; (3) clinical follow-up until the development of clinically definite multiple sclerosis (CDMS) or at least 2 years after onset; (4) spinal cord MRI available if patients presented with spinal cord syndrome; (5) orbit MRI or visual evoked potentials (VEPs) available if patients presented with optic neuritis; and (6) negative for aquaporin-4 immunoglobulin G (AQP4-IgG). We excluded patients with neuromyelitis optica spectrum disorder (NMOSD), as defined by the new 2015 criteria for NMOSD. 5 We also excluded vascular and compressive lesions identified by clinical history with MRI findings and patients with other infectious, metabolic causes and a history of cancer and/or radiation.
Brain and spinal MRIs were performed on 1.5 or 3.0 T scanners and slice thickness varied from 3 to 7 mm. Axial fluid-attenuated inversion recovery (FLAIR) for brain MRI and/or T2-weighted fast spin echo and/or proton density for brain and spinal MRI were included in all cases.
Lesions were identified independently by two readers (J.-W.H. and S.-Y.H.) who were blinded to the final clinical outcome. In cases of a discrepancy, a final decision was made by consensus with an experienced neuroradiologist (S.-H.L.). Two sets of MRI criteria were used to evaluate DIS: the 2010 McDonald criteria and the 2016 MAGNIMS criteria.1,2 Symptomatic lesions in the brainstem and spinal cord were defined by clinically associated MRI lesion(s) with high signal intensity on T2 or FLAIR imaging, regardless of contrast enhancement.
CDMS was defined when a second attack developed with a new neurological disability. Patients were classified into two subgroups based on the clinical outcome at the last visit: CDMS converters and non-converters. The diagnostic performance of the MRI criteria was evaluated depending on the main outcome.
First, we investigated the validity of the 2016 and 2010 DIS criteria in the entire cohort. Because disease-modifying treatment (DMT) can delay or prevent the conversion to CDMS, we sub-analyzed the data after excluding 80 patients who underwent DMT before the second clinical event. We analyzed the different performances of the individual parameters of the two sets of criteria. Second, we assessed the diagnostic performance of the modified 2010 McDonald DIS criteria,6,7 which included only symptomatic brainstem and spinal cord lesions but not included optic nerve involvement and further modified 2010 DIS criteria which changed the first criterion from one or more periventricular lesions to three or more periventricular lesions (referred to as the modified 2010 DIS criteria (1PV) and the modified 2010 DIS criteria (3PV) in this study, respectively). To ensure a fair comparison between previous studies and the modified 2010 DIS criteria (1PV),6,7 we also sub-analyzed the patients with brainstem or spinal cord syndromes, including (n = 119) or excluding (n = 65) patients who received DMT before CDMS in the current cohort. Finally, we analyzed the diagnostic performance of our previously reported cohort 8 at the current time point.
Inter-observer agreement was estimated using kappa statistics. Differences between variables were assessed with Pearson’s χ2 or Fisher’s exact test.
This study was approved by the institutional review board committee at each center. A written informed consent was obtained from all patients.
Results
Demographics
In total, 170 (109 women, 61 men) patients presenting with CIS suggestive of MS were enrolled. The mean age at onset was 32 (range, 15–62) years and the mean follow-up duration was 54 (range, 7–148) months. The first clinical manifestations were spinal cord syndrome in 36%, brainstem syndrome in 34%, unilateral optic neuritis in 21%, cerebral syndrome in 7%, and multiregional syndrome in 2%. Of the 170 patients, 86 (51%) converted to CDMS after a mean of 17 (range, 1–94) months from CIS onset, despite 80 (47%) patients underwent DMT before a second attack: interferon-β in 72, cladribine followed by interferon-β in 1, glatiramer acetate in 4, teriflunomide in 1, fingolimod in 1, and laquinimod in 1. At baseline MRI, 139 (82%) patients had abnormal brain lesions regardless of symptoms and 88 of 125 (70%) for whom spinal cord MRIs were available had abnormal spinal cord lesions. Of 15 and 68 for whom orbit MRIs and VEPs were available, 14 (93%) and 38 (56%) showed abnormalities suggestive of optic nerve lesions, respectively. All of the 22 optic neuritis patients without orbit MRIs showed VEP abnormalities.
The diagnostic performances of the 2010 McDonald and 2016 MAGNIMS DIS criteria
The 2010 McDonald DIS criteria were met in 92 (54%) and the 2016 MAGNIMS DIS criteria in 121 (71%) patients with CIS. Inter-observer agreement between two readers was excellent for both the 2010 (κ = 0.85) and the 2016 (κ = 0.88) criteria. The sensitivity, specificity, accuracy, and positive and negative predictive values for the 2010 DIS criteria were 70.9%, 63.1%, 67.1%, 66.3%, and 67.9%, and those for the 2016 DIS criteria were 88.4%, 46.4%, 67.7%, 62.8%, and 79.6%, respectively (Table 1). When we sub-analyzed the data after excluding 80 patients who received DMT before a second clinical event, the specificity increased to 92.3% and 84.6%, whereas the sensitivity decreased to 58.8% and 82.4% for the 2010 and the 2016 DIS criteria, respectively. In another sub-analysis with patients who underwent MRI on 3.0 T scanners, 2016 MAGNIMS MRI criteria for DIS still revealed higher sensitivity but lower specificity than 2010 McDonald criteria in predicting conversion to CDMS in CIS patients (supplementary table 1).
The validity of the 2010 McDonald and the 2016 MAGNIMS MRI criteria for DIS in the Korean cohort.
MAGNIMS: Magnetic Resonance Imaging in Multiple Sclerosis; MRI: magnetic resonance imaging; DIS: dissemination in space; TP: true positive; FP: false positive; FN: false negative; TN: true negative; PPV: positive predictive value; NPV: negative predictive value; PLR: positive likelihood ratio; NLR: negative likelihood ratio; ( ): 95% confidence interval; DMT: disease-modifying treatment.
The diagnostic performances of individual parameters of the 2010 and 2016 DIS criteria
The performances of individual parameters of the 2010 and 2016 DIS criteria are presented in Table 2. The performances of the criteria of symptomatic spinal cord syndrome or involvement of the optic nerve from the 2016 DIS criteria did not differ significantly between CDMS converters and non-converters. Among the individual parameters, the lowest sensitivity was observed with the criterion of asymptomatic spinal cord syndrome, followed by asymptomatic infra-tentorial syndrome from the 2010 DIS criteria. The lowest specificity was seen for the criterion of optic nerve involvement, followed by symptomatic spinal cord syndrome from the 2016 DIS criteria. The same tendencies were observed after excluding patients who underwent DMT before developing CDMS.
Performance of individual MRI parameters of the 2010 and 2016 DIS criteria.
MRI: magnetic resonance imaging; DIS: dissemination in space; CDMS: clinically definite multiple sclerosis; PPV: positive predictive value; NPV: negative predictive value; PV: periventricular; JC: juxtacortical; IT: infra-tentorial; SC: spinal cord; ON: optic nerve; DMT: disease-modifying treatment.
The diagnostic performance of modified 2010 DIS criteria
When we applied the modified 2010 criteria (Table 3), which include symptomatic brainstem/spinal cord syndrome, the sensitivity, specificity, accuracy, and positive and negative predictive values for the modified 2010 DIS criteria (1PV) were 87.2%, 45.2%, 66.5%, 62.0%, and 77.6%, and those for the modified 2010 DIS criteria (3PV) were 80.2%, 51.2%, 65.9%, 62.7%, and 71.7%, respectively. After excluding DMT-treated patients, the specificity increased to 79.5% and 87.2%, but the sensitivity decreased to 82.4% and 72.6%, with the modified 2010 DIS (1PV) and the modified 2010 DIS criteria (3PV), respectively. In comparison of four criteria in the current cohort (Tables 1 and 3), the sensitivity was highest for the 2016 DIS criteria, followed by the modified 2010 DIS criteria (1PV), the modified 2010 DIS criteria (3PV), and the 2010 DIS criteria. In contrast, the specificity was highest for the 2010 DIS criteria, followed by the 2010 modified DIS criteria (3PV), the 2016 DIS criteria, and the 2010 modified DIS criteria (1PV).
The validity of the 2010 McDonald MRI and the modified 2010 McDonald MRI criteria for DIS in the previous and current studies.
MRI: magnetic resonance imaging; DIS: dissemination in space; TP: true positive; FP: false positive; FN: false negative; TN: true negative; PPV: positive predictive value; NPV: negative predictive value; PLR: positive likelihood ratio; NLR: negative likelihood ratio; ( ): 95% confidence interval; PV: periventricular; DMT: disease-modifying treatment; CDMS: clinically definite multiple sclerosis.
For fair comparison to the previous studies (Table 3), when we applied the modified 2010 DIS criteria (1PV) in 119 patients with only brainstem/spinal cord syndromes, the sensitivity, specificity, accuracy, and positive and negative predictive values were 93.3%, 44.1%, 68.9%, 62.9%, and 86.7%. In 65 patients presenting with only brainstem/spinal cord syndromes who did not receive DMT before CDMS, the sensitivity, specificity, accuracy, and positive and negative predictive values of the modified 2010 DIS criteria (1PV) were 91.7%, 79.3%, 86.2%, 84.6%, and 88.5%, respectively.
Discussion
The 2016 MAGNIMS MRI criteria for DIS showed higher sensitivity (88.4% vs 70.9%) but lower specificity (46.4% vs 63.1%) than the 2010 McDonald DIS criteria in predicting the conversion to CDMS in patients with CIS. After excluding patients who received DMT before developing CDMS, the difference between the specificity of the two sets of criteria decreased (84.6% vs 92.3%), but lower specificity was still observed with the 2016 DIS criteria versus the 2010 DIS criteria. To minimize heterogeneity of MRI, we sub-analyzed the patients who underwent MRI on 3.0 T scanners and the results remained the same. The different performances of the two sets of criteria could be explained by the low specificity of the optic nerve lesion and the symptomatic spinal cord lesion criteria in the 2016 DIS criteria, and the low sensitivity of the asymptomatic spinal cord lesion and infra-tentorial lesion criteria in the 2010 DIS criteria. However, this speculation needs to be confirmed by further investigation with full MRI assessment of spinal cord and optic nerve.
Previously, we reported the validity of the 2010 DIS criteria in 94 Korean patients with CIS, 8 all of whom were also included in this study (Table 4). Of these 94 patients, 3 patients who fulfilled the criteria later converted to CDMS during further follow-up, which lowered the false positivity. In comparing the previous and the current results in the entire cohort, the specificity (60.0% vs 63.1%) increased. These differences could be explained by the longer follow-up duration (mean, 37 vs 54 months) and larger sample size (n = 94 vs 170).
The comparison of validity of the 2010 McDonald MRI criteria for DIS in previous studies.
DIS: dissemination in space; TP: true positive; FP: false positive; FN: false negative; TN: true negative; PPV: positive predictive value; NPV: negative predictive value; PLR: positive likelihood ratio; NLR: negative likelihood ratio; ( ): 95% confidence interval.
The sensitivity (61.6%–94% vs 70.9%) and specificity (52.9%–73% vs 63.1%) of the 2010 DIS criteria in this study were comparable to those of previous Western studies (Table 4).6,9–14 After excluding DMT-treated patients, the sensitivity of the 2010 DIS criteria in our cohort was lower than that of previous Western results (58.8% vs 76.6%–85.9%), but the specificity of the 2010 DIS criteria was higher than that of previous results (92.3% vs 59.4%–66.7%).12–14 One factor contributing to the difference is that a high proportion of patients (47%) in our cohort received DMT before a second event compared with that in the previous Western study (4.6%). 14 In general, those patients are likely to fulfill the DIS criteria for MS and prone to convert CDMS.
In comparison of four criteria in the current cohort, the sensitivity was highest for the 2016 DIS criteria, followed by the modified 2010 DIS criteria (1PV), the modified 2010 DIS criteria (3PV), and the original 2010 DIS criteria. The order of specificity was almost opposite: highest for the original 2010 DIS criteria, followed by the 2010 modified DIS criteria (3PV), the 2016 DIS criteria, and the 2010 modified DIS criteria (1PV).
In a recent study that evaluated the modified 2010 DIS criteria (1PV) in 99 patients with brainstem/spinal cord syndromes, the sensitivity (82.6% vs 76.0%) was higher but specificity (49.1% vs 54.7%) was lower than that of the original 2010 DIS criteria. 6 In this study, the consistent results with higher sensitivity (93.3% vs 70.0%) but lower specificity (44.1% vs 71.2%) in the modified 2010 DIS criteria (1PV) than that of the original 2010 DIS criteria were observed in 119 patients with brainstem/spinal cord syndromes (Table 3). Of note is that both studies included patients who received DMT before a second event. In another study that evaluated the modified 2010 DIS criteria in 30 treatment-naive patients with brainstem/spinal cord syndromes, the sensitivity (87% vs 73%) of the modified 2010 DIS criteria (1PV) was higher than that of the original 2010 DIS criteria without compromising the specificity (73% vs 73%). 7 However, in the current 65 patients with brainstem/spinal cord syndromes who did not receive DMT before developing CDMS, the sensitivity was higher (91.7% vs 58.3%), but the specificity was lower (79.3% vs 96.6%) with the modified 2010 DIS criteria (1PV) versus the original 2010 DIS criteria. Further validation is needed to assess the 2016 DIS criteria as well as the modified 2010 DIS criteria.
Limitations of this study include the following: (1) only DIS and not dissemination in time (DIT) was evaluated; (2) its retrospective nature; (3) the limited availability of initial spinal MRI, although previous studies have shown that the inclusion of spinal cord lesions only improved sensitivity and specificity slightly; 14 (4) limited availability of initial orbit MRI, but the lack of orbit MRI was substituted with VEP; (5) a double inversion recovery sequence for cortical lesions was not performed; (6) the recommended MRI protocols were not performed in full, although the validity of the MS criteria in this study was comparable to that in previous studies, and for the same reason, this study reflected real-world practice; (7) not all patients were evaluated for AQP4-IgG at the initial manifestation; and (8) potential unintentional selection bias because patients were enrolled from referral centers. These altogether warrant further longitudinal prospective study including full neuraxis MRI with united protocol.
Nevertheless, based on a well-documented multicenter cohort of patients with CIS through careful exclusion of NMOSD, we provide evidence that the 2016 MAGNIMS DIS criteria have higher sensitivity but lower specificity than the 2010 McDonald DIS criteria to predict conversion to CDMS in CIS patients. Further studies should be performed in larger cohorts of diverse ethnic groups, with prospective designs including full evaluation of spinal cord and optic nerve MRI, particularly to assess the specificity of the 2016 MAGNIMS DIS criteria combined with additional evaluations of the modified 2010 McDonald DIS criteria including criterion of three or more periventricular lesions.
Footnotes
Acknowledgements
J.-W.H. and H.J.K. had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. J.-W.H., H.J.K., and S.-H.K. contributed to study concept and design. J.-W.H., H.J.K., and S.-H.K. contributed to drafting of the manuscript. All authors contributed to acquisition, analysis, and interpretation of data. J.-W.H., H.J.K., and S.-H.K. contributed to statistical analysis. All authors contributed to critical revision of the manuscript for important intellectual content.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: J.-W.H., S.-Y.H., W.K., M.S.P., S.-W.A., J.-Y.C., B.-J.K., S.-H.L., and S.-H.K. report no financial disclosures. H.J.K. has lectured, consulted, and received honoraria from Bayer Schering Pharma, Biogen, Genzyme, HanAll BioPharma, MedImmune, Merck Serono, Novartis, Teva-Handok, and UCB; received a grant from the Ministry of Science, ICT & Future Planning; and accepted research funding from Genzyme, Kael-GemVax, Merck Serono, Teva-Handok, and UCB; serves on a steering committee for MedImmune; and is a co-editor for the Multiple Sclerosis Journal—Experimental, Translational, and Clinical and an associated editor for the Journal of Clinical Neurology.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
