The clinical value of the patient-reported multiple sclerosis neuropsychological screening questionnaire

Abstract

Background:

Cognitive problems are difficult to identify in patients with multiple sclerosis (MS).

Objective:

To investigate the clinical applicability of the patient-reported MS Neuropsychological Screening Questionnaire (MSNQ-P).

Methods:

Cut-off scores were determined to differentiate between cognitively impaired (n = 90), mildly cognitively impaired (n = 115), and cognitively preserved (n = 147) MS patients using receiver operating characteristic analyses.

Results:

We could not define specific and sensitive cut-off scores. Higher scores (≥27) did indicate cognitive impairment. Among patients with a higher education, lower scores (<12) indicated intact cognition.

Conclusion:

Certain scores can indicate intact or impaired cognitive function. Still, MSNQ-P scores should be interpreted with caution.

Keywords

Multiple sclerosis cognitive impairment subjective cognitive complaints screening self-report

Introduction

Cognitive problems are prevalent and disabling in patients with multiple sclerosis (MS).¹ Early detection is therefore highly relevant. However, cognitive problems are difficult to detect during routine neurological care due to their subtle nature and weak relation with physical disabilities.¹ Neuropsychological evaluation is a useful tool to assess cognitive problems, and concise neuropsychological batteries are available for MS patients (e.g. Rao’s Brief Repeatable Neuropsychological Battery (BRB-N)² and the Brief International Cognitive Assessment for MS (BICAMS)).³ However, it is not feasible for all healthcare facilities to routinely perform neuropsychological evaluations during neurological care visits. In order to identify patients who may benefit most from neuropsychological evaluation, it is essential to know the clinical value of patient-reported cognitive complaints.

A brief self-report questionnaire to identify cognitive and neurobehavioral problems is the MS Neuro-psychological Screening Questionnaire (MSNQ).⁴ The informant version of the MSNQ (MSNQ-I) is considered a reliable cognitive screening tool,^4–7 but informants do not generally accompany MS patients at routine neurological care visits. The clinical value of the patient-reported MSNQ (MSNQ-P) varied across studies,^4–7 which may be due to small sample sizes or differences in cognitive classification methods. Importantly, psychological factors seem to play an important role in patient-reported cognitive complaints and should therefore be taken into account.^4–7

This study aimed to extend on previous work by investigating the clinical applicability of the MSNQ-P.

Materials and methods

Participants

We selected 352 patients from the Amsterdam MS cohort (65.9% female, age: 50.7 ± 11.8 years, disease duration: 16.8 ± 9.1 years and phenotype: clinically isolated syndrome (6.0%), relapsing–remitting (58.5%), secondary progressive (20.2%), primary progressive (10.5%) or unknown (4.8%)). The patients (1) completed the MSNQ-P (≤1 missing item), (2) had scores on all cognitive domains of the neuropsychological examination and (3) their visit was not included in our previous study⁶ to ensure independence between studies. A total of 88 matched healthy controls (HCs) were also included.

Outcomes

The MSNQ-P measured subjective cognitive complaints⁴ and is validated in a Dutch MS population.⁶ Scores range from 0 to 60 and higher scores indicate more cognitive complaints. The Hospital Anxiety and Depression Scale measured anxiety and depression,⁸ the Checklist Individual Strength-20-R measured fatigue,⁹ and the Expanded Disability Status Scale indicated neurological disability. An extended version of the BRB-N measured cognitive functioning,² including verbal memory, visuospatial memory, information processing speed, attention, working memory, executive function and verbal fluency. Scores were corrected for age, education and gender and transformed into domain-specific z-scores based on HC scores. Patients were classified as (1) cognitively impaired (CI; ≥2 SDs below HCs on ≥2 domains), (2) mildly cognitively impaired (MCI; ≥1.5 SDs below HCs on ≥2 domains and not CI or ≥2 SDs below HCs on 1 domain and <1.5 SDs on the other domains) and (3) cognitively preserved (CP; remainder).

Statistical analyses

We performed receiver operating characteristic (ROC) analyses and Bayesian statistics to determine the optimal cut-off scores (i.e. the highest value for sensitivity and specificity combined) and clinically relevant score ranges. Logistic regression analyses were performed to correct for depression, anxiety and fatigue. To identify relevant predictors of MSNQ-P scores, two backward linear regression models were performed including (1) cognition, depression, anxiety, fatigue, age, disease duration, education and neurological disability (complete data were available for 245 patients) and (2) domain-specific cognitive z-scores (p < 0.05).

Results

The optimal cut-off scores and psychometric properties are presented in Table 1. The optimal cut-off score to screen for CI or both MCI and CI resulted in an area under the curve (AUC) of 0.61 and 0.64, respectively. The AUCs did not increase when correcting for depression (0.63 and 0.66, respectively), anxiety (0.62 and 0.63, respectively), or fatigue (0.59 and 0.59, respectively). The AUCs also did not increase when neurobehavioral MSNQ-P items (13–15) were excluded (0.61 and 0.63, respectively). Since MSNQ-P scores were related to education, we performed stratified analyses for education (see Supplementary Table 1). The AUCs were higher among patients with a higher education (0.68 and 0.69, respectively), compared to a middle (0.50 and 0.55, respectively) or lower education (0.58 and 0.56, respectively). In patients with a higher education, the optimal cut-off score to screen for CI and for both MCI and CI was 12, which corresponded to a high sensitivity (0.87 and 0.85, respectively) and negative predictive value (0.93 and 0.81, respectively), yet a relatively lower specificity (0.40 and 0.52, respectively) and positive predictive value (0.25 and 0.59, respectively). Figure 1 shows the distribution of MSNQ-P scores and clinically relevant score ranges.

Table 1.

MSNQ-P cut-off scores and psychometric properties.

	Cut-off^a	Sensitivity	Specificity	PPV	NPV	Accuracy^b	AUC
CI vs MCI and CP	18	0.66	0.51	0.31	0.81	0.55	0.61*
CI and MCI vs CP	12	0.82	0.40	0.66	0.62	0.65	0.64*

CI: cognitively impaired; MCI: mildly cognitively impaired; CP: cognitively preserved; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the curve.

Optimal cut-off score on the patient-reported Multiple Sclerosis Neuropsychological Questionnaire (MSNQ-P) to detect the cognitively impaired patient group (either CI or a combination of both CI and MCI patients).

Proportion of patients correctly classified as cognitively impaired and cognitively preserved.

Significantly different from 0.50 (p < 0.01).

Figure 1.

Distribution of MSNQ-P scores across the cognitive groups. The bold lines represent the means and SDs (*p < 0.001). Of all patients, 41.8% were categorized as CP (n = 147), 32.7% as MCI (n = 115), and 25.6% as CI (n = 90). (a) The CI and MCI groups both had significantly higher MSNQ-P scores than the CP group. (b) The CI and MCI groups combined had significantly higher MSNQ-P scores than the CP group. The dotted lines indicate the clinically relevant score ranges (defined as a positive predictive value (PPV) or negative predictive value (NPV) of at least 0.70). The upper line indicates that 71% of the patients with scores of 27 or higher were MCI or CI patients. The lower line indicates that 70% of the patients with scores of 5 or lower were CP patients.

The final backward regression model showed that cognition (MCI vs CP: beta = 0.13, p = 0.020; CI vs CP: beta = 0.12, p = 0.045), depression (beta = 0.20, p = 0.008), anxiety (beta = 0.19, p = 0.006), fatigue (beta = 0.39, p < 0.001), education (middle vs low: beta = −0.16, p = 0.008; high vs low: beta = −0.14, p = 0.027) and neurological disability (middle vs low: beta = −0.04, p = 0.441; high vs low: beta = −0.17, p = 0.004) independently predicted MSNQ-P scores (R² = 0.49). In terms of cognitive domains, visuospatial memory (beta = −0.12, p = 0.026) and attention (beta = −0.24, p < 0.001) predicted MSNQ-P scores (R² = 0.08).

Discussion

The optimal MSNQ-P cut-off scores to identify cognitive impairment had insufficient psychometric properties, which is in line with most previous studies.^5–7 Our findings do indicate that certain score ranges are clinically relevant. High MSNQ-P scores (≥27) indicated mild or severe cognitive impairment in 71% of the cases, and we advise neuropsychological evaluation when MS patients obtain these scores. Very low MSNQ-P scores (≤5) indicated intact cognitive function, but we consider this range too small to be clinically relevant. Interestingly, MSNQ-P scores were a better reflection of cognitive status among patients with a higher education than among patients with a middle or lower education. For patients with a higher education, MSNQ-P scores below 12 indicated intact cognitive function. Thus, the MSNQ-P can be clinically useful when interpreting low or high score ranges.

Multiple factors influenced patient-reported cognitive complaints, which may explain the difficulty of defining specific and sensitive cut-off scores. Cognitive function itself, but also depression, anxiety, fatigue, education and neurological disability predicted MSNQ-P scores. Benedict et al. argued that the MSNQ-P might be a useful screening tool when combined with a depression questionnaire.⁴ However, the ability of the MSNQ-P to screen for cognitive impairment did not improve when we excluded patients with a depressed mood (data not shown). In general, the clinical applicability did not improve when either depression, anxiety or fatigue was taken into account.

A promising line of research focuses on short neuropsychological tests as a cognitive screening tool (e.g. the Symbol Digit Modalities Test),¹⁰ which may reflect cognitive impairment more accurately. However, the need for guided assistance in administration could challenge implementation in routine neurological care.

To conclude, MSNQ-P scores in the lowest and highest ranges can indicate whether patients are cognitively intact or impaired. Still, patient-reported cognitive complaints assessed by the MSNQ-P should be interpreted with caution.

Supplemental Material

MSJ777295_supplementary_table_1 – Supplemental material for The clinical value of the patient-reported multiple sclerosis neuropsychological screening questionnaire

Supplemental material, MSJ777295_supplementary_table_1 for The clinical value of the patient-reported multiple sclerosis neuropsychological screening questionnaire by Ilse M Nauta, Lisanne J Balk, Judith M Sonder, Hanneke E Hulst, Bernard MJ Uitdehaag, Luciano Fasotti and Brigit A de Jong in Multiple Sclerosis Journal

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: Dutch MS Research Foundation (protocol number 15-911).

References

Amato

Zipoli

Portaccio

Multiple sclerosis-related cognitive changes: A review of cross-sectional and longitudinal studies. J Neurol Sci 2006; 245: 41–46.

Rao

SM.

A manual for the brief repeatable battery of neuropsychological tests in multiple sclerosis. Med Coll Winsconsin: Milwaukee, WI., 1990.

Langdon

Amato

Boringa

, et al. Recommendations for a brief international cognitive assessment for multiple sclerosis (BICAMS). Mult Scler 2012; 18: 891–898.

Benedict

Cox

Thompson

, et al. Reliable screening for neuropsychological impairment in multiple sclerosis. Mult Scler 2004; 10: 675–678.

Vanotti

Benedict

Acion

, et al. Validation of the multiple sclerosis neuropsychological screening questionnaire in Argentina. Mult Scler 2009; 15: 244–250.

Sonder

Mokkink

van der Linden

, et al. Validation and interpretation of the Dutch version of the multiple sclerosis neuropsychological screening questionnaire. J Neurol Sci 2012; 320: 91–96.

O’Brien

Gaudino-Goering

Shawaryn

, et al. Relationship of the multiple sclerosis neuropsychological questionnaire (MSNQ) to functional, emotional, and neuropsychological outcomes. Arch Clin Neuropsych 2007; 22: 933–948.

Zigmond

Snaith

RP.

The hospital anxiety and depression scale. Acta Psychiat Scand 1983; 67: 361–370.

Vercoulen

JHMM

Alberts

Bleijenberg

De checklist individuele spankracht (CIS). Gedragstherapie 1999; 32: 131–136.

10.

Parmenter

Weinstock-Guttman

Garg

, et al. Screening for cognitive impairment in multiple sclerosis using the symbol digit modalities test. Mult Scler 2007; 13: 52–57.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB