Development and psychometric properties of the Multiple Sclerosis Knowledge Assessment Scale: Rasch analysis of a novel tool for evaluating MS knowledge

Abstract

Background:

Multiple sclerosis (MS)-related knowledge is an important evaluation metric for health education interventions. However, few MS knowledge assessment tools are currently available for use.

Objectives:

This study aims to develop a reliable and valid Multiple Sclerosis Knowledge Assessment Scale (MSKAS) for use in the MS community and the general public.

Methods:

The MSKAS was developed using a Delphi study methodology and was administered to participants in the first open enrolment of the Understanding Multiple Sclerosis (UMS) online course. Rasch analysis was used to examine its psychometric properties and develop the final scale.

Results:

Experts from across the MS community participated in the development of the MSKAS, resulting in an initial scale of 42 items. Five hundred and forty-three UMS participants completed the MSKAS; 89% were female and 30% were people with MS. The final unidimensional 22-item scale has a person separation index of 2.16, a person reliability index of 0.82, an item separation index of 11.19, and a Cronbach’s alpha (kr-20) test reliability of 0.87.

Conclusion:

The MSKAS is a unidimensional scale with good construct validity and internal consistency. The MSKAS has the potential to be useful for the assessment of MS knowledge in research and clinical practice.

Keywords

Multiple sclerosis Rasch analysis psychometric properties knowledge assessment

Introduction

Multiple sclerosis (MS) is a neurodegenerative disease that causes the immune system to attack and gradually impair the function of the central nervous system.¹ Globally, the prevalence of MS is increasing.² In Australia, the number of people with multiple sclerosis (PwMS) has increased from 21,283 in 2010 to 25,607 in 2017³ and MS creates a significant burden for individuals, families and society.² A recent health economic impact analysis indicated that the cost of MS has increased by 41% over a 7-year period, rising from AU$ 1.24 billion in 2010 to US$1.75 billion in 2017.⁴ Reducing this burden requires a multifactorial approach, including the provision of accurate and reliable information about MS to PwMS and health care providers to improve knowledge and health outcomes.⁵

Having adequate information and increased MS-related knowledge has been shown to improve accurate diagnosis and preventive strategies in PwMS and others,¹ shared decision making,^6,7 relapse management,⁸ disease-modifying therapy choices,⁹ treatment adherence¹⁰ and satisfaction with care.¹¹ Providing PwMS with pertinent up-to-date information has been shown to be therapeutic, foster inclusion, and debunk negative perceptions of MS that often result in prejudice, inequality and poor employment outcomes.⁵

However, a Cochrane review of randomised controlled trials evaluating the effect of information provision aimed to promote informed choice and improve patient-relevant outcomes for PwMS indicated MS-related knowledge is poor.¹² An international survey investigating the level of MS risk knowledge among people with relapsing-remitting MS in eight countries found that overall MS risk knowledge was low.¹³ Similarly, an assessment of knowledge and attitudes among PwMS living in Iran and taking interferon beta showed that participants had very little knowledge about the medication they received.¹⁴ This MS knowledge and information gap was highlighted by the MS in the 21st Century Steering Group as one of the key unmet needs in MS.⁵

To address this need, the Menzies Institute for Medical Research (Menzies) MS Research Group developed a free 6-week online course entitled Understanding Multiple Sclerosis (UMS) in collaboration with Multiple Sclerosis Limited (MSL), an Australian-based MS service organisation, and the Wicking Dementia Research and Education Centre. The aim of the project was to improve MS knowledge in the MS community (e.g. PwMS, family and friends of PwMS, health care providers) and the general public through information provision. To evaluate the success of this project, it was essential to quantify the effect of the UMS course on participant knowledge.

Previous work has assessed MS knowledge using questionnaires^10,15–18 or a scoring scale.^14,19 While these tools were well developed, validated and evaluated in different groups,^16–18 they may not be ideal for use in all settings. Tools such as the Multiple Sclerosis Knowledge Questionnaire (MSKQ) and the Risk Knowledge in Relapsing MS (RIKNO) present items as multiple-choice questions with distractor items. Consequently, they require a relatively high verbal cognitive load to read and compare statements. Tools with multiple-choice questionnaires require a longer time to complete and are more difficult to answer than a knowledge assessment tool using a simpler structure such as a true/false response format.^20,21 A true/false response format therefore has the potential for more adequate sampling of knowledge within a restricted time frame.²² Furthermore, although previous tools have been validated for use on various MS community groups (e.g. PwMS, family members of PwMS and healthcare practitioners), they have not been validated for use among people who do not identify as members of the MS community, a key demographic included in the UMS course. In order to address the potential limitations and applicability of previous tools and appropriately quantify the effect of the UMS course on participant knowledge, we developed a simple, rapid and easy-to-apply Multiple Sclerosis Knowledge Assessment Scale (MSKAS). In this paper, we discuss the development of the MSKAS and evaluate its psychometric properties.

Methods

Ethics

The studies described below were approved by the University of Tasmania’s Social Science Human Research Ethics Committee (H0017404; H0017930; H0017924). All participants gave their informed consent.

Stage 1: developing the MSKAS

We developed the initial MSKAS using a three-phase Delphi study (Figure 1). In Phase 1, we identified and recruited seven MS experts, including members of the research team, and asked them to nominate local, national and international MS experts from across the MS community to participate in Phase 2 of the study. Phase 1 participants responded via email within 2 weeks, and their responses were de-identified and used to compile a list of potential participants for Phase 2 of the study.

In Phase 2, the nominated experts were sent a series of three online surveys. In the first survey (Phase 2, part 1), they were queried about what they considered to be essential information about MS with the questions presented in Table 1.

Table 1.

Questions asked of MS community experts in the first Delphi study survey (Phase 2, part 1).

1. How would you define MS for a lay person (a person with little to no knowledge or experience with MS)?
2. What key facts are essential to understanding MS?
3. What key facts about MS are frequently misunderstood by lay people (people with little or no knowledge or experience with MS)?
4. What key facts about MS should people in your field know?
5. Please describe any facts or topics that you feel are inadequately addressed in other measures of MS knowledge or that might be overlooked in an assessment of MS knowledge.

MS: multiple sclerosis.

We compiled Phase 2, part 1 survey responses into a list of statements, retaining as much of the original language as possible and removing duplicate statements. In the second survey (Phase 2, part 2), this list of statements was sent back to participants and they were asked to rank the statements on a 5-point Likert-type scale (1 – not important at all to 5 – very important). The results were analysed by determining the median and interquartile range (IQR) for each statement. The statements were sorted into four groups (very high importance, high importance, medium importance and low importance) using the criteria presented in Supplementary Table 1.

To evaluate consensus, the statements were re-presented to participants in a third survey (Phase 2, part 3) in ranking groups. Participants were asked to rank them on the same 5-point Likert-type scale as in Phase 2, part 2. These three surveys were carried out within 2 months, and responses were collected anonymously. The results of the part 3 survey were evaluated in the same way as part 2. We then compared the results from both surveys and sorted the statements into three groups:

Ranked of high importance or very high importance in both surveys;

Ranked of medium importance in at least one survey;

Ranked of low importance in at least one survey.

Statements in group 3 were excluded for further consideration for the MSKAS. All other statements were included in Phase 3. Phase 3 consisted of a half-day discussion workshop with MS experts from across the MS community identified through existing relationships with Menzies and MSL and included members of the research team. The purpose of this workshop was to refine the language of the statements so that they could be used in a questionnaire with a yes/no/I don’t know format, falsify at least 30% of statements, and to reach group consensus on which items were essential.

Stage 2: pilot testing

The initial tool was pilot tested within the UMS course’s first iteration cohort study. However, the MSKAS was not based on the UMS course. The UMS course consists of six 2-hour modules presenting evidence-based information about MS, from its underlying pathology to its impact on everyday life.²³ The course was advertised widely through participating organisations and through social media. Participants in this study completed the initial survey prior to beginning course materials, including demographic questions and the MSKAS items, along with other assessment tools measuring resilience, health literacy, self-efficacy, self-care and health outcomes, including quality of life. The data were de-identified for analysis.

Statistical analysis

Stata 15.1 was used for data cleaning, management and descriptive statistics. Rasch analysis was conducted using Winsteps software, version 4.4.6.²⁴ Rasch analysis is a method of psychometric probability-based analysis that predicts if observed responses fit the pattern of the Rasch model.²⁵ If the model requirement is met, Rasch analysis identifies the measurement and structural properties of a scale (or instrument). Rasch analysis identifies the relative difficulty of each item on the scale and determines the individual skill or lack of skill that impedes what the scale aims to measure.^25,26 Rasch modelling is widely used to assess the psychometric properties of scales, test items or questionnaires in health and education.^25,26 A dichotomous model comparing correct and incorrect responses (including ‘I don’t know’ as an incorrect response) was used in our analysis. The properties assessed are described in Table 2.

Table 2.

Rasch measurement properties and assessment criteria.

Rasch measurement property	Definition	Statistical test in Winsteps and acceptable values
Unidimensionality	The ability of each of the 22 MSKAS subscales to measure a single factor (MS knowledge).	Principal component analysis (PCA) of the residuals with first contrast eigenvalue of ⩽2.0 indicates unidimensionality.²⁵
Fit statistics	A test to determine the extent to which the data fits the Rasch model, for both items and persons, and the whole subscale.	The fit statistics are based on mean square (MNSQ) and Z-standardised Scores (ZSTD). Infit and Outfit MNSQ between 0.7 and 1.3 is considered acceptable for multiple-choice questions (run of the mill assessment).^24,25
Internal consistency and reliability	The extent to which the items in each of the 22 MSKAS subscales measures the same concept.	A person or item reliability of ⩾0.7, and person or item separation index of ⩾1.5 indicates good internal consistency and reliability. A Cronbach’s alpha (Kuder–Richardson) test reliability ⩾0.7 also supports good internal consistency and reliability.^24,25,27,28
Scale targeting (Item difficulty)	A measure of the ability of the MS MOOC cohort to answer the MSKAS items correctly against the difficulty level of the MSKAS items arrayed along the same continuum.	This is assessed with a person-item graph. A well-targeted scale should have participants and items spread across the continuum. The mean person-item location is zero log-odds unit (logit). Items located at the bottom (below zero) are the easiest and the people closest to these items are those with less ability. The items at the top (above zero), on the other hand, are the most difficult and the people closest to them are those with greater ability.²⁹
Differential item functioning	Differential item functioning (DIF) measures if there is any bias in response to the 22 MSKAS items within groups in the sample who have similar levels of knowledge.	We assessed DIF for gender and MS status (Living with MS or not). The Mantel–Haenszel approach was used and a contrast DIF of ⩾0.64 logit with p-value (two-tailed) ⩽0.05 is considered statistically significant.²⁵ For dichotomous outcomes, at least 1000 per category is needed to detect a true difference.²⁴ Although our sample size was too small to detect true differences, it is still considered best practice to evaluate DIF.²⁴

Infit: overfit coefficient; Outfit: underfit coefficient; MNSQ: mean square; ZSTD: Z-standardised scores; MSKAS: Multiple Sclerosis Knowledge Assessment Scale; MOOC: Massive Open Online Course.

Results

Delphi study

Delphi study participant characteristics are presented in Table 3. The Phase 1 participants nominated a total of 49 MS experts for Phase 2. Of these, 21 participated in the first survey (part 1). Seventeen experts participated in the second survey (part 2) and 17 experts participated in the third survey (part 3). The mean age of Phase 2 participants was 54 years, with most (88%–94%) residing in Australia.

Table 3.

Characteristics of Delphi study participants.

	Phase 1 (N = 7)	Phase 2			Phase 3 (N = 9)
	Phase 1 (N = 7)	Part 1 (N = 21)	Part 2 (N = 17)	Part 3 (N = 17)	Phase 3 (N = 9)
Age, mean (SD)	NA	51 (12.9)	53 (12.7)	59 (13.3)	NA
Gender, n (%)
Female	5 (71)	11 (52)	10 (59)	11 (65)	6 (67)
Male	2 (29)	10 (48)	7 (41)	6 (35)	3 (33)
MS community roles,^a n (%)
Person with MS	1 (14)	7 (33)	7 (41)	10 (59)	3 (33)
Family or friend	0 (0)	2 (10)	3 (18)	5 (29)	0 (0)
Carer	0 (0)	1 (5)	1 (6)	2 (12)	0 (0)
Neurologist	1 (14)	4 (19)	2 (12)	3 (18)	1 (11)
MS nurse	1 (14)	5 (24)	3 (18)	0 (0)	2 (22)
Allied health practitioner	0 (0)	0 (0)	0 (0)	0 (0)	1 (11)
General practitioner	0 (0)	0 (0)	0 (0)	0 (0)	0 (0)
Service provider	2 (29)	3 (14)	2 (12)	0 (0)	1 (11)
Advocate	1 (14)	3 (14)	7 (41)	6 (35)	1 (11)
Researcher	2 (29)	4 (19)	3 (18)	1 (6)	4 (44)
Other/Not MS community member,	0 (0)	2 (10)	5 (29)	5 (29)	0 (0)
Country of residence, n (%)
Australia	7 (100)	NA	15 (88)	16 (94)	9 (100)
New Zealand	0 (0)	NA	0 (0)	1 (6)	0 (0)
United Kingdom	0 (0)	NA	2 (12)	0 (0)	0 (0)

SD: standard deviation; MS: multiple sclerosis; NA: not applicable.

Variables are multiple selections.

Phase 2 resulted in 97 unique statements. Of these, 27 were ranked of low importance in at least one survey and were excluded from further consideration. The remaining 70 statements were evaluated in the tool refinement workshop (Phase 3).

Phase 3 participants represented a range of MS expertise, from advocacy to research. The group eliminated 28 statements because they could not be framed as true or false statements, because they were ambiguous or because they were incorrect. The group falsified 14 (33.3%) of the remaining statements, resulting in the initial 42-item MSKAS, which was presented as a series of true/false/I don’t know questions.

Rasch analysis

Participants

We excluded 11 duplicate surveys, 9 surveys with incomplete data and 2 surveys with suspect response patterns (all true, all false or all I don’t know), leaving 543 participant surveys for inclusion in the analysis.

Pilot study participant characteristics are presented in Table 4. The average age was 48.4 years (standard deviation (SD) = 12.6), and 90% of participants were below 65 years. Most participants (89%) were female and 70% were married or in a de facto partnership. Participants were highly educated, with 58% having an associate degree or higher. Nearly all spoke English at home (93%).

Table 4.

Demographic characteristics of pilot study participants used in the Rasch analysis.

Variable	Participants (n = 543)
Age, mean (SD)	48.42 (12.61)
MS duration since diagnosis (years), mean(SD)	9.26 (8.46)
Age groups, n (%)
<35	90 (16.57)
35–44	105 (19.34)
45–54	165 (30.39)
55–64	129 (23.76)
65–74	49 (9.02)
75+	5 (0.92)
Gender, n (%)
Female	484 (89.13)
Male	59 (10.87)
Relationship status, n (%)
Married or in de facto partnership	380 (69.98)
Separated, divorced or widowed	77 (14.14)
Single/never married	86 (15.84)
Education level, n (%)
Primary/secondary school	59 (10.87)
Occupational/trade certificate	168 (30.94)
Associate degree	17 (3.13)
Bachelor/honours degree	171 (31.49)
Postgraduate degree	128 (23.57)
Speak English at home, n (%)	506 (93.19)
Living with MS, n (%)	161 (29.65)
Have you had relapse before
Yes	15 (2.76)
No	146 (26.89)
Not applicable	382 (70.35)
Family/friend with MS, n (%)	138 (25.41)^a
Carer of person living with MS, n (%)	49 (9.02)^a
MS service provider, n (%)	24 (4.42)^a
Neurologist, n (%)	5 (0.92)^a
General practitioner, n (%)	7 (1.29)^a
MS Nurse, n (%)	12 (2.21)^a
Advocate, n (%)	14 (2.58)^a
Researcher, n (%)	10 (1.84)^a
Allied health practitioner, n (%)	115 (21.18)^a
Other/not MS community member, n (%)	49 (9.02)^a

SD: standard deviation; MS: multiple sclerosis.

Variables are multiple selections.

Dimensionality

The principal component analysis (PCA) of the residuals supported unidimensionality of the Rasch model. The observed variance explained by the items was 43.2%, comparable to the expected variance in the model (43.0%). The first contrast eigenvalue was 1.7.

Item and person fit

An initial analysis indicated several misfitting items. Twenty-two misfit items were removed in 22 separate item-by-item deletion iterations. After these deletions, all items had a good fit for the model (Table 5).

Table 5.

Item fit statistics for the final 22 MSKAS items from the dichotomous Rasch analysis (n = 543).

	MSKAS items	LOGIT(SE)	Infit statistics		Outfit statistics
	MSKAS items	LOGIT(SE)	MNSQ	ZSTD	MNSQ	ZSTD
MSKAS34	Over 200 genes, mostly related to the immune system, have been found to influence a person’s risk of developing MS (True).	3.41 (0.13)	1.13	1.76	1.21	1.04
MSKAS37	Long-term MS disease remission is not possible (False).	2.24 (0.11)	0.99	−0.13	0.87	−0.99
MSKAS42	Exposure to Epstein–Barr virus (which causes glandular fever), a lack of sunlight, vitamin D deficiency, smoking, and obesity all increase the risk of developing MS (True).	1.64 (0.11)	1.01	0.19	0.94	−0.55
MSKAS15	The long-term outcomes of MS are not affected by starting treatment earlier or later (False).	1.59 (0.11)	0.95	−1.14	0.92	−0.70
MSKAS5	Treating the underlying disease is the same as treating MS symptoms (False).	1.42 (0.11)	1.03	0.75	1.17	1.58
MSKAS11	In people with progressive MS, the disease progressively gets worse, often without obvious bouts of inflammation (True).	1.08 (0.11)	1.00	0.12	0.93	−0.71
MSKAS10	MS always results in impairment in thinking and memory (False).	1.05 (0.11)	1.01	0.32	1.20	1.88
MSKAS14	There are a range of effective therapies for relapsing-remitting MS (True).	1.05 (0.11)	0.87	−2.97	0.73	−2.95
MSKAS24	The risk of developing MS is due to a mix of genetic, environmental and immune system-related factors (True).	0.85 (0.11)	1.01	0.26	0.91	−0.84
MSKAS36	A relapse is the significant improvement of symptoms caused by inflammation from active disease (False).	0.58 (0.11)	1.08	1.44	1.16	1.39
MSKAS28	MS always results in people needing a wheelchair (False).	0.16 (0.12)	0.88	−2.12	0.74	−2.17
MSKAS17	There are effective symptom management options, such as self-management strategies for fatigue (True).	0.08 (0.12)	0.95	−0.82	1.02	0.16
MSKAS20	Smoking slows the rate of MS disability progression over time (False).	0.02 (0.12)	1.04	0.66	0.90	−0.71
MSKAS16	MS causes lesions in the brain and spinal cord that impair the way the nerves and neural pathways conduct messages from the brain to the rest of the body (True).	−0.60 (0.13)	0.93	−0.83	0.84	−0.84
MSKAS40	MS symptom severity can fluctuate from day to day and this fluctuation does not necessarily represent a relapse or disease worsening (True).	−0.69 (0.13)	0.98	−0.28	1.22	1.10
MSKAS30	People with MS can live full and successful lives (True).	−1.02 (0.14)	0.97	−0.33	0.88	−0.47
MSKAS1	Positively adjusting lifestyle factors, such as quitting smoking, benefits the health and well-being of people with MS (True).	−1.44 (0.16)	1.01	0.13	0.96	−0.06
MSKAS7	Monitoring for disease activity is important (True).	−1.74 (0.17)	1.10	0.85	1.23	0.76
MSKAS41	MS fatigue is the same as disinterest or laziness (False).	−1.80 (0.18)	1.17	1.37	1.11	0.42
MSKAS2	Exercise is beneficial for people living with MS (True).	−1.90 (0.18)	1.04	0.35	0.94	−0.07
MSKAS32	Counselling has no role in the management of MS (False).	−2.19 (0.20)	0.98	−0.12	0.96	0.01
MSKAS31	Good communication is important for MS care (True).	−3.79 (0.33)	0.85	−0.50	0.92	−0.01

Infit: overfit coefficient; Outfit: underfit coefficient; MNSQ: mean square; ZSTD: Z-standardised scores; MSKAS: Multiple Sclerosis Knowledge Assessment Scale; SE: standard error; MS: multiple sclerosis.

Correct response = 1; incorrect response = 0. The correct response for each item is indicated in brackets.

The MNSQ acceptable limits for productive measurement were 0.7–1.3. Infit and outfit values close to 1.0 show acceptable fit and that items are productive for measurement. Statements are administered as true/false/I don’t know questions and analysed as correct/incorrect with ‘I don’t know’ included in the incorrect group.

Reliability and internal consistency

MSKAS person and item reliability were adequate. The person separation index was 2.16 with person reliability index of 0.82. This indicates there were enough participants to differentiate between higher and lower ability participants. The item separation index was 11.19 with a reliability of 0.99. This indicates there was a large enough sample to confirm the item difficulty continuum seen in Figure 2. The Kuder–Richardson test reliability was 0.87, indicating that the final 22 MSKAS items had good internal consistency and the items are measuring the unidimensional construct (i.e. MS knowledge) reliably.

Item difficulty

The person-item map of the final 22 MSKAS items is shown in Figure 2. The range of participant ability to respond to the items (−5.0 to 5.0 logits) was larger than the range of item response difficulty (−3.79 to 3.41 logits). Although the items were spread across the continuum indicating good item targeting, there were gaps in the scale, making it difficult to differentiate participants at select ability levels (see Figure 2). For instance, there was a 1.59 logit gap between the most difficulty item (3.41 logits) and the highest participant ability (5.0 logits). Similarly, a gap of −1.21 logits was seen between the easiest item (−3.79 logits) and the lowest participant ability (−5.0 logits). Items measuring the same ability were retained when they measured different concepts that were deemed important to MS knowledge because there were few items measuring each subcomponent.

Differential item functioning

Our analysis of variance (see Figure 3(a)) shows there is no significant differential item functioning (DIF) by gender based on the minor DIF contrast coefficients (DIF contrast < 0.60 with Mantel chi-square p > 0.05) for each item on the scale. However, DIF response bias was identified as a potential issue with several items based on MS status because of significant Mantel chi-square coefficients (p < 0.05) and DIF contrast coefficients > 0.60 (see Figure 3(b)). PwMS perceived items MSKAS37, MSKAS24, MSKAS32, MSKAS34 and MSKAS41 to be more difficult than those not living with MS by 0.04 logit (p = 0.01), 0.08 logit (p = 0.003), 0.03 logit (p = 0.004), 0.14 logit (p < 0.001) and 0.03 logit (p = 0.02), respectively. Conversely, PwMS perceived items MSKAS14, MSKAS16, MSKAS28 and MSKAS42 to be less difficult than those not living with MS by 0.16 logit (p < 0.001), 0.09 logit (p < 0.001), 0.09 logit (p < 0.001) and 0.10 logit (p = 0.001) respectively.

Discussion

The primary aim of this study was to develop a reliable and validated MSKAS for use in the MS community and the general public. We adopted two methods of instrument development: the Delphi method and Rasch analysis. The combination of these methods ensures synergy of experimental and expert knowledge, and assesses psychometric properties in agreement with instrument development best practice.³⁰ Using the Delphi process ensured content validity and the multiple rounds of data collection enhanced the refinement process. Including a diverse group of experts fostered objective consensus-building and ensured that we obtained acceptable content for the new tool.³¹ In line with previous research, we consider the 21 participants who took part to support the development of the MSKAS.³¹

Overall, the MSKAS has acceptable psychometric properties related to accuracy, precision in measurement and applicability. It is unidimensional, indicating that it only measures MS-related knowledge. Our findings were consistent with previous studies using Rasch analyses to confirm unidimensionality in an 88-item MS spasticity scale³² and a participation scale for individuals with orthopaedic or neurological conditions.³³ There was a good model fit for the 22 items in the final tool (52% of the original items).^24,25 As in previous research, this means the items are highly productive for the consistent measurement of a single construct.³⁴

The item reliability of the MSKAS was very high (reliability index of 0.99), well above the minimum threshold of 0.80.²⁵ This was comparable to previous work, including an 88-item MS spasticity scale,³² and higher than a similar dementia knowledge assessment scale³¹ and participation scale for individuals with orthopaedic or neurological conditions.³³ This suggests the items on the MSKAS scale can locate a latent measure with a high level of precision,³⁵ and good internal consistency and reliability. Although we found evidence of DIF based on MS status, we found no evidence of DIF by gender. However, it is important to note that our sample is too small to detect true differences, and therefore, further analyses of DIF by gender and MS status are needed to confirm our findings.

The hierarchy of item difficulty was similar to previous studies^32,34 and the span of item difficulty was acceptable.²⁵ Participant ability level shows a unimodal slightly skewed distribution,³⁵ with most items located around 1.57 to −1.90 logit ability levels. The items also did not target participants with ability levels at the extremes (ability levels above 3.41 or below −3.79 logits). The gaps observed between item difficulty and participant ability levels may have influenced the person separation index of 2.16.²⁵ As Linacre explained, this measure is highly influenced by the match between item difficulty and participant ability level, length of the scale and number of categories per item.²⁴ The MSKAS may require a few more difficult and less difficult items to properly measure individuals with extremely low and high ability levels, in agreement with previous research.³³

Strengths, limitations and further research

Our methodology had two main strengths. First, experts from across the MS community took part in all stages of tool development. Second, a large international cohort including both members of the MS community and the general public pilot tested the tool. This work had three main limitations. First, the gaps in item difficulty coverage discussed above. Second, the tool was developed and tested in English among English speakers, possibly limiting its use for non-English speakers. Third, the Delphi study participants were largely limited to residents of Australia. Future research should test the utility of adding additional items and assess translated versions of the tool among other language groups. Because this is a new scale, it requires test–retest reliability assessment and comparison with other similar tools^15–17 to determine its construct/predictive and convergent/divergent validity in different groups.

Conclusion

Lack of MS knowledge among PwMS and the general public is a challenge.^5,12,14 Consequently, a validated, reliable tool for measuring MS knowledge in PwMS and general public has both clinical and research implications as a means to identify deficits in MS knowledge and evaluate knowledge change. The MSKAS is a 22-item comprehensive scale that tests individual MS knowledge, covering a range of subject areas from biology and pathology to disease management. The tool has good internal consistency, reliability and validity and is unidimensional. It is short and easy to administer and score. The MSKAS is fit for use to assess MS knowledge in the MS community and general public in English-speaking populations internationally.

Supplemental Material

MSJ929626_supplementary_table_1 – Supplemental material for Development and psychometric properties of the Multiple Sclerosis Knowledge Assessment Scale: Rasch analysis of a novel tool for evaluating MS knowledge

Supplemental material, MSJ929626_supplementary_table_1 for Development and psychometric properties of the Multiple Sclerosis Knowledge Assessment Scale: Rasch analysis of a novel tool for evaluating MS knowledge by Barnabas Bessing, Cynthia A Honan, Ingrid van der Mei, Bruce V Taylor and Suzi B Claflin in Multiple Sclerosis Journal

Footnotes

Acknowledgements

The authors would like to thank all the participants of Delphi study and the Understanding MS MOOC cohort study who committed their time and energy to complete the surveys for this project.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This study was supported by Multiple Sclerosis Limited.

ORCID iD

Barnabas Bessing

Supplemental material

Supplemental material for this article is available online.

References

Wilkins

. Progressive multiple sclerosis. New York: Springer, 2017.

GBD 2016 Multiple Sclerosis Collaborators. Global, regional, and national burden of multiple sclerosis 1990–2016: A systematic analysis for the global burden of disease study 2016. Lancet Neurol 2019; 18(3): 269–285.

Campbell

Simpson

Ahmad

, et al. Change in multiple sclerosis prevalence over time in Australia 2010-2017 utilising disease-modifying therapy prescription data. Mult Scler J 2020; 26(11): 1315–1328.

Ahmad

Campbell

van der Mei

, et al. Health economic impact of multiple sclerosis in Australia in 2017: An analysis of MS Research Australia’s platform: The Australian MS Longitudinal Study (AMSLS), 2018, http://ecite.utas.edu.au/132297

Rieckmann

Centonze

Elovaara

, et al. Unmet needs, burden of treatment, and patient engagement in multiple sclerosis: A combined perspective from the MS in the 21st Century Steering Group. Mult Scler Relat Disord 2018; 19: 153–160.

Colligan

Metzler

Tiryaki

. Shared decision-making in multiple sclerosis. Mult Scler J 2017; 23(2): 185–190.

Brown

Gabriele

White

. Physician and patient treatment decision-making in relapsing-remitting multiple sclerosis in Europe and the USA. Neurodegener Dis Manag 2018; 8(6): 371–376.

Köpke

e S

Kasper

Mühlhauser

, et al. Patient education program to enhance decision autonomy in multiple sclerosis relapse management: A randomized-controlled trial. Mult Scler 2009; 15(1): 96–104.

Kasper

Köpke

Mühlhauser

, et al. Informed shared decision making about immunotherapy for patients with multiple sclerosis (ISDIMS): A randomized controlled trial. Eur J Neurol 2008; 15(12): 1345–1352.

10.

Heesen

Kasper

Segal

, et al. Decisional role preferences, risk knowledge and information interests in patients with multiple sclerosis. Mult Scler 2004; 10(6): 643–650.

11.

Solari

Martinelli

Trojano

, et al. An information aid for newly diagnosed multiple sclerosis patients improves disease knowledge and satisfaction with care. Mult Scler 2010; 16(11): 1393–1405.

12.

Köpke

Solari

Rahn

, et al. Information provision for people with multiple sclerosis. Cochrane Database Syst Rev 2018; 10: CD008757.

13.

Giordano

Liethmann

Köpke

, et al. Risk knowledge of people with relapsing-remitting multiple sclerosis – Results of an international survey. PLoS ONE 2018; 13(11): e0208004.

14.

Abolfazli

Elyasi

Javadi

, et al. Knowledge and attitude assessment of Iranian multiple sclerosis patients receiving interferon beta. Iran J Neurol 2014; 13(3): 160.

15.

Giordano

Uccelli

Pucci

, et al. The Multiple Sclerosis knowledge questionnaire: A self-administered instrument for recently diagnosed patients. Mult Scler 2010; 16(1): 100–111.

16.

Heesen

Kasper

Fischer

, et al. Risk knowledge in relapsing multiple sclerosis (RIKNO 1.0): Development of an outcome instrument for educational interventions. PLoS ONE 2015; 10(10): e0138364.

17.

Heesen

Pottgen

Rahn

, et al. What should a person with relapsing-remitting multiple sclerosis know? – Focus group and survey data of a risk knowledge questionnaire (RIKNO 2.0). Mult Scler Relat Disord 2017; 18: 186–195.

18.

Uccelli

Traversa

Trojano

, et al. Lack of information about multiple sclerosis in children can impact parents’ sense of competency and satisfaction within the couple. J Neurol Sci 2013; 324(1-2): 100–105.

19.

Maybury

Brewin

. Social relationships, knowledge and adjustment to multiple sclerosis. J Neurol Neurosurg Psychiatry 1984; 47(4): 372–376.

20.

de Bruin

Fischhoff

. The effect of question format on measured HIV/AIDS knowledge: Detention center teens, high school students, and adults. AIDS Educ Prev 2000; 12(3): 187–198.

21.

Put your thinking cap on: Detecting cognitive load using EEG during learning. In: Proceedings of the seventh international learning analytics & knowledge conference, Vancouver, BC, Canada, 13–17 March 2017.

22.

Frisbie

. Multiple choice versus true-false: A comparison of reliabilities and concurrent validities. J Educ Meas 1973; 10(4): 297–304.

23.

Claflin

Gates

Maher

, et al. Building a successful massive open online course (MOOC): A description of our process and lessons learned. JMIR, accepted.

24.

Linacre

. Winsteps® Rasch measurement computer program user’s guide. Beaverton, OR: Winsteps Com, 2019.

25.

Bond

Fox

. Applying the Rasch model: Fundamental measurement in the human sciences. 3rd ed. New York: Routledge, 2015.

26.

Boone

Staver

Yale

. Rasch analysis in the human sciences. New York: Springer, 2013.

27.

Tennant

Conaghan

. The Rasch measurement model in rheumatology: What is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthrit Care Res 2007; 57(8): 1358–1362.

28.

Linacre

. Predicting responses from Rasch measures. J Appl Meas 2010; 11(1): 1–10.

29.

Pallant

Tennant

. An introduction to the Rasch measurement model: An example using the hospital anxiety and depression Scale (HADS). Br J Clin Psychol 2007; 46(Pt. 1): 1–18.

30.

Hepworth

Rowe

Burnside

. Development of a patient reported outcome measures for measuring the impact of visual impairment following stroke. BMC Health Serv Res 2019; 19(1): 348.

31.

Annear

Toye

Eccleston

, et al. Dementia knowledge assessment scale: Development and preliminary psychometric properties. J Am Geriatr Soc 2015; 63(11): 2375–2381.

32.

Pellicciari

Ottonello

Giordano

, et al. The 88-item multiple sclerosis spasticity scale: A Rasch validation of the Italian version and suggestions for refinement of the original scale. Qual Life Res 2019; 28(1): 221–231.

33.

Souza

MAP

Coster

Mancini

, et al. Rasch analysis of the participation scale (P-scale): Usefulness of the P-scale to a rehabilitation services network. BMC Public Health 2017; 17(1): 934.

34.

Honan

McDonald

Sufani

, et al. The awareness of social inference test: Development of a shortened version for use in adults with acquired brain injury. Clin Neuropsychol 2016; 30(2): 243–264.

35.

Dutt

Tan

Alagumalai

, et al. Development and validation of the ability in behavior assessment and interventions for teachers using Delphi technique and Rasch analysis. J Autism Dev Disord 2019; 49(5): 1976–1987.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.06 MB