Validation of the preference-based multiple sclerosis index

Abstract

Background:

Preference-based measures of health-related quality of life (HRQL) are used as primary or secondary endpoints in multiple sclerosis (MS) research.

Objective:

The purpose of this paper was to evaluate the structural, convergent, and known-groups validity of the preference-based multiple sclerosis index (PBMSI) of HRQL in people with MS.

Methods:

Participants were recruited from three MS clinics in Montreal. Structural validity was assessed using polychoric correlation coefficients and factor analysis. To assess convergent validity, hypotheses were formulated about the strength of correlations between the PBMSI and other HRQL measures. Known-groups validity was assessed against different measures of disability.

Results:

The average age of the sample was 46 and 77% were women. Factor analysis supported the structural validity of the PBMSI; the items collectively were measuring one underlying construct. The PBMSI showed convergent validity against generic measures of HRQL, and known-groups validity between persons with different levels of disability.

Conclusion:

The results of this study support the construct validity of the PBMSI as an outcome measure of HRQL in MS. The PBMSI overcomes limitations observed with currently used HRQL measures in MS and may be used to contrast different interventions for people with MS.

Keywords

Preference-based measures quality of life patient-reported outcomes

Introduction

Health-related quality of life (HRQL) refers to the health aspects of quality of life, reflecting the impact of disease and treatment on disability and daily functioning.¹ This important construct is often used as a primary or secondary endpoint in multiple sclerosis (MS) research to evaluate the effectiveness of existing and new therapies from the patient’s perspective.²

An established approach to measuring HRQL is through the use of preference-based measures³ such as the EuroQol-5 dimensions (EQ-5D),⁴ the Health Utilities Index 2 and 3 (HUI 2 and 3),^5,6 and the Short Form-6 dimension (SF-6D).⁷ A disease-specific preference-based measure for people with MS was recently developed called the preference-based multiple sclerosis index (PBMSI).⁸ The domains for the PBMSI were established based on semi-structured interviews with a random sample of 185 people with MS recruited from three different MS clinics.⁹ Individual items best reflecting each domain of quality of life were identified using Rasch analysis.¹⁰ As per the US Food and Drug Administration (FDA) guidelines, the items then underwent qualitative reviewing using both expert (n = 24) and patient feedback (n = 22).¹¹ Patient preferences were then elicited for the PBMSI items using the rating scale, and a scoring algorithm was developed.⁸

The PBMSI comprises five items that patients with MS identified to be most important to their quality of life: walking, fatigue, concentration, mood, and roles and responsibilities. Each item includes three response levels, producing 243 (35) combination of responses. The PBMSI items have been previously published.¹¹

The next step is to evaluate the construct validity of the PBMSI in the population for which it was developed for (i.e. people with MS). Construct validity refers to the extent to which scores of a measure are consistent with hypotheses formulated regarding internal relationships (structural validity), relationships with other measures (convergent validity), and differences between groups (known-groups validity).¹²

Therefore, the objective of this paper was to evaluate the structural, convergent, and known-groups validity of the PBMSI in people with MS.

Methods

Setting and procedure

The sample under study were people with MS participating in a randomized clinical trial of exercise. The protocol for this study has been published.¹³ Participants were recruited from three MS clinics in the Montreal area and were aged 19–65, diagnosed after 1994, ambulatory, and able to speak and read English or French. Participants were excluded if they had an additional illness that restricted their function, had suffered at least one relapse during the past 30 days, or were unable to respond to simple questions on orientation and memory.

The study was approved by the hospital’s ethics committee and written informed consent was obtained from participants prior to study commencement.

Measures

Measures of HRQL

PBMSI

The PBMSI is a brief patient-reported outcome measure of HRQL. It consists of five items: walking, fatigue, mood, concentration, and roles and responsibilities.¹¹ Each item has three response options, and the recall time frame is ‘over the past week’. The PBMSI scoring algorithm was developed using a multiplicative multi-attribute utility function.¹⁴ The algorithm provides a score of HRQL from 0 (dead) to 1 (perfect health).

RAND-36

The RAND-36 Item Health Survey (RAND-36) is a generic health profile that consists of two summary scales: a physical component summary (PCS) and a mental component summary (MCS). The RAND-36 was included as a comparison measure because a published systematic review revealed that the RAND-36 was the most widely used generic health profile in MS.² Furthermore, it has demonstrated good internal consistency, convergent and discriminate validity in MS.^15,16

SF-6D

The SF-6D is a generic preference-based measure derived from the SF-36 Health Survey (or RAND-36).⁷ The SF-6D has six domains: physical functioning, role limitation, social functioning, pain, mental health, and vitality. The SF-6D has demonstrated moderate to strong correlations against other measures of participation and HRQL in MS¹⁷ and is part of the set of preference-based measures recommended by the Canadian Agency for Drugs and Technologies in Health (CADTH).

EQ-5D

The EQ-5D is a generic preference-based measure of HRQL that consists of five items or domains: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression.^4,18–20 Each domain has three levels: no problems, some problems, and extreme problems. The EQ-5D has demonstrated small to moderate correlations against other measures evaluating activity, participation, and HRQL in MS.¹⁷ The EQ-5D is a recommended preference-based measure for economic evaluation by CADTH and the National Institute for Health and Care Excellence (NICE).

The patient-generated index

The patient-generated index (PGI) is an individualized measure of quality of life.²¹ Participants were asked to identify up to five of the most important areas of their lives affected by MS. The PGI produces a score from 0 to 100. The PGI has demonstrated moderate correlations with the generic preference-based measures EQ-5D and SF-6D in MS.¹⁷

Self-rated health

Self-rated health is as an individual’s rating of health based on his or her own perception, experience, and frame of reference.¹ Participants were asked to rate their health state today and over the past week using a visual analogue scale (VAS) from 0 to 100, where 0 is the worst imaginable health state and 100 is the best imaginable health state. These two scales are labeled as VAS-today and VAS-week, respectively.

MS symptom checklist

Subjects were provided with a symptom list and asked to identify if they had experienced any of the symptoms over the past week. Symptoms listed were as follows: loss of co-ordination, weakness in the lower extremities, unsteadiness or loss of balance, problems with bladder, muscle stiffness or spasms, pain, feeling frustrated, and problems with sleep.

Measures of disability

6-minute walk test

The 6-minute walk test (6-MWT) is a simple performance-based test that measures functional exercise capacity. Individuals are instructed to walk as far as possible in an empty corridor, in 6 minutes at their own intensity. The 6-MWT has demonstrated excellent intra- and inter-rater reliability in MS.²²

Patient-determined disease steps

The patient-determined disease steps (PDDS) is a self-reported outcome of disability in MS.²³ It has nine ordinal levels ranging between 0 (normal) and 8 (Bedridden) and PDDS scores can be converted into classifications of mild, moderate, or severe disability. The PDDS is a surrogate measure of the Expanded Disability Status Scale (EDSS) and has shown to be strongly correlated with the EDSS.²³

Peak power

Peak power was measured using an incremental graded cycle ergometer test. All persons started the test at a minimal power output of 10 W with a gradual increase of power output by 10 W per minute. The measure of performance was the highest power output individuals were able to complete. Peak power output was included as a disability measure as it has been shown to be directly associated with peak exercise capacity (VO_2peak).²⁴ Individuals who are able to achieve higher work load have better lung function and greater breathing reserve.²⁵

Statistical analysis

Floor and ceiling effects

Floor and ceiling effects were calculated for the PBMSI, EQ-5D, SF-6D, and self-rated health. The percentage of respondents who had minimum and maximum scores on the measures was calculated. Values >15% were indicative of a floor or ceiling effect.²⁶

Structural validity

Preference-based measures developed using multi-attribute utility theory like the PBMSI should comprise items that are independent or semi-independent from each other.¹⁴ Therefore, polychoric correlation coefficients were calculated between the items in the PBMSI to assess structural independence between the items. Polychoric correlations are an extension of tetrachoric correlations but for variables that have been categorized into three or more classes (i.e. PBMSI had three response levels per item).¹ We hypothesized that the correlation coefficient values between items would be low to moderate.

Structural validity was further assessed using exploratory common factor analysis. The Kaiser–Guttman rule,²⁷ which states that all factors having an eigenvalue greater than 1 should be retained, was used to identify the number of factors in the PBMSI. We hypothesized that one factor would be retained from the analysis representing the construct of HRQL.

Convergent validity

To demonstrate convergent validity, hypotheses were formulated about the strength of correlations between the PBMSI and other HRQL measures. A correlation ⩽0.30 was considered small, between 0.31 and 0.59 moderate, and ⩾0.60 as strong.^28,29

We anticipated low to moderate correlations between the PBMSI and measures of disability (PDDS, 6-MWT, and peak power). We expected to observe moderate correlations between the PBMSI and the generic HRQL measures (RAND-36, EQ-5D, SF-6D, VAS-today, and VAS-week) as some items would be similar while others different (specific to MS). Furthermore, we expected to observe moderate correlations with the individualized measure (PGI), as the PGI measures the broader construct of quality of life.

Known-groups validity

Known-groups validity for the PBMSI was assessed against different measures of disability, namely, the PDDS, the 6-MWT, and peak power. For the PDDS, individuals with PDDS scores between 0 and 2 were classified as mild disability, scores 3 and 4 were classified as moderate disability, and scores of 5 and greater were severe disability. These cut-offs were based on individual’s level of impairment with walking as described by the PDDS descriptive system. Individuals in levels 1 and 2 had mild symptoms but no limitations in walking ability. Individuals in levels 3 and 4 had some limitations in walking and needed to use a cane occasionally, and individuals in levels 5 and 6 needed constant use of a cane for walking or a scooter for long distances. For the 6-MWT, individuals who were able to walk greater than 500 m^30,31 were categorized as having high functional walking capacity, those who walked between 300 and 499 m were classified as moderate, and less than 300 as poor. As for peak power (in watts) measured using the cycle ergometer test, power output between 140 and 240 W was high, 60–139 W moderate, and less than 60 W poor. We hypothesized that individuals with higher levels of disability would have lower scores on the PBMSI than those with lower levels of disability. The EQ-5D and SF-6D’s ability to discriminate between different levels of disability were also assessed and compared with the PBMSI.

Known-groups validity was also assessed against the presence/absence of MS symptoms: specifically, loss of co-ordination, weakness in the lower extremities, unsteadiness or loss of balance, problems with bladder, muscle stiffness or spasms, pain, feeling frustrated, and problems with sleep. We hypothesized that PBMSI scores would be lower in individuals who reported experiencing a symptom than those who did not. For example, we expected people who experienced unsteadiness or loss of balance to have lower scores on the PBMSI than individuals who did not experience this symptom.

Statistically significant differences between known-groups were assessed using independent t-test for dichotomous variables and analysis of variance (ANOVA) for variables with more than two categories. Effect sizes (ESs) and 95% confidence intervals (CIs)³² were calculated to determine the magnitude of difference between the different known groups. Cohen’s criteria³³ were used for interpreting magnitude of ES, where an ES of ~0.2 is small, ~0.5 is moderate, and ~0.8 is large. An ES was statistically significant if the CI excluded 0.

Results

Sample

Table 1 presents the demographic and clinical characteristics for women and men. The average age for women and men was similar at 46 and 47 years, respectively. The average number of years since diagnosis was 6.4 for women and 9.6 for men. As for the PDDS, approximately the same proportion of men and women (30%–34%) reported having minor MS symptoms. In total, 41% of women and 26% of men reported having limitations in daily activities or physically demanding activities.

Table 1.

Characteristics of study sample (n = 113).

Sample characteristics	Mean ± SD or N (%)/n (%)
Sample characteristics	Total sample (n = 113)	Women (n = 87)	Men (n = 26)
Age (years)	46.0 ± 10.1	45.7 ± 9.7	47 ± 11.5
Language spoken: English/French	41 (36)/72 (64)	31 (36)/56 (64)	10 (38)/16 (62)
Education: High school/college or university*	6 (6)/96 (94)	3 (3)/83 (97)	3 (12)/21 (88)
Years since diagnosis	7.1 ± 5.4	6.4 ± 5.2	9.6 ± 5.5
Expanded Disability Status Scale* (median, interquartile range)	2.1 (SD, 1.1) 2.0 (1.5–3.0)	2.0 (0.9) 2.0 (1.5–2.8)	2.4 (1.4) 2.3 (1.5–3.0)
Patient-determined disease steps*
0–1 (Normal to minor MS symptoms)	33 (34)	25 (34)	7 (30)
2–3 (Limitations in daily activities or physically demanding activities)	36 (38)	30 (41)	6 (26)
4–5 (Need of 1 cane or crutch for walking)	27 (28)	18 (25)	9 (39)
6+ (Need of 2 canes or crutches for walking)	1 (1)	0 (0)	1 (4)
General health perception*
Excellent–Very good	31 (31)	22 (28)	9 (37)
Good	58 (57)	48 (62)	10 (42)
Fair–Poor	13 (13)	8 (10)	5 (21)

SD: standard deviation.

Total sample size = 110 for Education, 52 for Expanded Disability Status Scale, 96 for patient-determined disease steps, and 102 for general health perception.

Structural validity

Table 2 presents the correlations between the items, which ranged from −0.05 to 0.60, thus supporting our a priori hypothesis that the items were low to moderately correlated with each other.

Table 2.

Correlation matrix of PBMSI items.

	PBMSI1 (Walking)	PBMSI2 (Fatigue)	PBMSI3 (Mood)	PBMSI4 (Concentration)	PBMSI5 (Roles and responsibilities)
PBMSI1 (Walking)	1.0	0.37	−0.05	0.01	0.32
PBMSI2 (Fatigue)		1.00	0.34	0.34	0.60
PBMSI3 (Mood)			1.00	0.51	0.46
PBMSI4 (Concentration)				1.00	0.52
PBMSI5 (Roles and responsibilities)					1.00

PBMSI: preference-based multiple sclerosis index.

The first factor had an eigenvalue greater than 1 (= 1.19) and was the only one retained. The remaining eigenvalues were 0.28, −0.11, −0.20, and −0.21.

Table 3 presents the mean scores on the PBMSI, EQ-5D, and SF-6D, which were 0.25, 0.81, and 0.68, respectively. The mean values on the VAS for health state today and over the past week were similar, with the former being 74 and the latter being 70 out of 100. The PBMSI demonstrated no floor effects. As for ceiling effects, there were 10 individuals (9% of the sample) who had a score of 1.0 on the PBMSI, but this value was less than the cut-off of 15%. The EQ-5D had no floor effects but did demonstrate ceiling effects as 19% of the sample had a score of 1.0. For the SF-6D, the full range of health (0–1) is not represented in this scale, as the theoretical range is between 0.3 and 1.0. However, within this range, there were no individuals (0%) who reported the lowest and highest scores possible.

Table 3.

Percentage of respondents with minimum (floor effect) and maximum (ceiling effect) scores on the PBMSI, EQ-5D, SF-6D, PGI, and VAS.

Measures	Theoretical range	Observed range	Mean ± SD	Floor effectN (%)	Ceiling effectN (%)
PBMSI	0, 1	0.005, 1	0.25 ± 0.27	0 (0)	10 (9)
EQ-5D	−0.590, 1	0.397, 1	0.81 ± 0.13	0 (0)	19 (19)
SF-6D	0.296, 1	0.424, 0.894	0.68 ± 0.11	0 (0)	0 (0)
PGI	0, 1	0, 1	0.46 ± 0.21	2 (3)	2 (3)
VAS-week	0, 100	30, 100	70.0 ± 13.0	0 (0)	2 (3)
VAS-today	0, 100	40, 100	73.6 ± 14.4	0 (0)	5 (5)

EQ-5D: EuroQol-5 dimensions; PBMSI: preference-based multiple sclerosis index; SF-6D: Short Form-6 dimension; VAS: visual analogue scale.

For VAS-week n = 80 and VAS-today n = 103.

Table 4 presents the responses to the five items in the PBMSI and the EQ-5D. A person who responded as having no problems on the five items was classified as 11111. A person who reported having moderate problems on any one of the five items was classified as 21111 or 12111 or 11211. There were less people who reported having no problems (11111) on the five items for the PBMSI (9%) than the EQ-5D (19%). There were more people who reported having some problems (i.e. chose response option 2) and severe problems (i.e. chose response option 3) on the PBMSI than the EQ-5D.

Table 4.

Reported health states on the PBMSI and EQ-5D.

Health state	PBMSI* N (%)	EQ-5D** N (%)
No problems on any of the items (i.e. 11111)	10 (9%)	19 (19%)
Moderate problem on one item (i.e. 21111, 12111, 11211, 11121, 11112)	12 (11%)	27 (26%)
Moderate problem on two items (i.e. 22111, 12211, 11221)	31 (28%)	23 (23%)
Moderate problems on three items (i.e. 22211, 12221, 11222)	23 (21%)	17 (17%)
Moderate problems on four items (i.e. 22221, 12222, 21222)	14 (13%)	9 (9%)
Moderate problems on all items (i.e. 22222)	9 (8%)	1 (1%)
Severe problem on one or more items (i.e. 11323, 11231, 12231)	12 (11%)	6 (6%)
Severe problem on all items (i.e. 33333)	0 (0%)	0 (0%)

PBMSI: preference-based multiple sclerosis index; EQ-5D: EuroQol-5 dimensions.

n = 111; **n = 102.

Convergent validity

Table 5 presents the correlation coefficient values between the PBMSI and other measures of HRQL. The correlation between the PBMSI and EQ-5D was 0.37, and the SF-6D was 0.66. The association between the PBMSI and the PGI was low to moderate at r = 0.32 (p value = 0.001). Also as expected, the correlations between the PBMSI and the physical tests were moderate.

Table 5.

Convergent validity: correlation (and p values) between the PBMSI, other measures of HRQL, self-rated health, disease severity, and functional capacity.

	PBMSI	EQ-5D	SF-6D	PGI	RAND-36 PCS	RAND-36 MCS	VAS-week	VAS-today	PDDS	6-MWT	Peak power
PBMSI	1.0	0.37 (0.0001)	0.66 (<0.0001)	0.32 (0.001)	0.40 (<0.0001)	0.42 (<0.0001)	0.48 (<0.0001)	0.38 (<0.0001)	−0.27 (0.0084)	0.32 (0.0006)	0.29 (0.0024)
EQ-5D		1.0	0.53 (<0.0001)	0.27 (0.007)	0.39 (<0.0001)	0.36 (0.0003)	0.51 (<0.0001)	0.56 (<0.0001)	0.31 (0.003)	0.35 (0.0004)	0.20 (0.041)
SF-6D			1.0	0.22 (0.034)	0.46 (<0.0001)	0.65 (<0.0001)	0.57 (<0.0001)	0.47 (<0.0001)	−0.12 (0.281)	0.20 (0.05)	0.21 (0.04)
PGI				1.0	0.27 (0.006)	0.14 (0.155)	0.37 (0.001)	0.28 (0.004)	−0.20 (0.056)	0.25 (0.007)	0.21 (0.03)
RAND-36 PCS					1.0	−0.22 (0.032)	0.55 (<0.0001)	0.44 (<0.0001)	−0.44 (<0.0001)	0.46 (<0.0001)	0.32 0.002
RAND-36 MCS						1.0	0.31 (0.011)	0.27 (0.007)	0.08 (0.462)	−0.12 (0.237)	−0.03 (0.774)
VAS-week							1.0	0.74 (<0.0001)	−0.34 (0.006)	0.27 (0.015)	0.20 (0.085)
VAS-today								1.0	−0.34 (0.001)	0.30 (0.002)	0.22 (0.033)
PDDS									1.0	−0.48 (<0.0001)	−0.18 (0.073)
6-MWT										1.0	0.65 (<0.0001)
Peak power											1.0

PBMSI: preference-based multiple sclerosis index; EQ-5D: EuroQol-5 dimensions; HRQL: health-related quality of life; PDDS: patient-determined disease step; SF-6D: Short Form-6 dimension; VAS: visual analogue scale; 6-MWT: 6-minute walk test; PGI: patient-generated index; PCS: physical component summary; MCS: mental component summary.

Furthermore, moderate correlations were observed between the PBMSI and the RAND-36 PCS (r = 0.40, p value = <0.0001) and MCS (r = 0.42, p value = <0.0001). Moderate correlations were also observed between the PBMSI and the VAS for health state today (r = 0.4, p value = <0.0001), and slightly higher correlations between the PBMSI and the VAS for health state over the past week (r = 0.5, p value = <0.0001).

Known-groups validity

Table 6 presents the known-groups validity results for the PBMSI, EQ-5D, and SF-6D against different levels of the PDDS, 6-MWT, and peak power.

Table 6.

Known-groups validity: PBMSI, EQ-5D, and SF-6D scores by disease severity, distance walked in 6 minutes, and peak power in watts.

	PBMSI		EQ-5D		SF-6D
	Mean (SD)	Effect size (95% CI)	Mean (SD)	Effect size (95% CI)	Mean (SD)	Effect size (95% CI)
PDDS*
Mild (n = 49)	0.32 (0.35)	0.57 (0.14 to 1.0)	0.85 (0.12)	0.67 (0.24 to 1.10)	0.70 (0.12)	0.36 (−0.07 to 0.78)
Moderate (n = 39)	0.16 (0.15)	0.21 (−0.65 to −1.11)	0.77 (0.12)	0.09 (−0.77 to 0.95)	0.66 (0.10)	−0.31 (−1.12 to 0.56)
Severe (n = 6)	0.13 (0.10)		0.76 (0.09)		0.69 (0.08)
6-minute walk test*
>500 m (n = 56)	0.33 (0.33)	0.55 (0.16 to 0.95)	0.86 (0.09)	0.71 (0.31 to 1.10)	0.71 (0.11)	0.46 (0.07 to 0.84)
300–499 m (n = 49)	0.18 (0.18)	0.46 (−0.57 to 1.48)	0.78 (0.13)	1.14 (0.10 to 2.19)	0.66 (0.11)	0 (−1.10 to 1.02)
0–299 m (n = 4)	0.10 (0.06)		0.63 (0.15)		0.66 (0.11)
Peak power*
140–240 W (n = 23)	0.37 (0.33)	0.62 (0.13 to 1.10)	0.84 (0.13)	0.15 (−0.32 to 0.63)	0.69 (0.12)	0.18 (−0.3 to 0.66)
60–139 W (n = 64)	0.21 (0.23)	0.50 (−0.20 to 1.21)	0.82 (0.13)	0.91 (0.19 to 1.61)	0.67 (0.11)	0.37 (−0.33 to 1.07)
<60 W (n = 9)	0.10 (0.08)		0.70 (0.15)		0.63 (0.08)

PBMSI: preference-based multiple sclerosis index; CI: confidence interval; EQ-5D: EuroQol-5 dimensions; PDDS: patient-determined disease step; SD: standard deviation; SF-6D: Short Form-6 dimension; 6-MWT: 6-minute walk test.

p < 0.05 using analysis of variance (ANOVA) to discriminate across the three levels for the PBMSI and EQ-5D only.

For the PDDS, the PBMSI was able to differentiate between individuals with mild and moderate disability (ES = 0.57), as was the EQ-5D (ES = 0.67) and the SF-6D (ES = 0.36). The PBMSI was able to differentiate between people with moderate and severe disability (ES = 0.21), whereas the EQ-5D and SF-6D were not able to (ES = 0.09 and ES = –0.31, respectively).

For the 6-MWT, the PBMSI was able to differentiate between individuals with high and moderate functional capacity (ES = 0.55) and moderate and low functional capacity (ES = 0.46). The EQ-5D was also able to differentiate between the different levels of walking capacity, whereas the SF-6D was only able to differentiate between high and moderate. An ES of 0 was observed between moderate and low functional capacity on the SF-6D.

PBMSI scores decreased as peak power output decreased. The PBMSI was able to differentiate between high and moderate peak power better than the EQ-5D and SF-6D.

Figure 1 presents the results of the symptom checklist. The most prevalent symptoms were unsteadiness or loss of balance (47%), weakness in the legs (42%), and muscle stiffness or spasms (38%). Figure 2 shows that the PBMSI was able to differentiate between people who reported having a symptom versus those who did not. The ES values ranged from moderate (ES = 0.41) to large (ES = 0.91). The ES values for the PBMSI were all statistically significant, as the 95% CI excluded zero. ES values were larger for the PBMSI, than the EQ-5D and the SF-6D, when differentiating between people with and without balance problems, weakness in legs, muscle stiffness, and feeling frustrated.

Figure 1.

Results of the MS symptom checklist.

Figure 2.

Known-groups validity of the PBMSI, EQ-5D and SF-6D in terms of ability to discriminate between people with and without MS symptoms.

Discussion

This study evaluated the construct validity of the PBMSI in the setting of a randomized controlled trial (RCT) in people with MS, where one-third of the sample reported minor symptoms and described themselves as being in excellent or very good health. The PBMSI showed good convergent validity against generic measures of HRQL, supporting a priori hypothesized correlation values. The PBMSI was moderately correlated with the EQ-5D and strongly correlated with the SF-6D. According to the Kaiser and Guttman rule, only one factor was retained. In other words, factor analysis supported the structural validity of the PBMSI that the items collectively were measuring one underlying construct (HRQL). Known-groups validity was demonstrated between persons with different levels of disability. The PBMSI was able to discriminate better than the EQ-5D and the SF-6D for certain MS-specific symptoms, including balance, weakness in legs, muscle stiffness, and feeling frustrated.

The PBMSI in comparison to the EQ-5D and SF-6D

The PBMSI was moderately associated with the EQ-5D and strongly associated with the SF-6D. Higher correlations were observed between the PBMSI and SF-6D than between the PBMSI and EQ-5D probably because the SF-6D includes an item on fatigue (i.e. vitality) but the EQ-5D does not.

The PBMSI scores were on average lower than the EQ-5D and SF-6D scores. The sample had a mean HRQL score of 0.25 on the PBMSI, but a mean score of 0.81 and 0.68 on the EQ-5D and SF-6D, respectively. This may be because the items in the PBMSI were more relevant and likely to be impacted by MS than the items in the generic measures. As demonstrated in Table 4, there were less people who reported having no problems on the PBMSI (n = 10) than the EQ-5D (n = 19). Conversely, there were more people who reported having moderate or severe problems on the PBMSI (n = 101) than the EQ-5D (n = 83).

Another explanation for the discrepancy in scores may be because the preference weights for the PBMSI were obtained from people with MS, whereas the weights for the EQ-5D and SF-6D were obtained from the general public. Interesting to note is that the average score for the PGI, whose weighting system is based on patient preferences, was also less than the generic preference-based measures. Furthermore, the methods that were used to obtain preference weights differed between each of the preference-based measures. The PBMSI scoring algorithm was based on the rating scale method, the EQ-5D was based on the time trade-off, and the SF-6D on the standard gamble, which may also account for the discrepancies observed in scores.

Bias and generalizability

A limitation of this study was that the sample was predominantly individuals with mild to moderate disease severity. The inclusion criteria for the clinical trial required participants to be ambulatory, therefore the highest PDDS score observed was a level 5 or 6 (requiring support to walk 25 feet, wheelchair for greater distances), who were only 6% of the sample. The other 94% were individuals between levels 1 (minor symptoms) and 4 (need of a cane or crutch). For this reason, we were unable to thoroughly assess the ability of the PBMSI to differentiate between moderate and severe MS disability.

Furthermore, the PBMSI was not compared to disease-specific health measures, such as the Multiple Sclerosis Impact Scale (MSIS)-29^34,35 or the MSIS-8.^36,37 Future work should entail performing a direct comparison of the PBMSI against these measures.

Conclusion

Disease-specific preference-based measures have been developed for different conditions such as stroke,³⁸ cancer,³⁹ and asthma.⁴⁰ Disease-specific measures are designed to fill in the gaps in generic measures by tapping specific domains. The PBMSI is the first preference-based measure developed in MS using patient preferences.

In conclusion, the domains and items for the PBMSI were developed based on interviews with people with MS, ensuring that the measure had content validity in this clinical population. The results of this study support the structural validity of the PBMSI as an outcome measure of HRQL and its convergent validity against other measures of HRQL. Moreover, the PBMSI was able to differentiate between individuals with and without MS symptoms, as well as those with mild and moderate disability. However, validation is a continuous process that develops as a measure is applied in new situations and contexts. Therefore, future work with the PBMSI will need to involve assessing its longitudinal validity including responsiveness and sensitivity to change.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The randomized controlled trial was supported by the Canadian Institutes of Health Research (grant number 119282).

References

Mayo

ISOQOL dictionary of quality of life and health outcomes measurement. Montreal, QC, Canada: Reseau De Recherche Em Santé Des Populations Du Quebec, 2015.

Kuspinar

Rodriguez

Mayo

NE.

The effects of clinical interventions on health-related quality of life in multiple sclerosis: A meta-analysis. Mult Scler 2012; 18: 1686–1704.

Guyatt

Feeny

Patrick

DL.

Measuring health-related quality of life. Ann Intern Med 1993; 118: 622–629.

Kind

The EuroQoL instrument: An index of health-related quality of life. Qual Life Pharmacoeconomics Clin Trials 1996; 2: 191–201.

Torrance

Feeny

Furlong

, et al. Multiattribute utility function for a comprehensive health status classification system: Health Utilities Index Mark 2. Med Care 1996; 34: 702–722.

Feeny

Furlong

Torrance

, et al. Multiattribute and single-attribute utility functions for the health utilities index mark 3 system. Med Care 2002; 40: 113–128.

Brazier

Roberts

Deverill

The estimation of a preference-based measure of health from the SF-36. J Health Econ 2002; 21: 271–292.

Kuspinar

Pickard

Mayo

NE.

Developing a valuation function for the preference-based multiple sclerosis index: Comparison of standard gamble and rating scale. PLoS ONE 2016; 11: e0151905.

Kuspinar

Mayo

NE.

Do generic utility measures capture what is important to the quality of life of people with multiple sclerosis?

Health Qual Life Outcomes 2013; 11: 71.

10.

Kuspinar

Finch

Pickard

, et al. Using existing data to identify candidate items for a health state classification system in multiple sclerosis. Qual Life Res 2014; 23: 1445–1457.

11.

Kuspinar

Bouchard

Moriello

, et al. Development of a bilingual MS-specific health classification system: The preference-based multiple sclerosis index. Int J MS Care 2016; 18: 63–70.

12.

De Vet

Terwee

Mokkink

, et al. Measurement in medicine: A practical guide. Cambridge: Cambridge University Press, 2011.

13.

Mayo

Bayley

Duquette

, et al. The role of exercise in modifying outcomes for people with multiple sclerosis: A randomized trial. BMC Neurol 2013; 13: 69.

14.

Torrance

Boyle

Horwood

SP.

Application of multi-attribute utility theory to measure social preferences for health states. Oper Res 1982; 30: 1043–1069.

15.

Freeman

Hobart

Langdon

, et al. Clinical appropriateness: A key factor in outcome measure selection: The 36 item short form health survey in multiple sclerosis. J Neurol Neurosurg Psychiatry 2000; 68: 150–156.

16.

Nortvedt

Riise

Myhr

K-M

, et al. Performance of the SF-36, SF-12, and RAND-36 summary scales in a multiple sclerosis population. Med Care 2000; 38: 1022–1028.

17.

Kuspinar

Mayo

NE.

A review of the psychometric properties of generic utility measures in multiple sclerosis. Pharmacoeconomics 2014; 32: 759–773.

18.

Dolan

Modeling valuations for EuroQol health states. Med Care 1997; 35: 1095–1108.

19.

Johnson

Luo

Shaw

, et al. Valuations of EQ-5D health states: Are the United States and United Kingdom different? Med Care 2005; 43: 221–228.

20.

Shaw

Johnson

Coons

SJ.

US valuation of the EQ-5D health states: Development and testing of the D1 valuation model. Med Care 2005; 43: 203–220.

21.

Ruta

Garratt

Leng

, et al. A new approach to the measurement of quality of life: The patient-generated index. Med Care 1994; 32: 1109–1126.

22.

Paltamaa

West

Sarasoja

, et al. Reliability of physical functioning measures in ambulatory subjects with MS. Physiother Res Int 2005; 10: 93–109.

23.

Hohol

Orav

Weiner

Disease steps in multiple sclerosis: A longitudinal study comparing disease steps and EDSS to evaluate disease progression. Mult Scler 1999; 5: 349–354.

24.

Storer

Davis

Caiozzo

VJ.

Accurate prediction of VO2max in cycle ergometry. Med Sci Sports Exerc 1990; 22: 704–712.

25.

Kaminsky

Knyazhitskiy

Sadeghi

, et al. Assessing maximal exercise capacity: Peak work or peak oxygen consumption? Respir Care 2014; 59: 90–96.

26.

McHorney

Tarlov

AR.

Individual-patient monitoring in clinical practice: Are available health status surveys adequate?

Qual Life Res 1995; 4: 293–307.

27.

Kaiser

HF.

A second generation little jiffy. Psychometrika 1970; 35: 401–415.

28.

Katherine Salter

Teasell

. Outcome measures in stroke rehabilitation, 2013, http://www.ebrsr.com/sites/default/files/Chapter%2020_Outcome%20Measures.pdf

29.

Andresen

EM.

Criteria for assessing the tools of disability outcomes research. Arch Phys Med Rehabil 2000; 81: S15–S20.

30.

Casanova

Celli

Barria

, et al. The 6-min walk distance in healthy subjects: Reference standards from seven countries. Eur Respir J 2011; 37: 150–156.

31.

Enright

Sherrill

DL.

Reference equations for the six-minute walk in healthy adults. Am J Respir Crit Care Med 1998; 158: 1384–1387.

32.

Lee

DK.

Alternatives to P value: Confidence interval and effect size. Korean J Anesthesiol 2016; 69: 555–562.

33.

Cohen

A power primer. Psychol Bull 1992; 112: 155–159.

34.

Hobart

Riazi

Lamping

, et al. Improving the evaluation of therapeutic interventions in multiple sclerosis: Development of a patient-based measure of outcome. Health Technol Assess 2004; 8: iii, 1–48.

35.

Hobart

Cano

Improving the evaluation of therapeutic interventions in multiple sclerosis: The role of new psychometric methods. Health Technol Assess 2009; 13: iii, ix–x, 1–177.

36.

Goodwin

Green

A quality-adjusted life-year measure for multiple sclerosis: Developing a patient-reported health state classification system for a multiple sclerosis-specific preference-based measure. Value Health 2015; 18: 1016–1024.

37.

Goodwin

Green

Spencer

Estimating a preference-based index for an eight-dimensional health state classification system for multiple sclerosis. Value Health 2015; 18: 1025–1036.

38.

Poissant

Mayo

Wood-Dauphinee

, et al. The development and preliminary validation of a preference-based stroke index (PBSI). Health Qual Life Outcomes 2003; 1: 43.

39.

Rowen

Brazier

Young

, et al. Deriving a preference-based measure for cancer using the EORTC QLQ-C30. Value Health 2011; 14: 721–731.

40.

Revicki

Leidy

Brennan-Diemer

, et al. Integrating patient preferences into health outcomes assessment: The multiattribute Asthma symptom utility index. Chest 1998; 114: 998–1007.