The reliability,precision and clinically meaningful change of walking assessments in multiple sclerosis

Abstract

Background:

Assessing walking impairment in those with multiple sclerosis (MS) is common, however little is known about the reliability, precision and clinically important change of walking outcomes.

Objective:

The purpose of this study was to determine the reliability, precision and clinically important change of the Timed 25-Foot Walk (T25FW), Six-Minute Walk (6MW), Multiple Sclerosis Walking Scale-12 (MSWS-12) and accelerometry.

Methods:

Data were collected from 82 persons with MS at two time points, six months apart. Analyses were undertaken for the whole sample and stratified based on disability level and usage of walking aids. Intraclass correlation coefficient (ICC) analyses established reliability: standard error of measurement (SEM) and coefficient of variation (CV) determined precision; and minimal detectable change (MDC) defined clinically important change.

Results:

All outcome measures were reliable with precision and MDC varying between measures in the whole sample: T25FW: ICC=0.991; SEM=1 s; CV=6.2%; MDC=2.7 s (36%), 6MW: ICC=0.959; SEM=32 m; CV=6.2%; MDC=88 m (20%), MSWS-12: ICC=0.927; SEM=8; CV=27%; MDC=22 (53%), accelerometry counts/day: ICC=0.883; SEM=28450; CV=17%; MDC=78860 (52%), accelerometry steps/day: ICC=0.907; SEM=726; CV=16%; MDC=2011 (45%). Variation in these estimates was seen based on disability level and walking aid.

Conclusion:

The reliability of these outcomes is good and falls within acceptable ranges. Precision and clinically important change estimates provide guidelines for interpreting these outcomes in clinical and research settings.

Keywords

Multiple sclerosis walking mobility EDSS reliability precision clinical change outcome measures

Introduction

Multiple sclerosis (MS) is a prevalent immune-mediated, progressive and disabling neurological disease. Walking impairment is an almost ubiquitous feature of MS,¹ affecting approximately 80% of cases² and represents a valued function across the disability spectrum.³ Thus, walking is a common metric for monitoring disease progression and a central outcome of pharmacotherapy and rehabilitation interventions.^4,5

Multiple outcomes exist for measuring walking in MS, including the Timed 25-Foot Walk (T25FW),⁶ Six-Minute Walk (6MW),⁷ Multiple Sclerosis Walking Scale-12 (MSWS-12)⁸ and free-living accelerometry.^9,10 It is acknowledged within the literature that different walking outcome assessments used in MS research capture different aspects of walking,¹¹ and to better inform users of these scales there is a strong need to acknowledge validity and reliability to enhance decision making when choosing outcome assessments. There is evidence to support the validity of inferences from scores on all four outcomes as measures of walking impairment.^12–14 The T25FW and 6MW have differentiated between MS and controls^12,13 and between persons with MS who vary in disability based on the Expanded Disability Status Scale (EDSS).^12,14,15 MSWS-12 scores have correlated with T25FW, 6MW and gait kinematics in MS¹⁶ and differed between persons with MS who vary in disability.⁸ Accelerometer output of counts/day and steps/day have differed between MS and controls and disability level within those with MS^10,17 and correlated with T25FW, 6MW and MSWS-12 in MS.¹⁸

Less is known about the reliability of T25FW, 6MW, MSWS-12 and accelerometry metrics over time.^19,20 Reliability provides an indication of a measure’s consistency and precision over time in the absence of a change based on the intraclass correlation (ICC) coefficient, standard error of measurement (SEM) and coefficient of variation (CV). The clinical importance of change can be captured using the minimal detectable change (MDC).²¹ T25FW scores have been reliable over a one-week time period in those with MS who had EDSS scores between 5–6.5 based on an ICC of 0.94 with a SEM of 4.56 s and an MDC of 12.6 s.¹⁹ 6MW distance has been reliable over one week based on an ICC of 0.96, SEM of 30 m and MDC of 76 m. Nilsagård et al.²² has demonstrated that reliability of outcomes assessing walking may be affected by level of disability in MS, when assessed over one week.

There are limitations of previous research and reasons for further examining the reliability, precision and clinically detectable change of walking measures in MS. One limitation is the lack of research reporting ICC, SEM, CV and MDC values for MSWS-12 scores or accelerometry metrics. Another limitation is that reliability has only been established over a one-week period and primarily in homogeneous groups. These psychometric characteristics should be examined over a longer period and between MS patient characteristics such as disability status or assistive device use. Indeed, clinicians monitor disease progression based on walking outcomes during clinical visits that typically occur bi-annually. Researchers who conduct clinical trials often include walking outcomes to monitor patients over longer intervals.^23–27 We further note that ICC estimates are important components of power analyses for planning longitudinal and clinical trials. The default ICC value in some power analysis software is 0.50²⁸ but this value may be incorrect, thereby threatening the veracity of the power analysis and sample size estimates.

This study examined the reliability, precision and clinically detectable change of the T25FW, 6MW, MSWS-12 and accelerometry over a six-month period of time. We further examined those psychometric parameters between benchmarks of disability and use of a walking aid.

Method

Ethical approval was granted by a University Institutional Review Board, with all participants providing written informed consent. The protocol comprised two testing sessions, separated by six months with no intervention provided during this time.

Recruitment and participants

The data were secondary outcomes from a non-intervention, six-month observational period of an ongoing intervention targeted toward changing physical activity in MS. Participants were contacted either by a flyer that was distributed amongst patients in the North American Research Committee on Multiple Sclerosis (NARCOMS) registry or by e-mail through a flyer that was distributed amongst participants in a database from previous studies conducted. There were 511 participants who initially expressed interest and who were contacted by the project-coordinator. After explaining the study protocol, the project-coordinator undertook screening for inclusion with 230 individuals who remained interested. The inclusion criteria involved: (a) diagnosis of MS; (b) relapse-free for the past 30 days; (c) ability to walk with or without an assistive device; (d) age between 18–64 years; (e) willingness and ability to travel to the research site, complete the walking assessments and wear an accelerometer for one week; and (f) physician’s approval for participation in the study. For this study 106 individuals did not meet one or more inclusion criteria with their primary reasons being too physically active and unwilling to travel, 39 people did not provide physician’s approval and three people cancelled the testing session due to scheduling conflicts. The final sample included 82 persons with MS.

Procedures

Disability was ascertained using the Self-Report Expanded Disability Status Scale (SR-EDSS).²⁹ This SR-EDSS has demonstrated validity based on good overall agreement (ICC=0.90) and correlation (r=0.90) with the clinician-administered EDSS and a non-significant difference in overall mean scores and distributions between EDSS versions.²⁹ Four walking measures were administered during this study, the T25FW, 6MW, MSWS-12 and accelerometry and each is described in Table 1. The protocols have previously been described.^7,8,12,30,31 The same procedure was repeated six months later.

Table 1.

Description of each mobility outcome measure assessed.

Outcome measure	What is being assessed	Description
T25FW	Walking speed (ft/s) and time to walk 25 ft (s)	Following standardised instructions the participant walked as fast and safely as they could across a marked 25 ft course, static start, assistive device if required, repeated twice. Time and walking speed for each trial recorded.^{30, 31}
6MW	Walking speed (m/min)and total distance (m) covered in 6 min	Following standardised instructions the participant walked as far as possible for 6 min as fast and safely as they could along a 22 m hallway. Turning at each end, rests not permitted, walking aids allowed, 1 min time updates provided, no encouragement. Distance walked measured by assessor following 1–2 m behind with a trundle wheel. Distance and walking speed recorded.^{7, 12, 31}
MSWS-12	Impact of MS on walking ability.	Participant rated 12-item questionnaire: to assess limitations to the participant’s walking ability because of MS in the past two weeks (on a scale of 1 (not at all) to 5 (extremely)).^{8, 31}
Accelerometry	Daily step and movement count per day over seven days.	Participants wore an accelerometer (Actigraph GT3X) around their waist for seven days during waking hours (exclusive of bathing or swimming). Verbal and written instructions on use provided. Mean daily movement count per day and steps per day recorded. Activity logs completed for verification.³¹

6MW: Six-Minute Walk; MS: multiple sclerosis; MSWS-12: Multiple Sclerosis Walking Scale-12; T25FW: Timed 25-Foot Walk.

Statistical analysis

Data were analysed using PASW v18 (SPSS Inc., Chicago, Illinois, USA). ICC analyses (2,1 mixed model) were performed to assess test-retest reliability over time based on average group performance.³² Estimates close to 1.0 indicate strong reliability, scores of 0.8 suggest good reliability and ICC estimates of 0.6 suggest moderate, but still acceptable, reliability.²⁸ The SEM was calculated by multiplying the baseline standard deviation (SD) of each outcome measure by the square root of one minus the reliability coefficient; SEM = SD_baseline × √(1–ICC).^28,32,33 This value indicates the amount of variability inherent in the measurement attributable to measurement error.^20,33 CV was calculated by dividing the SD of the difference between the two time-points for the whole sample, by the mean difference between the two time points multiplied by 100.^20,33 The presented CV is a mean of this calculation which provides an estimate of consistency over time and reflects the percentage of change within the measure due to measurement error.³³ MDC was calculated by multiplying 1.96 (derived from the 95% confidence interval (CI)) by the square root of two (as two measurements were taken) times the SEM; MDC=1.96×√(2)×SEM.³⁴ The MDC reflects the amount of change not due to measurement error and therefore, may provide an indication of real change.^21,35 We further expressed the MDC as a percentage of the overall mean from both time-points. Pairwise analyses were undertaken for all reliability, precision and MDC analyses as missing data varied dependant on the outcome.

Results

Participant characteristics

Participant demographic characteristics are provided in Table 2. Participants appeared to be stable over the six-month study, based on no significant change in disability status over the two time points (SR-EDSS, p=0.871, PDDS (Patient Determined Disease Steps), p=0.118). Furthermore, no participant reported symptom relapse or clinically diagnosed relapse during the study.

Table 2.

Demographic data and multiple sclerosis (MS)-related characteristics of participants at baseline.

Common Sex	Female (n)	62
Common Sex	Male (n)	20
Age (years)	Mean (SD)	49.2 (9)
Age (years)	Range	27–64
Type of MS	Relapsing–Remitting, n (%)	65 (79)
	Secondary Progressive, n (%)	10 (12)
	Primary Progressive, n (%)	7 (9)
Disease duration since diagnosis	Mean (SD)	11.8 (8.2)
Disease duration since diagnosis	Range	0.5–32
SR-EDSS score	Mean (SD)	3.5 (2)
	Median (IQR)	3.5 (3.5)
	Range	0–6.5
	<4, n (%)	51 (62)
	4–6.5,n (%)	31 (38)
PDDS score	Mean (SD)	2.6 (1.8)
	Median (IQR)	3 (3)
	Range	0–6
	<4, n (%)	53 (65)
	4–6.5, n (%)	29 (35)
Use of walking aid	Unilateral, n (%)	10 (12)
Use of walking aid	Bilateral, n (%)	10 (12)

IQR: interquartile range; SD: standard deviation; SR-EDSS: Self-Report-Expanded Disability Status Scale. PDDS = Patient Determined Disease Steps scale.

Overall sample

Table 3 contains estimates of reliability (ICC), precision (SEM and CV) and clinically important change (MDC) for the whole sample. The ICC values for the test-retest reliability over six months ranged between 0.883–0.991. This indicates strong (ICC>0.8) reliability. The SEM and CV estimates provided an indicator of precision (i.e. error of the measurement) and should be considered alongside the mean scores. The T25FW and the 6MW had the best estimates of precision (CV and SEM), whereas the MSWS-12 and accelerometer metrics had less, but still acceptable, precision. The SEM for the 6MW was 32 m (where the mean score was 439 m), indicating that a change of 32 m or less may be due to measurement noise. The CV for the 6MW indicated that a change in score of 6.2% or less over six months may be expected with the 6MW and thus interpreted as no change. By comparison, the SEM for the MSWS-12 was eight points (where the mean score was 41), indicating that a change of eight points or less may be due to measurement noise. The CV for the MSWS-12 indicated that a change of 27% or less over six months may be interpreted as normal.

Table 3.

Showing mean score, ICC score, SEM, CV and MDC results for mobility outcomes in 82 persons with multiple sclerosis (MS).

Test	Baseline mean (SD, SE)	Six month mean (SD, SE)	Overall mean (SD, SE)	ICC	95% CI ICC	SEM	CV (%)	MDC₉₅	% MDC₉₅
Timed 25-Foot Walk time (s)^a	7.6 (10.3, 1.2)	7.5 (11.1, 1.3)	7.5 (10.6, 1.2)	0.991	0.986–0.994	1	6.2	2.7	36
Timed 25-Foot speed (ft/s)^a	0.3 (0.41, 0.05)	0.3 (0.44, 0.05)	0.3 (0.43, 0.05)	0.991	0.986–0.994	0.1	6.2	0.1	36
Six-Minute Walk test distance (m)^a	428 (156, 58.1)	449 (159.5, 59.3)	439 (156, 58.1)	0.959	0.936–0.974	32	6.2	88	20
Six-Minute Walk test speed (m/min)^a	71.4 (26.1, 3)	74.9 (26.6, 3)	73.1 (26.1, 3)	0.959	0.936–0.974	5.3	6.2	14.6	20
Multiple Sclerosis Walking Scale	40 (29.6, 3.3)	41 (30.8, 3.4)	41 (30, 3.3)	0.927	0.889–0.952	8	27	22	53
Accelerometer count (/day)^b	148,550 (83,175, 9439)	157,500 (100,334, 11,746.1)	153,025 (87,191, 10,065)	0.883	0.815–0.926	28,450	17	78,860	52
Accelerometer steps(/day)^b	4417 (2380, 285)	4557 (2922, 350)	4487 (2549, 308)	0.907	0.853–0.941	726	16	2011	45

CI: confidence interval; CV: coefficient of variation; ICC: intraclass correlation coefficient (2,1); MDC₉₅: minimal detectable change (at 95% CI); SD: standard deviation; SE: standard error of the mean; SEM: standard error of the measurement. ^a4 missing cases, ^b7 missing cases.

The MDC provides an indication of a clinically important change and is based on the SEM: it too should be considered alongside the overall mean. The 6MW and T25FW had the smallest estimates of clinical importance, whereas the MSWS-12 and accelerometer metrics had the largest estimates. The MDC for the 6MW was 88 m and a 20% change would indicate a clinical change. The MDC for MSWS-12 scores was 22 points and a 53% change would indicate a clinically important change.

Subsamples of disability and aid use

Tables 3 and 4 provided estimates of reliability, precision and clinically important change for subsamples based on disability categories (i.e. mild disability=SR-EDSS<4; moderate disability=SR-EDSS≥4) and mobility use (i.e. no walking aid; unilateral or bilateral support), respectively. When the results were dichotomised based on SR-EDSS status, distribution of scores for the MSWS-12 were similar in both groups (i.e. SD and SE results) but for the timed walking outcomes (T25FW, 6MW) and accelerometer metrics, scores were distributed wider in the more disabled group. ICC estimates indicated that all walking measures were stable over time for the group with mild disability (ICC range=0.875–0.981). The ICC estimates were strong for T25FW, 6MW and accelerometer steps/day for the group with moderate disability, but weaker, although still acceptable, for accelerometer counts/day and MSWS-12 score (i.e. ICC<0.8). Precision estimates for the T25FW and 6MW were better for those who were less disabled. By comparison, precision estimates for accelerometer metrics were comparable between disability groups and precision for the MSWS-12 was better in the group with moderate disability (Table 4). MDC was higher in those who had moderate disability for the 6MW, T25FW and accelerometer metrics, but lower for the MSWS-12, compared with those who had mild disability.

Table 4.

Showing mean score, ICC score, SEM, CV and MDC results for mobility outcomes in 82 persons with multiple sclerosis (MS) based on Self-Report Expanded Disability Status (SR-EDSS) score.

Test	SR-EDSS	Baseline mean (SD, SE)	Six month mean (SD, SE)	Overall mean (SD, SE)	ICC	95% CI ICC	SEM	CV (%)	MDC₉₅	% MDC₉₅
Timed 25-Foot Walk time(s)^a	<4	4.9 (1.5, 0.2)	4.9 (1.6, 0.2)	4.9 (1.5, 0.2)	0.981	0.967–0.989	0.2	4.6	0.6	12
Timed 25-Foot Walk time(s)^a	4–6.5	12.5 (16.1, 3.1)	12.1 (17.6, 3.3)	12.3 (16.8, 3.2)	0.990	0.979–0.995	1.6	9.1	4.5	36
Timed 25-Foot Walk speed (ft/s)^a	<4	0.2 (0.06, 0.1)	0.2 (0.06, 0.1)	0.2 (0.06, 0.1)	0.981	0.967–0.989	0.01	4.6	.02	12
Timed 25-Foot Walk speed (ft/s)^a	4–6.5	0.5 (0.6, 1.2)	0.5 (0.7, 0.1)	0.5 (0.7, 0.1)	0.990	0.979–0.995	0.06	9.1	0.17	36
Six-Minute Walk test distance (m)^a	<4	507 (95.2, 44.6)	526 (93.2, 43.7)	516 (93.1, 43.6)	0.952	0.916–0.972	21	3.8	58	11
Six-Minute Walk test distance (m)^a	4–6.5	294.7 (149.6, 91.2)	319.5 (165.3, 100.7)	307 (154.3, 94.1)	0.916	0.830–0.960	43	10.3	120	39
Six-Minute Walk test speed (m/min)^a	<4	84.5 (15.9, 2.2)	87.6 (15.5, 2.2)	86.1 (15.5, 2.2)	0.952	0.916–0.972	3.5	3.8	9.7	11
Six-Minute Walk test speed (m/min)^a	4–6.5	49.1 (24.9, 4.6)	53.2 (27.6, 5.1)	51.2 (25.7, 4.8)	0.916	0.830–0.960	7.2	10.3	20	39
Multiple Sclerosis Walking Scale	<4	22 (19, 2.7)	23 (21, 3.0)	23 (20, 2.8)	0.875	0.792–0.927	7	37	19	81
Multiple Sclerosis Walking Scale	4–6.5	70 (18, 3.2)	71 (19, 3.3)	70 (17, 3.1)	0.716	0.488–0.852	10	12	27	38
Accelerometer counts/day^b	<4	171,951 (90,713, 13,086)	177,118 (107,843, 16,024	174,535 (95,036, 13,898)	0.901	0.822–0.945	28,542	15	79,115	45
Accelerometer counts/day^b	4–6.5	109,270 (48,867, 9316)	124,568 (97,344, 14,648)	116,919 (57,365, 10,976)	0.728	0.413–0.874	25,486	21	70,643	60
Accelerometer step/day^b	<4	5195 (2430, 398)	5289 (3106, 507)	5242 (2674, 441)	0.912	0.842–0.951	721	15	1998	38
Accelerometer step/day^b	4–6.5	3111 (1626, 394)	3329 (2115, 437)	3220 (1725,395)	0.805	0.578–0.910	718	18	1990	62

4 missing cases, ^b7 missing cases.

Reliability was strong (i.e. ICC≥0.8) for all measures in those who did not use walking aids (ICC range=0.878–0.989) (Table 5). This was not the case for accelerometer metrics among those who used a walking aid, whereby the ICC estimates were less than 0.80 for those who used walking aids (results were still within acceptable ranges of reliability).³² The ICCs for the other measures were strong in the subgroup that walked with assistance (ICC≥0.8). Precision and MDC estimates mimicked the pattern seen in the SR-EDSS group comparisons.

Table 5.

Showing mean score, ICC score, SEM, CV and MDC results for mobility outcomes in 82 persons with multiple sclerosis (MS) based on walking assistive device.

Test	Walking assistive device	Baseline mean (SD, SE)	Six month mean (SD, SE)	Overall mean (SD, SE)	ICC	95% CI ICC	SEM	CV	MDC₉₅	% MDC₉₅
Timed 25-Foot Walk time (s)^a	No aid	5.0 (1.25, 0.2)	4.8 (1.1, 0.1)	4.9 (1.1, 0.1)	0.946	0.835–0.938	0.3	5	0.8	16
Timed 25-Foot Walk time (s)^a	Walking aid	16.3 (19.2, 4.5)	16.2 (21, 5)	16.3 (20, 5)	0.989	0.944–0.992	2	10	5.6	34
Timed 25-Foot Walk speed (ft/s)^a	No aid	0.2 (0.05, 0.01)	0.2 (0.04, 0.01)	0.2 (0.05, 0.01)	0.946	0.835–0.938	0.01	5	0.03	16
Timed 25-Foot Walk speed (ft/s)^a	Walking aid	0.65 (0.77, 0.18)	0.65 (0.84, 0.2)	0.65 (0.8, 0.2)	0.989	0.944–0.992	0.08	10	0.2	34
Six-Minute Walk test distance (m)^a	No aid	498 (88.7, 37.9)	517 (85.1, 36.4)	508 (86, 36.6 )	0.943	0.906–0.966	21.2	4	58.7	12
Six-Minute Walk test distance (m)^a	Walking aid	210 (114, 85.6)	237.6 (152, 114.3)	224 (128, 96.7)	0.834	0.834–0.620	46.4	13	129	57
Six-Minute Walk test speed (m/min)^a	No aid	83 (14.8, 1.9)	86.2 (14.2, 1.8)	84.6 (14.3, 1.9)	0.943	0.906–0.966	3.5	4	9.8	12
Six-Minute Walk test speed (m/min)^a	Walking aid	35 (18.9, 4.35)	39.6 (25.3, 5.8)	37.3 (21.4, 4.9)	0.834	0.834–0.620	7.7	13	21.3	57
Multiple Sclerosis Walking Scale	No aid	29 (22.6, 2.9)	30 (25.9, 3.3)	29 (24, 3)	0.878	0.805 – 0.924	8	34	22	73
Multiple Sclerosis Walking Scale	Walking aid	76 (18, 4.1)	75 (17, 3.7)	76 (17, 3.7)	0.837	0.634 – 0.932	7	8	20	27
Accelerometer counts/day^b	No aid	168,423 (83,846, 10,957)	177,648.4 (99,635, 13,483)	173,036 (86,951, 11,569)	0.879	0.794–0.928	29,166	17	80,844	47
Accelerometer counts/day^b	Walking aid	85,620 (37,932, 8436)	93,695 (73,810, 16,959)	89,657 (50,916, 11,626)	0.672	0.123–0.877	21,724	18	60,216	67
Accelerometer steps/day^b	No aid	5103 (2254, 269)	5224 (2871, 380)	5163 (2452, 311)	0.891	0.815–0.936	744	17	2063	40
Accelerometer steps/day^b	Walking aid	2245 (1162, 773)	2445 (1950, 812)	2345 (1450, 785)	0.774	0.395–0.915	552	14	1531	65

4 missing cases, ^b7 missing cases.

Discussion

Walking impairment is an important consequence of MS that impacts function and quality of life³⁶ and has become a primary end-point in clinical research and practice. There is evidence for the validity of the T25FW, 6MW, MSWS-12 and accelerometry scores as measures of walking impairment in MS,³⁷ but less evidence exists for other psychometric properties. This study assessed the test-retest reliability, precision and minimal detectable change for the T25FW, 6MW, MSWS-12 and accelerometry over a six-month period based on group averages from 82 seemingly stable persons with MS.

Reliability

Overall, the outcome measures were highly reliable across six months in the entire sample, with ICC estimates exceeding 0.80. This complements and extends previous studies^19,20 which indicate that the T25FW and 6MW were reliable over a one-week period in homogeneous samples of 19 people who had an EDSS score of 6.5 or less²⁰ and 24 people who had an EDSS score of 5 to 6.5.¹⁹ We extend that evidence by demonstrating strong reliability of T25FW, 6MW, MSWS-12 and accelerometer metrics (i.e. multiple outcome measures) in a larger, heterogeneous sample across a longer time period that is consistent with observations performed in clinical trials and practice. This indicates that these outcomes are generally reliable for monitoring walking status over time among persons with MS.

The reliability of measures varied between groups who differed in disability and ambulatory device usage, a finding similar to the results of Nilsagard et al.²² The 6MW and T25FW were highly reliable across six months in both subsample groupings (i.e. disability and device usage) with all ICC estimates exceeding 0.80. By comparison, the accelerometer metrics and MSWS-12 scores had reliability estimates less than 0.80 in those with moderate disability and the accelerometer metrics further had reliability estimates less than 0.80 in those who used an aid for ambulation. Importantly, the reliability estimates, even when below 0.80, still satisfied criteria for acceptable reliability in the subsamples.³² Previous research has acknowledged that different walking scales perform differently across the MS disability range.²² For example the MSWS-12 is thought to provide an adequate assessment of walking performance across the disability range, whilst the T25FW may be less sensitive to change in those with milder disability,¹¹ The distribution of scores in this study imply that this may be the case. Further differences attributable to the difference in reliability between the scales may be that: the T25FW and 6MW were carried out under laboratory conditions, following strict protocols that regulate behavioral variation; accelerometer data were collected during walking undertaken in one’s daily life under free-living conditions (i.e. daily walking in one’s own environment); the MSWS-12 similarly reflects perceived real-world ambulatory impairment. This may impact the reliability of behavior captured by the accelerometer metrics. Nevertheless, collecting data in the natural environment is of high importance as it provides an indication of walking under real world conditions (i.e. ecological validity). Accelerometry should be included in MS research and practice and, considering the results of this study, it should be complemented with other measures of walking.

Precision and clinical important change

The precision varied amongst the outcome measures in the overall sample. The T25FW and 6MW were the most precise based on SEM and CV estimates followed by accelerometer metrics. The MSWS-12 was less precise based on SEM and CV estimates. The clinically important change estimates varied amongst the outcomes such that the 6MW and T25FW required the least change: larger changes were required in MSWS-12 and accelerometer metrics to indicate a clinically important change. This difference again makes sense, as discussed with ICC estimates, considering the conditions and context under which data were collected for the T25FW and 6MW compared with accelerometry and the MSWS-12.

Interesting results emerged when we considered precision and clinical important change for the outcomes between groups who differed in disability and ambulatory device usage. The precision and MDC for the 6MW, T25FW and accelerometer metrics were better among those who were less disabled and those who did not use walking aids. The exact opposite pattern was observed for the MSWS-12 where the precision and MDC were better among persons with moderate disability and who used walking aids. This is probably associated with the greater variability compared with mean MSWS-12 scores for those with less disability, for example, than those with moderate disability: the exact opposite is observed for the T25FW, 6MW and accelerometer metrics.

Implications for research and practice

The precision and clinically important change scores established in this study may guide clinicians and researchers in interpreting if change in participant performance is meaningful (MDC scores) or associated with noise in the measurement (SEM and CV scores). Importantly, previous research has indicated that a change between 20%^6,38 (based on the variability of mean scores over consecutive walks) and 70%¹⁹ (based on MDC calculations) in T25FW performance represents a clinically significant change. The present study established a 36% change in T25FW performance to be clinically meaningful in the overall sample, but this value was substantially lower in those with mild disability and those who walked without an aid (12% and 15% respectively) compared to the respective comparative subgroup. This highlights variability in estimates for interpreting the meaningfulness of a change in performance on walking outcomes and this might be associated with the characteristics of the sample and the conditions and design of data collection. Researchers and clinicians should be aware that the characteristics of the sample are important when selecting criteria for interpreting meaningful change in walking outcomes.

There were differences amongst the outcome measures in reliability, precision and clinically important change by disability level and use of walking aids. Interestingly, the T25FW, 6MW and accelerometry were less reliable in those who were more disabled (SR-EDSS≥4) and furthermore had less precision and a larger clinically important change score in the subgroups. The MSWS-12 appeared to be less reliable in the more disabled group, but had better precision and a smaller clinically important change score in the more impaired subgroups. There are many possible reasons for the differential results with the MSWS-12, but one implication is that this scale might not be an ideal choice for measuring walking impairment in those with mild disability or those who walk without a device. Yet, our results indicate that other walking outcome measures are more precise and capable of detecting clinical change in those who have less disability or mobility problems.

The study might have implications for sample size estimates in future clinical trials. To that end, we provide sample size estimates using reliability parameters identified herein for detecting a small interaction in a typical randomised controlled trial using G*Power Version 3.1.2. We assumed a mixed model analysis of variance (ANOVA) with group as a between-subjects factor (i.e. intervention vs control) and time as a within-subject factor (i.e. pre-post assessments). The parameters for α and β were 0.05 and 0.80, respectively, the effect size was small (f-value=0.10) and non-sphericity (ϵ) was 1.0. Using the lowest reliability estimate of 0.88 for the accelerometer output in the entire sample, the power analysis indicated a total sample size of 50 persons is necessary for detecting a small interaction effect. Using the lowest reliability estimate of 0.67 for the accelerometer output in those who walk with an aid, the power analysis indicated that a total sample size of 132 persons is necessary for detecting a small interaction effect. Using the default reliability estimate of 0.50 in G*Power, the power analysis indicated that a total sample size of 200 persons is necessary for detecting a small interaction effect. This illustrates the importance of empirically defining the reliability parameter for power analyses.

Limitations

This paper provides worthwhile information to guide clinicians and researchers. The strengths include: the inclusion of multiple outcome measures, clinical and real-world measures; analyses across a wide range of disability levels in MS and between groups such as those deemed to be mildly and moderately disabled; multiple statistical methods; and a six-month timeframe, similar to clinical trials and practice. Furthermore statistical methods to report reliability were chosen based on previous research, and can be easily replicated. However it is acknowledged that other statistical methods are available (such as Rasch analysis) which may complement the findings. In addition, other limitations within this work include the following: only walking outcomes were assessed in this study; comparison with other studies is limited by the use of the SR-EDSS scale, such that disability level was self-reported, rather than administered by a clinician. Lastly, we report data based on an apparently stable sample (i.e. no significant change in SR-EDSS or PDDS score with no reported symptom or clinically diagnosed relapse, and acknowledge that in the absence of objective clinical data it cannot be confirmed if individual participants were experiencing episodes of disease.

Summary

The reliability of the T25FW, 6MW, MSWS-12 and accelerometry were acceptable in the entire sample and subsamples. There is variation in the reliability, precision and clinically important changes amongst walking measures based on level of disability and use of walking aid. Researchers and clinicians should be aware of the subtle, yet important, differences in reliability, precision and clinically important changes when administering and interpreting walking measures in MS.

Footnotes

Acknowledgements

RM acquired funding. DD, LP, BS and RM initiated the overall study and oversaw all data collection. YL and RM undertook statistical analysis. YL wrote the draft manuscript, DD, LP, BS and RM revised the draft manuscript. All authors gave approval of the final submitted version. The authors wish to thank all participants in this research and all staff and students involved in data collection.

Declaration of conflicting interests

None declared.

Funding

This research was funded by a grant from the National Multiple Sclerosis Society (PP1695). YL was the recipient of a Du Prẻ Award from the Multiple Sclerosis International Federation.

References

Goldman

Marrie

Cohen

Evaluation of the six-minute walk in multiple sclerosis subjects and healthy controls. Mult Scler 2008; 14: 383–390.

Larocca

Impact of walking impairment in multiple sclerosis: Perspectives of patients and care partners. Patient 2011; 4: 189–201.

Heesen

. Patient perception of bodily functions in multiple sclerosis: Gait and visual function are the most valuable. Mult Scler 2008; 14: 988–991.

Goldman

Motl

Rudick

Possible clinical outcome measures for clinical trials in patients with multiple sclerosis. Ther Adv Neurol Disord 2010; 3: 229–239.

Motl

Ambulation in multiple sclerosis. Phys Med Rehabil Clin 2013 (published early online).

Kaufman

Moyer

Norton

The significant change for the Timed 25-foot Walk in the multiple sclerosis functional composite. Mult Scler 2000; 6: 286–290.

Butland

Pang

Gross

. Two-, six-, and 12-minute walking tests in respiratory disease. Brit Med J 1982; 284: 1607–1608.

Hobart

Riazi

Lamping

. Measuring the impact of MS on walking ability. The 12-Item MS Walking Scale (MSWS-12). Neurology 2003; 60: 31–36.

Hale

Pal

Becker

. Measuring free-living physical activity in adults with and without neurologic dysfunction with a triaxial accelerometer. Arch Phys Med Rehabil 2008; 89: 1765–1771.

10.

Weikert

. Accelerometry is associated with walking mobility, not physical activity, in persons with multiple sclerosis. Med Eng Phys 2012; 34, 590–597.

11.

Bethoux

Bennett

Evaluating walking in patients with multiple sclerosis. Int J MS Care 2011; 13: 4–14.

12.

Goldman

Marrie

Cohen

Evaluation of the six-minute walk in multiple sclerosis subjects and healthy controls. Mult Scler 2008; 14: 383–390.

13.

Heffernan

Ranadive

Weikert

. Pulse pressure is associated with walking impairment in multiple sclerosis. J Neurol Sci 2011; 309: 105–109.

14.

Motl

Weikert

Suh

. Accuracy of the actibelt® accelerometer for measuring walking speed in a controlled environment among persons with multiple sclerosis. Gait Posture 2012; 35: 192–196.

15.

Motl

Suh

Balantrapu

. Evidence for the different physiological significance of the 6- and 2-minute walk tests in multiple sclerosis. BMC Neurology 2012; 12: 6.

16.

Pilutti

Dlugonski

Sandroff

. Further validation of Multiple Sclerosis Walking Scale-12 scores based on spatiotemporal gait parameters. Arch Phys Med Rehab 2012. DOI: 10.1016/j.apmr.2012.08.214

17.

Sosnoff

Goldman

Motl

Real-life walking impairment in multiple sclerosis: Preliminary comparison of four methods for processing accelerometry data. Mult Scler 2010; 16: 868–877.

18.

Motl

Pilutti

Sandroff

. Accelerometry as a measure of walking behavior in multiple sclerosis. Acta Neurol Scan In press.

19.

Learmonth

Paul

McFadyen

. Reliability and clinical significance of mobility and balance assessments in multiple sclerosis. Int J Rehab Res 2012; 35: 69–74.

20.

Paltamaa

West

Sarasoja

. Reliability of physical functioning measures in ambulatory subjects with MS. Physiother Res Int 2005; 10: 93–109.

21.

Haley

Fragala-Pinkham

Interpreting change scores of tests and measures used in physical therapy. Phys Ther 2006; 86: 735–743.

22.

Nilsagard

Lundholm

Gunnarsson

. Clinical relevance using timed walk tests and ‘timed up and go’ testing in persons with multiple sclerosis. Physiother Res Int 2007; 12: 105–114.

23.

Goodman

Brown

Krupp

. Sustained-release oral fampridine in multiple sclerosis: A randomised, double-blind, controlled trial. Lancet 2009; 373: 732–738.

24.

Oken

Kishiyama

Zajdel

. Randomized controlled trial of yoga and exercise in multiple sclerosis. Neurology 2004; 62: 2058–2064.

25.

Ponichtera-Mulcare

Mathews

Barrett

. Change in aerobic fitness of patients with multiple sclerosis during a 6-month training program. Sports Med Train Rehabil 1997; 7: 265–272.

26.

Rodgers

Mulcare

King

. Gait characteristics of individuals with multiple sclerosis before and after a 6-month aerobic training program. J Rehabil Res Dev 1999; 36: 183–188.

27.

Dodd

Taylor

Shields

. Progressive resistance training did not improve walking but can improve muscle performance, quality of life and fatigue in adults with multiple sclerosis: A randomized controlled trial. Mult Scler 2011; 17: 1362–1374.

28.

Streiner

Norman

Health measurement scales: A practical guide to their development and use. London: Oxford University Press, 2008.

29.

Ratzker

Feldman

Scheinberg

. Self-assessment of neurologic impairment in multiple sclerosis. Neurorehabil and Neural Repair 1997; 11: 207–211.

30.

Cutter

Baier

Scheinberg

. Development of a multiple sclerosis functional composite as a clinical trial outcome measure. Brain 1999; 122: 871–882.

31.

Pilutti

Dlugonski

Pula

. Weight status in persons with multiple sclerosis: Implications for mobility outcomes. J Obes 2012; 1, 3, 139–144

32.

Denegar

Ball

Assessing reliability and precision of measurement: An introduction to intraclass correlation and standard error of measurement. J Sport Rehabil 1993; 2: 35–42.

33.

Atkinson

Nevill

Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Med 1998; 26: 217–238.

34.

Altman

Bland

Brackets (parentheses) in formulas. Brit Med J 2011; 343–344.

35.

Kovacs

Abraira

Royuela

. Minimum detectable and minimal clinically important changes for pain in patients with nonspecific neck pain. BMC Musculoskelet Disord 2008; 9: 43.

36.

Heesen

Bőhm

Reich

. Patient perception of bodily functions in multiple sclerosis: Gait and visual function are the most valuable. Mult Scler 2008; 14: 988–991.

37.

Kieseier

Pozzilli

Assessing walking disability in multiple sclerosis. Mult Scler 2012; 18: 914–924.

38.

Schwid

Goodman

McDermott

. Quantitative functional measures in MS: What is a reliable change? Neurology 2002; 58: 1294–1296.