Abstract
Decreased manual dexterity in people with Parkinson’s disease (PD) can affect the performance of daily self-care tasks, such as dressing, grooming, and eating (Raggi et al., 2011). Employment (Sturkenboom et al., 2013) and leisure activities (Manson & Caird, 1985) can also be compromised. Although dopaminergic medication improves performance of motor tasks, including hand activities requiring fine motor skills (Stewart, Fernandez, Okun, Alberts, et al., 2009), after several years of using such medication, people’s motor skills may decline at the end of the dose before the next PD medication dose is due. The period during which medication is working effectively is called “on” time (Goetz et al., 2008), and the period when medication is less effective is called “off” time. People with PD describe the loss of dexterity that occurs during off time as one of their main difficulties (Matthews et al., 2015), and they may experience several ons and offs in a single day. For some people, off periods can also be unpredictable. Because motor fluctuations have the potential to vary the performance of functional tasks, it is advisable to assess dexterity in different phases of the medication cycle.
To evaluate treatments and monitor the progression of PD, occupational and physical therapists require dexterity measures with established reliability. Although the 9-Hole Peg Test (9HPT) and Purdue Pegboard Test (PPT) are recommended for the evaluation of people with PD (Kegelmeyer et al., 2014; Sturkenboom et al., 2011) and are commonly used for clinical assessment (Proud et al., 2013), the evidence for their reliability is limited (Proud et al., 2015). When people with mild to severe PD were tested twice in a single session, the 9HPT had good test–retest reliability and small measurement error (Earhart et al., 2011).
PD typically presents asymmetrically, with one hand more impaired than the other. In participants with advanced PD, the test–retest reliability of PPT scores for the more affected hand (MAH) and the less affected hand (LAH) was moderate to good in both on and off phases, for tests repeated over 2–3 wk (Metman et al., 2004). Measurement error in PPT subtests remains to be explored at all levels of PD severity.
Strong interrater reliability is also important for outcome measures used with people with PD, because they are likely to be assessed by different clinicians over the course of the disease. The 9HPT showed good interrater reliability when administered to people with PD by face-to-face and telerehabilitation raters (Hoffmann et al., 2008), but no studies of the interrater reliability of the PPT in people with PD have been published.
The aims of this study were to investigate (1) the test–retest reliability of 9HPT and PPT scores in the on phase and at the end of dose (EOD) and (2) the interrater reliability of prerecorded 9HPT and PPT performance of people with PD rated by occupational and physical therapists.
Method
Participants
Test–Retest Reliability.
We recruited a convenience sample from sources that included PD-specific outpatient programs and support groups. Eligibility criteria were as follows: a diagnosis of idiopathic PD, daily oral levodopa medication, a Hoehn and Yahr (HY) score <5, adequate cognition (Mini-Mental State Examination score >23; Folstein et al., 1975), and English language skills to comprehend test instructions and give informed consent. Exclusion criteria were other neurological conditions, major upper limb pathology, major visual deficits, and unpredictable off periods that could independently affect score variability.
Interrater Reliability.
Interrater reliability was investigated using audiovisual recordings of pegboard test performance of 10 participants with PD. Raters were recruited via invitations to therapists in researcher networks. No eligibility criteria were associated with management of people with PD or tool administration because we did not expect experience to affect scoring of these objective measures (Tyson, 2007).
Procedure
Study methods were designed with reference to the COnsensus-based Standards for the selection of health status Measurement INstruments checklist (COSMIN; Mokkink et al., 2010). The study was approved by the University of Melbourne Human Research Ethics Committee, and all participants gave informed consent.
Test–Retest Reliability.
Data were collected from December 2012 to November 2013. Participants’ hand dominance was determined using the Edinburgh Handedness Inventory (Oldfield, 1971). The MAH and LAH were identified on the basis of upper limb scores from the Movement Disorder Society–sponsored revision of the Unified Parkinson’s Disease Rating Scale (MDS–UPDRS; Goetz et al., 2008), completed when participants were in an on period.
Both the 9HPT (Rolyan Model A851–5; Sammons Preston, Warrenville, IL) and PPT (Lafayette Instrument, Lafayette, IN) were administered according to manufacturers’ protocols, with a practice trial before the single timed 9HPT trial for each hand and a short practice before each PPT subtest. Scores recorded for PPT subtests were the mean of three trials.
Each participant was tested on 2 days 1 wk apart while in an on period (approximately 1 hr after medication) and at EOD (30 min before the next dose). Sessions were scheduled for the same time on both days. All testing was conducted by Elizabeth L. Proud in participants’ homes or at the university, and it was standardized for each participant (chair, table, placement of apparatus). Participants’ medication regimens remained constant over the testing period.
Interrater Reliability.
Pegboard performance of 10 participants with PD was recorded during the test–retest reliability study. Fourteen performances on both the PPT and the 9HPT were randomly selected for inclusion in an audiovisual presentation to raters. Interrater reliability testing was conducted in May and June 2014 by Elizabeth L. Proud, with groups of up to four raters scoring performance under similar conditions (quality of vision, size of screen). Scored trials were preceded by tool familiarization and two practice trials. Raters scored trials independently, using uniform stopwatches for 9HPT trials.
Sample Size Calculations and Data Analysis
Sample size calculations for test–retest and interrater reliability were based on hypothesized intraclass correlation coefficients (ICCs) ≥ 0.85 with 95% confidence intervals (CIs) of ±0.1. Thirty participants were required for the test–retest reliability study. A minimum of 10 raters were needed to evaluate 14 performances of each pegboard subtest in the interrater reliability segment of the study (de Vet et al., 2011).
Statistical analyses were completed with IBM SPSS Statistics (Version 22; IBM Corporation, Armonk, NY). Participant and rater characteristics were summarized using descriptive statistics, and levodopa equivalent daily dosages were calculated (Tomlinson et al., 2010).
Bland–Altman plots of test–retest reliability data were generated in Minitab (Version 16; Minitab, State College, PA) and examined for systematic bias (Bland & Altman, 2010). To investigate test–retest and interrater reliability, we calculated ICC(2,1) for 9HPT subtests and ICC(2,3) for PPT subtests. ICCs ≥ 0.75 indicate good reliability, and ICCs < 0.75 indicate moderate to poor reliability (Portney & Watkins, 2009). Minimal detectable change (MDC) was calculated as follows: MDC = ±1.96 × √2 × SEM (standard error of measurement; Portney & Watkins, 2009). To facilitate tool comparisons, MDCs were expressed as a percentage of the group mean (MDC%). Test–retest reliability was calculated for dominant and nondominant hands and MAH and LAH.
Results
Test–Retest Reliability
Participant Characteristics.
Participants (N = 30) had mild to moderate PD (modified HY score of 2–3). Twenty-eight participants were right-handed, and 15 were more affected in their dominant hand. Participant characteristics are described in Table 1.
Participant Characteristics (N = 30)
Note. Scores on the M–EDL range from 0 to 52. Scores on the ME range from 0 to 132. HY = Hoehn and Yahr; MDS–UPDRS = Movement Disorder Society–sponsored revision of the Unified Parkinson’s Disease Rating Scale; M–EDL = Motor Experiences of Daily Living; ME = Motor Examination; SD = standard deviation.
Reliability and Agreement.
Twenty-eight participants (16 men, 12 women) completed all testing sessions. Two participants dropped out because of unscheduled commitments (one missing on and EOD sessions in Wk 2, the other missing Wk 2 EOD). Bland–Altman plots of 9HPT subtests showed no pattern of systematic error but identified two outliers who had greater differences between scores than other participants. These outliers were also considerably slower (≥45 s). Because the number of slower performers was insufficient to identify these 2 participants as genuine outliers or to establish a trend for greater score variability in slower performers, we excluded these data and limited the analysis to participants with scores <45 s. Dominant–MAH scores were removed for a third outlier who, because of shoulder pain in Wk 2, was deemed not to have remained stable over the testing period. Bland–Altman plots of PPT subtests mostly showed systematic improvements in Wk 2, most prominently for the Dominant and Assembly subtests in the on phase (4.9% and 9.7% of the group mean, respectively; see Figure 1).
We found moderate to good test–retest reliability for 9HPT subtests in the on phase and at EOD (ICCs = 0.70–0.81; Table 2). Good test–retest reliability was observed for PPT subtests at both points in the medication cycle (ICCs ≥ 0.90; Table 2). MDC for both tools was mostly less than 25% of the group mean, and the PPT Combined subtest scores showed the smallest MDC% for each medication phase. Test–retest reliability and measurement error values for MAH–LAH scores were similar to those for the dominant–nondominant analysis (Table 2).
Test–Retest Reliability of the 9-Hole Peg Test
Note. CI = confidence interval; Dom = dominant hand; EOD = end of dose; ICC = intraclass correlation coefficient; LAH = less affected hand; MAH = more affected hand; MDC = minimal detectable change; MDC% = MDC as percentage of group mean; Nondom = nondominant hand; SD = standard deviation; SEM = standard error of measurement.
For the 9-Hole Peg Test, time in seconds; for the Purdue Pegboard Test, number of pegs placed.
Interrater Reliability
Participant Characteristics.
Eleven raters (2 occupational therapists and 9 physical therapists) scored the prerecorded performance of 4 men and 6 women with PD (mean age = 66.8 yr, standard deviation [SD] = 7.1; mean years since diagnosis = 9.1, SD = 6.0; median HY score = 2). Raters had a mean of 18 yr (SD = 13) of practice, with expertise in diverse specialties, and were employed in a range of settings. In the previous year, 6 raters had treated clients with PD, and 1 had administered the PPT.
Reliability and Agreement.
Interrater reliability was perfect for all pegboard subtests (ICCs = 1.0) except the 9HPT Dominant subtest (ICC > 0.99). There was no calculable measurement error (MDC).
Discussion
We investigated the test–retest and interrater reliability of the 9HPT and PPT in people with mild to moderately severe idiopathic PD. When tests were repeated over a 1-wk interval, test–retest reliability was higher for PPT subtests than for 9HPT subtests both in the on phase and at EOD. Relative measurement error was generally lower for the PPT subtests. We found similar results for dominant–nondominant and MAH–LAH analysis. Interrater reliability for the 9HPT and PPT subtests was >.99 for scoring of prerecorded performance by occupational and physical therapists, with effectively zero measurement error. Study findings suggest that the PPT was more sensitive to changes in dexterity in people with mild to moderate PD.
To our knowledge, this study is the first to report on test–retest reliability and measurement error when the 9HPT and the PPT battery are administered to people with PD on separate occasions at two points in the levodopa medication cycle. We found lower reliability and higher measurement error than was reported for 9HPT trials completed in a single session (Earhart et al., 2011). Although differing results could be associated with the relative homogeneity of our sample, it is likely that the 9HPT scores of people with PD are more variable when measured on different days. Therefore, these new results may more closely reflect test–retest reliability and measurement error values when 9HPT subtests are applied in clinical practice on separate occasions.
The test–retest reliability of PPT MAH and LAH scores in this study was higher than previously reported for people with advanced PD (Metman et al., 2004). The likelihood of motor fluctuations is greater in more severely affected samples, which may lead to greater performance variability (Ahlskog & Muenter, 2001). Higher ICC values in this study could also be associated with different administration methods, given that reliability increases when more trials are performed (de Vet et al., 2011). The Metman et al. (2004) study recorded the higher score of two trials, whereas we took the mean of three trials.
Bland–Altman plots revealed systematic improvements on some PPT subtests, indicating a possible practice effect. Practice effects have also been reported for PPT subtests administered to healthy adults (Reddon et al., 1988) and people with schizophrenia (Lee et al., 2013). The PPT instructions, developed for employment screening, recommend a short practice before timed trials (Lafayette Instrument, 1999). When assessing people with neurological conditions, especially conditions associated with cognitive impairments such as PD, it may be advisable to first complete a full practice trial to increase familiarity with each subtest. This may be particularly relevant for the complex assembly task, which showed the strongest practice effect.
This study provides new evidence for the interrater reliability of the 9HPT and PPT performance of people with PD when scored by occupational and physical therapists, and it builds on previous evidence of the good reliability of 9HPT subtests scored by face-to-face and telerehabilitation raters (Hoffmann et al., 2008). Rating of recorded performance eliminates some sources of potential measurement error, but it limits the generalizability of the results to face-to-face administration in clinical practice. However, given the high interrater reliability of face-to-face 9HPT trials in healthy adults (Oxford Grice et al., 2003) and people with multiple sclerosis (Rosti-Otajärvi et al., 2008), this may also be the case for people with PD, given the simple scoring method.
Although there is no defined acceptable value for the ICC, a minimum of 0.90 has been advocated for evaluative research outcomes (Portney & Watkins, 2009). Only PPT subtests had test–retest reliability ICCs >0.90, suggesting that this measure may be more suitable for evaluating dexterity in PD research. There are no established criteria for appraising measurement error when tools are used in clinical practice, although MDC%s of <30% are described as acceptable (Lee et al., 2013). Because measures with higher measurement error may be less sensitive to change, selecting measures with smaller error is preferable (Tyson, 2007). We found that test–retest measurement error was generally lower for PPT subtests than for 9HPT subtests, with the PPT Combined subtest showing the smallest error in each medication phase. A previous study of PPT subtests administered to healthy adults showed higher reliability when tests were scored as the mean of three trials than as a single trial (Buddenberg & Davis, 2000), suggesting that it may be possible to reduce measurement error when 9HPT subtests are used with people with PD by recording the mean of three trials.
Although the results provide new evidence for the test–retest reliability of 9HPT and PPT scores recorded on separate occasions, reliability and measurement error in people with early and advanced PD require further investigation. Future research could also examine the test–retest reliability of a three-trial administration of the 9HPT and of scores recorded over a longer time interval, as may occur in clinical practice. It will also be important to understand interrater reliability when these measures are administered to people with PD in face-to-face settings.
Limitations
Given the relative homogeneity of the sample, our test–retest reliability results are not generalizable to groups with early unilateral PD or advanced PD or people with atypical parkinsonian syndromes such as progressive supranuclear palsy. The 9HPT results are also limited to people who complete the test in <45 s. Another limitation of the 9HPT results was the small sample size, given the smaller than hypothesized ICCs. In addition, the interrater reliability results cannot be generalized to face-to-face administration.
Implications for Occupational Therapy Practice
Manual dexterity deficits in people with PD affect daily activities, such as handling coins and doing up buttons, and are identified in occupational therapy guidelines as a key focus for the treatment of people with PD (Aragon & Kings, 2010; Sturkenboom et al., 2011). Occupational therapists use a client-centered approach to identify functional limitations and deliver occupation-centered treatments. They are well placed within the movement disorders team to evaluate the influence of pharmacological and nonpharmacological interventions on dexterity. This study informs the selection by occupational therapists of reliable outcome measures for the evaluation of dexterity in people with PD. The results of this study have the following implications for occupational therapy practice:
Occupational therapists can be confident of the reliability of repeated 9HPT and PPT measurements over time in people with mild to moderately severe PD.
The results give clinicians new information regarding potential differences in the sensitivity of these tools to PD-related dexterity.
Excellent interrater reliability results for prerecorded performance suggest that consistency may also be good for face-to-face testing by individual clinicians.
Conclusion
We investigated the test–retest reliability of the 9HPT and PPT in people with mild to moderately severe PD in the on phase of their levodopa medication cycle and at EOD. Although reliability of both tools was adequate, PPT subtests showed higher reliability and lower measurement error. Interrater reliability for both tools was very high for the scoring of prerecorded performance.