Abstract
Minimal clinically important difference is the smallest difference in the score of an outcome instrument that patients perceive as important. A prospective observational study was conducted involving 66 consecutive patients undergoing limited open carpal tunnel release. Patients completed the Carpal Tunnel Questionnaire (CTQ) before surgery and 3 months post-surgery. A transition item for patient satisfaction after 3 months of surgery was used as an anchor. Receiver operator characteristic curves were used to determine the minimal clinically important differences for the CTQ and its two subscales that best separated satisfied and unsatisfied patients. A 0.92 point change in the CTQ, a 1.14 point change in the symptom subscale, and a 0.74 point change in the function subscale indicated a clinically relevant threshold of satisfaction, and those values were greater than what could be accounted for by measurement errors.
Introduction
Standardized instruments in the form of self- administered questionnaires that measure outcomes in terms of patient concerns, such as symptoms, functions, treatment satisfaction, and quality of life are being increasingly used in clinical practice (Katz et al., 1995). Such questionnaires can be used to evaluate the outcomes of treatment from the patient’s perspective and can facilitate comparisons between studies (Stucki et al., 1996). Validated generic (general health) instruments, such as the Medical Outcomes Study 36-item Short-Form Health Survey (SF-36), have been widely used for studying different populations and patient groups (McHorney et al., 1993; Ware and Sherbourne, 1992), and disease-specific questionnaires have been developed to measure important clinical features of specific diseases that might be undetectable using generic instruments. The Carpal Tunnel Questionnaire (CTQ) (Levine et al., 1993) has become the most widely used disease-specific self-administered questionnaire for the assessment of symptom severity and functional status in patients with carpal tunnel syndrome (CTS). The CTQ has been reported to be valid, reliable, and responsive in terms of the assessments of symptom severity and functional status in patients with CTS (Amadio et al., 1996; Atroshi et al., 1998; Gay et al., 2003; Levine et al., 1993).
The responsiveness of an outcome questionnaire is its ability to accurately measure change when it has occurred (Gay et al., 2003). One way of assessing this characteristic is to use statistical methods that include measures such as effect size and standardized response mean, and several studies have addressed the responsiveness of the CTQ in this manner (Amadio et al., 1996; Atroshi et al., 1998; Gay et al., 2003). These measures are useful when calculating sample sizes for studies and comparing instruments to determine which is more sensitive in terms of detection of clinical change.
However, to interpret the effect of a treatment, it is also important to assess how relevant the effects are for patients (Bago et al., 2009; Guyatt et al., 2002). For this reason, it is preferable to evaluate an instrument’s responsiveness by determining the relationship between score changes and patient self-reported clinical changes (Revicki et al., 2008). This approach leads to the determination of minimal clinically important difference; that is, the smallest difference in score of an outcome instrument that patients perceive as important.
The purpose of this study was to determine the minimal clinically important differences for the CTQ and its two subscales after carpal tunnel release (CTR), and to confirm that these values were greater than what could be accounted for by measurement errors.
Methods
Subjects
This study was approved by our Institutional Review Board and all subjects provided written informed consent before participating in the study. Patients that requested elective carpal tunnel release for idiopathic, electrodiagnostically confirmed CTS were included. Exclusion criteria were an inability to complete the questionnaire because of cognitive impairment and a history of CTR in the same extremity.
Between March 2008 and March 2010, 76 consecutive patients, who all provided written informed consent, underwent open CTR. None of the eligible patients declined enrolment. One surgeon (KJK) performed all surgical procedures. Local anaesthesia and a pneumatic tourniquet were used in all cases. Surgery was performed using a previously described limited open technique (Bromley, 1994; Cellocco et al., 2005; Serra et al., 1997).
However, 10 of the 76 patients dropped out before their scheduled 3 month follow-up visit. Accordingly, 66 patients who completed 3 months of follow-up were enrolled in this study. Mean patient age was 54 (range 32–78, SD 10.4) years and 9 (14%) were male.
Measurements
Patients completed a preoperative CTQ within a month before surgery in outpatient clinic. The CTQ (Levine et al., 1993) consists of two scales used to evaluate symptoms and function. The first scale consists of 11 questions that address the severity and frequency of pain, numbness, weakness, and loss of dexterity. Each of these multiple choice questions offers five possible responses scored from 1 (no symptom) to 5 (severe symptom). Results are expressed as average scores for the 11 responses. The second CTQ scale is composed of eight questions that address difficulties performing daily tasks. Responses to these eight questions are also scored using a 5-point scale (1 to 5, where 5 indicates greatest difficulty), and again, results are averaged.
Patients completed a postoperative CTQ while attending an outpatient clinic 3 months after surgery. An additional transition item for patient satisfaction following surgery was also included as an anchor. The question was: Overall, how pleased are you with the results of the surgery? The response options supplied were very pleased, fairly pleased, not so pleased, and disappointed.
Non-parametric correlations (Spearman’s rho) were used to examine the associations between change in CTQ scores and its two subscales, and the transition item. A correlation coefficient of 0.4–0.59 indicated a moderate correlation and 0.6–0.79 a strong correlation (Dawson et al., 2007).
Receiver operator characteristic (ROC) curves were used to determine minimal clinically important differences in the CTQ and its two subscales that best separated satisfied and unsatisfied patients (very pleased or fairly pleased vs. not so pleased and disappointed). ROC curves plot sensitivity (Y-axis) against one specificity (X-axis) for all possible cut-off points of the instrument. Sensitivity is defined as the number of patients who were satisfied (very pleased or fairly pleased) divided by the number of all patients with a score change above the cut-off point. Specificity refers to the number of dissatisfied patients (not so pleased or disappointed) divided by the number of all patients with a score change below this cut-off point. The most efficient cut-off value, with regard to specificity and sensitivity, is associated with the point closest to the top left-hand corner of the ROC curve. The greater the area under the ROC curve, the greater the ability of the scale to differentiate between those with and without a clinically important change. If the area under curve (AUC) is 0.5, the test is not predictive; whereas, an area close to 1.0 indicates better differentiation (Stratford et al., 1996).
Minimal detectable change is defined as a change between two time points expected by chance alone or measurement error (Stratford et al., 1996). For a conventional confidence level of 90%, the minimal detectable change is calculated as 1.65 × √2 × standard error of measurement (SEM). SEM is the error estimate for single use of the questionnaire and is directly related to the reliability of the scale. It is calculated using the formula SEM = SD × √1 – α, where SD is the standard deviation of the pretreatment score and α is the reliability coefficient of the questionnaire. In this study, Cronbach’s alpha was used as the reliability coefficient and was calculated using the unweighted item scores of the CTQ and its two subscales.
Results
Outcomes measures
The mean score for the CTQ and its two subscales before surgery and at 3 month follow-up are shown in Table 1. A statistically significant improvement was seen in CTQ scores and its two subscales. At follow-up interviews, 30 (45%) patients were very pleased with surgery, 23 (35%) were fairly pleased, 10 (15%) were not so pleased, and 3 (5%) were disappointed. A significant correlation was found between the transition item and CTQ score changes (γ = 0.58, p < 0.001), symptom subscale (γ = 0.67, p < 0.001), and function subscale (γ = 0.43, p < 0.001).
Mean scores for the Carpal Tunnel Questionnaire (CTQ) and its two subscales according to the transition item before surgery and 3 months after surgery
Mean score changes of the CTQ and its two subscales for satisfied patients (very pleased and fairly pleased) and dissatisfied patients (not so pleased and disappointed) are shown in Table 1. Mean score changes for satisfied patients were significantly greater for the CTQ and its two subscales than those of dissatisfied patients.
Examples of ROC curves for the CTQ and its two subscales are shown in Figure 1. The minimal clinically important differences were 0.92 points for the CTQ, 1.14 points for the symptom subscale, and 0.74 points for the function subscale. AUC was highest for the entire CTQ (AUC = 0.820, 95% CI 0.704–0.936), followed by the symptom subscale (AUC = 0.803, 95% CI 0.689–0.916), and function subscale (AUC = 0.683, 95% CI 0.534–0.831). The CTQ (p < 0.001) and its two subscales (p < 0.001 for symptom subscale and p = 0.02 for function subscale) had the predictive abilities to differentiate satisfied and dissatisfied patient after CTR.
Receiver operator characteristic curves of the Carpal Tunnel Questionnaire (CTQ) and its two subscales.
Minimally detectable changes were 0.58 points for the CTQ, 0.52 points for the symptom subscale, and 0.66 points for the function subscale.
Discussion
Evidence indicates that a statistically significant score change does not necessarily mean that the change is clinically important (Hagg et al., 2003); for example, given a sufficiently large sample size, any minor score change could be statistically significant. Thus, from the clinical perspective, we need to determine whether a treatment effect is important when making treatment decisions (Hagg et al., 2003). Consequently, there is increasing awareness that responsiveness should include the ability to measure a minimal clinically important difference (Liang, 2000). However, minimal clinically important differences have rarely been reported for the CTQ.
Ozyurekoglu et al. (2006) determined the minimal clinically important difference of the symptom subscale of the CTQ after carpal tunnel injection. They concluded that a decrease of 1.04 points or more in the symptom subscale probably indicates a clinically important change in health state. In fact, this is the only study to have previously addressed minimal clinically important difference of the CTQ. However, the authors did not measure the minimal clinically important differences of the entire CTQ and the function subscale. Therefore, this study is the first to present minimal clinically important differences for scores of the CTQ and its two subscales after CTR.
The minimal clinically important difference is dependent on the study population, treatment methods, and transition item. In a previous study, the minimal clinically important difference of the symptom subscale of the CTQ after carpal tunnel injection was 1.04 points (Ozyurekoglu et al., 2006), which is smaller than that found in the present study. This finding is not surprising considering that surgical intervention should be considered for advanced cases or for patients that continue to be symptomatic despite conservative treatment (Hudson et al., 1997; Scholten et al., 2002). Therefore, slight symptom improvements would not meet expectations of surgically treated patients.
The minimal clinically important difference should ideally be greater than the minimal detectable change to ensure that this value is free from measurement error. In this study, minimal clinically important differences of the CTQ and its two subscales were all greater than minimal detectable changes, which imply that minimal clinically important differences represent a true change in the CTQ scores associated with clinical benefits.
We found that ROC curves indicate that the CTQ and its two subscales had predictive ability and that the AUC was highest for the entire CTQ. As the questionnaire contains more information than its subscales, it is not surprising that CTQ scores were found to be more sensitive to clinical changes than the subscale scores. Furthermore, this finding supports the use of the entire questionnaire.
This study is limited by the use of the transition item. Although the transition item is usually applied as an anchor, its use has been criticized (Hagg et al., 2003). Some authors questioned the reliability of a single-item scale compared with a multi-item scale (Norman et al., 1997). In this study, we used the transition item composed of four possible answers, but no consensus has been reached regarding how many categories of answer the transition item should have. An additional disadvantage of using the transition item is that patients must be able to recall their initial state and compare this with their current state to be able to assess change in health status, which could introduce bias (van der Roer et al., 2006). However, the strong correlations found between transition item and score changes of the CTQ and its two subscales support the use of the transition item in this study. In previous studies, this correlation has been generally considered necessary to confirm the validity of the transition item (Bago et al., 2009; Revicki et al., 2008).
Footnotes
Conflict of interests
None declared.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.