Minimal clinically important difference for the Boston Carpal Tunnel Questionnaire: new insights and review of literature

Introduction

Many studies have been done to determine the outcome after surgical decompression in carpal tunnel syndrome (CTS) (Burke et al., 2006; Conzen et al., 2016; Katz et al., 2001; Louie et al., 2012). In order to obtain reliable results in these studies, validated outcome measures are essential. Patient-reported outcome measures (PROM) derived from symptom scales, function scales, satisfaction and quality of life, are being increasingly used in studies to evaluate success of treatment (Kim and Jeon, 2013). In CTS, the Boston Carpal Tunnel Questionnaire (BCTQ) is the most widely used self-administered outcome scale. There is strong evidence for the validity, reliability and responsiveness in patients with CTS for assessing patients’ perceived symptom severity (Symptom Severity Scale, SSS) and functional status (Functional Status Scale, FSS) (Atroshi et al., 1998; Gay et al., 2003; Greenslade et al., 2004; Leite et al., 2006; Levine et al., 1993). To interpret changes in scores during follow-up, it is essential to know how relevant these changes are for patients. The minimal clinically important difference (MCID) is an outcome instrument’s smallest difference in score that is perceived as important or meaningful by patients (Kim and Jeon, 2013). The MCID is a patient-centred concept, capturing the magnitude of improvement as well as the value patients place on the change (McGlothlin and Lewis, 2014). Only a few studies have been published for MCID of the BCTQ with varying fixed cut-off values in a wide range and differences in applied methods (Amirfeyz et al., 2009; Atroshi et al., 1998; Bessette et al., 1998; Jerosch-Herold et al., 2011; Kim and Jeon, 2013; Ozer et al., 2013; Ozyurekoglu et al., 2006). Therefore, uncertainty may arise among researchers about which MCID should be used.

We hypothesized that MCID of the BCTQ depends on the severity of symptoms that patients have before treatment, and that a relative MCID taking preoperative severity of symptoms into account would be more accurate than a fixed cut-off for all patients. The aim of this study is to calculate the MCID for the BCTQ in a valid way, which can then be widely used in future trials and in clinical practice to measure patients’ perceived outcome in CTS treatment. In addition, we give an overview of previously reported studies into MCID, emphasizing the variability in the field of MCID.

Methods

Patients with symptoms suggestive of CTS were referred to our outpatient clinic by their general practitioner. Patients were included if they fulfilled the clinical criteria for CTS. Inclusion and exclusion criteria were similar to those reported by Claes et al. (2014) (See supplementary online Appendix S1). Electrodiagnostic tests and ultrasound were performed, but patients were included irrespective of the result. They all underwent open carpal tunnel release under local anaesthesia. Before carpal tunnel release and at about 8 months follow-up, patients completed the BCTQ (for both SSS and FSS score 1 to 5). They also rated their perception of the effect of treatment on a six-point scale, with 1 representing ‘I am completely asymptomatic’, 2 ‘I very rarely still have complaints’, 3 ‘I occasionally still have complaints’, 4 ‘I often still have complaints’, 5 ‘My complaints are the same as before treatment’ and 6 representing ‘My complaints have increased’.

For statistical analyses, we dichotomized the six-point scale to distinguish patients who did or did not notice clinically relevant improvement. Clinically relevant improvement was defined as ‘I am completely asymptomatic’ (score 1), which was used as an external reference. After this, we calculated the absolute changes in SSS and FSS between the baseline scores and follow-up (absolute Δ SSS and absolute Δ FSS, respectively), by subtracting the SSS and FSS score at follow-up from the SSS and FSS score at baseline (absolute Δ SSS = SSS score at baseline – SSS score at follow-up). We also calculated relative changes by dividing the absolute Δ SSS by the SSS score at baseline (relative Δ SSS = absolute Δ SSS / SSS score at baseline) and in an identical way for FSS score (relative Δ FSS). Receiver operator characteristics (ROC) curves were then constructed in order to distinguish whether it is absolute or relative changes that correspond best to a clinically relevant improvement as experienced by patients. Then, we created a scatterplot to identify the optimal cut-off point for MCID. Differences in SSS and FSS scores between groups were compared using an unpaired t-test.

In addition, a review of literature was performed in PubMed and Google Scholar, to identify previously reported studies about MCID of the BCTQ. The reference list of every study included was checked to find additional studies of interest.

Results

Two hundred and twenty-nine consecutive patients were included in our study and underwent carpal tunnel release. At follow-up, 192 patients completed the SSS, 188 the FSS and 195 the six-point scale for perceived improvement. A total of 180 patients (143 women, 37 men) with an average age of 52 years (SD 13) accomplished all questionnaires, and data for these patients were used in the analysis.

Absolute change in scores

The overall mean SSS score at baseline was 2.95 (SD 0.68) and mean FSS score 2.27 (SD 0.71). Mean absolute change after carpal tunnel release was 1.33 (SD 0.89) in SSS and 0.67 (SD 0.95) in FSS. A total of 102 (57%) patients reported that they were completely asymptomatic after carpal tunnel release. As demonstrated in Figure 1, mean absolute change in SSS (1.72, SD 0.77) and FSS (1.00, SD 0.80) was higher in completely asymptomatic patients than in patients still experiencing complaints (0.83, SD 0.76 and 0.18, SD 0.71 respectively; p < 0.001).

OPEN IN VIEWER

Figure 1.

Absolute change in SSS and FSS scores in patients after surgery. The boxplots represent the median (central horizontal lines), mean (asterisks), 1st and 3rd quartile (edges of the box), 1.5 times inter-quartile range (whiskers) and outliers (circles). (a) Mean absolute change in SSS score is higher for patients who are completely asymptomatic (1.72, SD 0.77) compared with patients who are not completely asymptomatic (0.83, SD 0.76; p < 0.001). (b) Mean absolute change in FSS score in patients who are completely asymptomatic (1.00, SD 0.80) is also higher than patients who are not completely asymptomatic (0.18, SD 0.71; p < 0.001).

Relative compared with absolute change in scores

Figure 2 provides the constructed ROC curves. The ROC curves show how accurately different changes in SSS (left) and FSS (right) distinguish those who are completely asymptomatic from those who are not. The area under the curve (AUC) for relative changes was higher than the absolute changes in SSS as well as in FSS, although confidence intervals (CIs) overlap. The AUC was 0.81 (95% CI 0.75–0.88) for the absolute change in SSS and 0.88 (95% CI 0.83–0.93) for the relative change in SSS. For FSS, AUC was 0.79 (95% CI 0.73–0.86) for absolute change and 0.84 (95% CI 0.78–0.89) for relative change.

OPEN IN VIEWER

Figure 2.

ROC curves of the SSS and FSS, applying ‘completely asymptomatic’ as anchor. (a) The area under the curve (AUC) of relative changes of SSS score (0.88, dashed line) is higher than the AUC of absolute changes (0.81, dotted line). (b) The same concept is seen in FSS, in which the AUC for relative changes in score (0.84, dashed line) is greater than the AUC for absolute changes (0.79, dotted line).

MCID for relative change in scores

To determine the optimal cut-off point for relative change in SSS and FSS scores (i.e. MCID), we converted the ROC curves into a scatterplot. As can be seen in Figure 3, the MCID is 0.46 for SSS and 0.28 for FSS. This means that an improvement in SSS score of 0.46 relative to the preoperative SSS (absolute Δ SSS / SSS preoperatively = 0.46) is consistent with a clinically important change and thus a successful surgical result. For example, in case of an SSS score of 4 preoperatively, absolute Δ SSS has to be 4 x 0.46 = 1.84 or more to notice a clinically relevant difference. Therefore, MCID for a SSS of 4 preoperatively is 1.84. In case of a preoperative SSS score of 3, MCID is 1.38. To calculate the MCID for each SSS and FSS score prior to CTR, we constructed a regression equation. MCID for each individual can be calculated by the following regression equations: MCID SSS = 0.46x (in which x is baseline SSS preoperatively) and MCID FSS = 0.28x (in which x is baseline FSS preoperatively). In Figure 4, the regression equations for MCID are shown in plots in which the absolute change in SSS and FSS scores are displayed for patients who reported to be either completely asymptomatic or not completely asymptomatic.

OPEN IN VIEWER

Figure 3.

Scatterplots of sensitivity and specificity for different relative cut-off points for relative SSS and FSS. (a) The vertical line illustrates the optimal cut-off point for SSS (0.46) with the highest combination of sensitivity and specificity (83% and 82%, respectively). (b) The optimal cut-off point for FSS (0.28; sensitivity 77%, specificity 72%).

OPEN IN VIEWER

Figure 4.

Regression lines indicating MCID for SSS and FSS depending on baseline SSS and FSS scores. (a) Absolute change in SSS score in patients who reported to be completely asymptomatic (black circles) and not completely asymptomatic (squares). It is clearly visible how they are related to the regression line for MCID (MCID = 0.46x). (b) Regression line for MCID of FSS (MCID = 0.28x) in relation to absolute change in patients who are completely asymptomatic and not completely asymptomatic.

Our review of literature identified only seven previously reported studies about MCID of the BCTQ (Table S1). Only fixed cut-off values are used, which vary in a wide range of 0.16–1.45 for SSS and 0.47–1.6 for FSS. There is a great variation in the methods used.

Discussion

Our study confirms our hypothesis that an MCID derived from baseline SSS or FSS scores is more accurate than a fixed cut-off value for MCID. In other words, CTS patients suffering from more symptoms and more limitations in hand function need greater improvement in SSS and FSS after surgery to perceive clinically important improvement in symptoms and to consider carpal tunnel release as successful.

Figure 4 illustrates that patients with more severe symptoms preoperatively (higher baseline SSS or FSS scores) need more improvement to consider themselves as asymptomatic. Moreover, Figure 1 shows that there is overlap in absolute improvement of SSS and FSS scores. Therefore, one can assume that a fixed cut-off value for absolute change will not differentiate accurately between clinically relevant improvement or not. This was supported by the ROC curves shown in Figure 2. In general, an AUC greater than 0.8 is considered to be a good measure of accuracy. The AUC of the relative change in both SSS and FSS scores was higher than the absolute changes, and accuracy of those relative values proved to be good (i.e. > 0.8). Sensitivity and specificity for the relative cut-off value for SSS are 83% and 82%, and for FSS 77% and 72%, respectively. Thus, to obtain a more accurate estimation for MCID, a relative MCID based on baseline SSS and FSS scores should be applied.

By application of the derived formulas, one can calculate the MCID for SSS and FSS for each individual. In CTS, PROMs can be very helpful to determine treatment effect, because clinical examination as well as neurophysiology tests are not by themselves sensitive enough to adequately evaluate outcome, and standardized questionnaires like BCTQ are more sensitive to the clinical change produced by carpal tunnel surgery (Amadio et al., 1996; Greenslade et al., 2004; Merolli et al., 2013; Schrijver et al., 2005). Unfortunately, clinical application of the BCTQ is limited by variable results about MCID in earlier studies. As shown in Table S1, previously reported cut-off values for MCID vary in a wide range of 0.16–1.45 for SSS and 0.47–1.6 for FSS. The highest values were found by Ozer et al. (2013). Remarkably, they only included patients with moderate or severe CTS according to electrodiagnostic criteria, and mean baseline SSS and FSS scores were as high as 3.5 (Ozer et al., 2013). According to our results, therefore, it is not surprising that the values for MCID are that high. On the other hand, much lower values for MCID were found by Atroshi et al. (1998), with baseline scores of SSS (3.1) and FSS (2.5). These data support our findings that MCID depends on the severity of symptoms before surgery.

Another possible reason for the varying values for MCID in literature is the method for determining MCID that was used. In general, three methods can be used to calculate MCID. Consensus methods are based on independent assessments of an expert panel, eventually reaching consensus of what numerical value constitutes a clinically relevant change. The main criticism of this method is that MCID is based on expert opinion rather than patients’ opinion. Distribution-based methods rely on statistical properties of the distribution of outcome scores and determine what magnitude of change is required to demonstrate that the change in an outcome measure in response to an intervention is more than would be expected from chance alone (McGlothlin and Lewis, 2014). The major disadvantage of this approach is that it is not derived from what is important from patients’ point of view, and, therefore, distribution-based methods are not recommended for determining MCID (McGlothlin and Lewis, 2014; Turner et al., 2010). The anchor-based method, applied in our study, is considered as the most appropriate approach. The relation is examined between the change in a numerical scale (e.g. BCTQ) and an independent external judgement of meaningful change for patients (anchor or standard) (Johnston et al., 2015). Results of anchor-based methods are, however, influenced by the choice of anchor. As this is a subjective assessment, the main concern of this method is that it might be susceptible to recall bias (McGlothlin and Lewis, 2014).

In our study, ‘I am completely asymptomatic’ was used as the anchor for clinical important change, since we suppose that these patients will definitely notice clinically relevant changes in symptoms. Including patients who reported that they still very rarely have complaints, gave very similar results (Table S2 and supplementary Figures S1 and S2). However, Amirfeyz et al. (2009) applied a less strict anchor (‘complaints are moderately better, slight but noticeable change’), which resulted in extremely low values for MCID. In our opinion, the use of such mild anchors would compromise clinical applicability of the BCTQ, as almost every patient will achieve an improvement of 0.16 in SSS score (Figure 4).

Differences in MCID in previously reported studies can also be explained by duration of follow-up. The length of interval of completing the BCTQ after intervention varies from 2 weeks up to 6 months. A period of 2 weeks may be too short to notice maximal treatment effect, and therefore clinically relevant changes can be missed. We used an interval of about 8 months after surgery, as previous research has found clinical improvement to peak at 6 months postoperatively and no further improvement after that time (Guyette and Wilgis, 2004).

Our study has some limitations. First, we used a self-developed six-point scale as an anchor. This scale has not been validated. However, we only used one item, namely ‘I am completely asymptomatic’, as external reference. In our opinion, this answer is unequivocal and reflected actual evidence of satisfactory improvement for the patient, and it therefore will not lead to misinterpretation and negative influence on reliability of this study. Patients were included if CTS was diagnosed clinically. Hence, it is likely that also patients without abnormalities on nerve conduction studies were included. Another limitation could be the lack of a control group. This is complicated for this study, as sham surgery raises real ethical issues. However, without a sham surgery control group, it is unclear to what extent these results might represent a placebo effect, which would be more likely to wear off over longer follow-up periods.

We conclude that the MCID of the BCTQ should be calculated from baseline SSS and FSS scores, as patients experiencing more complaints and functional restrictions require more improvement in order to report a clinically important difference. By taking this into account, the BCTQ is more meaningful as an outcome measure in research and clinical practice.

Supplemental Material

Supplemental Material

Supplemental Material