Abstract
Background:
The Numeric Pain Rating Scale (NPRS) is a popular method to assess pain. Recently, the Patient-Reported Outcomes Measurement Information System (PROMIS) has been suggested to be more accurate in measuring pain. This study aimed to compare NPRS and PROMIS Pain Interference (PI) scores in a population of foot and ankle patients to determine which method demonstrated a stronger correlation with preoperative and postoperative function, as measured by PROMIS Physical Function (PF).
Methods:
Prospective PROMIS PF and PI and NPRS data were obtained for 8 common elective foot and ankle surgical procedures. Data were collected preoperatively and postoperatively at a follow-up visit at least 6 months after surgery. Spearman correlation coefficients were calculated to determine the relationship among NPRS (0-10) and PROMIS domains (PI, PF) pre- and postoperatively. A total of 500 patients fit our inclusion criteria.
Results:
PROMIS PF demonstrated a stronger correlation to PROMIS PI in both the pre- and postoperative settings (preoperative: ρ = −0.66; postoperative: ρ = −0.69) compared with the NPRS (preoperative: ρ = −0.32; postoperative:ρ = −0.33). Similar results were found when data were grouped by Current Procedural Terminology (CPT) code.
Conclusion:
PROMIS PI was a superior tool to gauge a patient’s preoperative level of pain and functional ability. This information may assist surgeons and patients in setting postoperative functional expectations and pain management.
Level of Evidence:
Level II, prognostic.
Introduction
As our healthcare system continues to shift its focus from quantity to value—defined as dollar spent per health outcome 14 —the need to utilize patient-reported outcomes (PROs) in clinical care becomes critical to ensuring that quality and outcomes most important to patients are prioritized. With this transition comes the implementation and use of numerous PRO tools, currently utilized to improve patient care. 2 However, it becomes both prudent and necessary to evaluate each PRO tool with the goal of having a focused and comprehensive set of instruments. This would allow for a complete evaluation of a patient, while reducing redundancies and protecting patients from questionnaire fatigue.
For common elective foot and ankle procedures, physical function and pain improvement are important outcomes for patients. The ability to accurately assess a patient’s physical function and pain level preoperatively is advantageous to both the clinician and the patient in order to set appropriate expectations regarding postoperative outcomes and pain management. 4 Historically, a patient’s pain level has not been determined uniformly, 15 creating concerns for inconsistencies and misunderstanding between providers and patients. One common tool, the Numeric Pain Rating Scale (NPRS), asks the patient to quantify their pain on a scale from 0 to 10 using whole Arabic numbers. More recently, the Patient-Reported Outcomes Measurement Information System (PROMIS), a PRO tool developed with notable support from the National Institutes of Health (NIH), offers a more robust, universal method to capture PROs across a number of domains utilizing item response theory (IRT) and computerized adaptive testing (CAT). 3 Prior research involving PROMIS has demonstrated its value and power in evaluating foot and ankle patient outcomes utilizing 2 of these domains: Physical Function (PF) and Pain Interference (PI).1,4,6,7 While observer-based measurements (eg, gait speed) may be valuable physical function outcomes of interest, it is not always plausible and efficient to gather such data on each patient. We feel PROMIS is a good proxy for such observer-based measurements given its documented robustness in foot and ankle care1,4,6,7 and easily implementation in the standard flow of current clinic models. 13
With the introduction of new PRO tools, it becomes necessary to compare their validity and utility with commonly employed legacy instruments (eg, NPRS). The NPRS assesses pain intensity in contrast to PROMIS PI, which assesses pain interference with daily tasks. 15 However, both pain intensity and interference are thought to influence physical abilities. 10 A recent study argued that pain scales with stronger correlations to the PROMIS PF scale were more useful clinically because the instruments were capturing the effect of pain on patient activity. 10 Further, floor and ceiling effects influence the clinical utility of PRO scales. 7 Low floor or ceiling effects suggest that a scale is able to detect high and low levels of pain in a wide variety of patients. Thus, the aim of the current study was to assess the concurrent validity between PROMIS PI and NPRS with PROMIS PF pre- and postoperatively in patients undergoing common foot and ankle procedures. A second purpose was to assess floor and ceiling effects across these scales to determine whether PROMIS PI was able to detect higher and lower pain levels better than NPRS.
Methods
Consecutive patients presenting to a single, large academic medical center foot and ankle clinic from February 2015 to November 2017 (33 months) were prospectively asked to complete PROMIS PF and PI domains on an Apple iPad as part of the normal flow of patient care. 13 Prior work in orthopedic surgery has demonstrated that collecting PROs via iPads tends to be more efficient and often preferable over traditional pen-and-paper collection methods. 17 All PROMIS domains are designed to have a mean T score of 50 with a standard deviation (SD) of 10. Of note, increasing PF scores signify improvement, while the opposite (ie, decreasing scores) is true for the PI domain.
The total sample size of 500 patients varied by PRO variable and CPT code (Table 1). The sample size for each PRO varied from 411 to 483 patients, a range of 3.4% to 17% lower than the total sample size. There were more female patients (74%), and the average age was 54 years (SD, 15 years) (Table 2). A majority of patients were white (92%) and identified as non-Hispanic (97%) (Table 2).
Breakdown of CPT Codes Analyzed.
Abbreviation: CPT, Current Procedural Terminology.
Patient Characteristics (n = 500).
Abbreviations: NPRS, Numeric Pain Rating Scale; PF, Physical Function; PI, Pain Interference; PRO, patient-reported outcome; PROMIS, Patient-Reported Outcomes Measurement Information System; SD, standard deviation.
In addition to completing the PROMIS questionnaires, patients were asked at each clinic visit to rate their pain using the NPRS of 0 to 10, with 0 being “no pain” and 10 being “the worst pain ever.” The recorded NPRS scores used in our analyses represent the patient’s level of pain on the day of each clinic visit. Current Procedural Terminology (CPT) codes were then used to identify patients undergoing common elective foot and ankle procedures: repair of collateral ankle ligament (27698); open ankle arthrodesis (27870); repair of hammer toe (28285); repair of hallux rigidus (28289); calcaneus osteotomy (28300); pantalar fusion (28705); fusion foot bone, midtarsal (28730); and first metatarsalphalangeal (MTP) joint fusion (28750).
Data Analysis
Patient records with at least 6 months of follow-up from their initial appointment that included the identified PRO scales were screened for inclusion. To maximize sample size, we included all data when any of the PRO pairs of interest were present for a patient (PROMIS PF vs PI; PROMIS PF vs NPRS; PROMIS PI vs NPRS). Descriptive statistics were reported for all patients. Because the assumption of normality (Shapiro-Wilk test, skewness, kurtosis, and visual assessment of histograms) was not met for all variables, Spearman rho (ρ) correlation was used to determine the concurrent validity between each PROMIS domain (PF, PI) and the NPRS for each individual CPT code, as well as for all patients. Occurrence of floor and ceiling effects was assessed by examining the frequency of the lowest and highest possible scores for each scale (eg, NPRS, low = 0; NPRS, high =10). The low and high possible scores for the PROMIS scales based on the current sample were as follows: PF, low: 19.1; PF, high: 73.4; PI, low: 38.7; and PI, high: 80.7. Only one CPT code (pantalar fusion, CPT code 28705) was not analyzed individually, as there were only 5 patients in the sample. Similar to prior studies utilizing PROMIS, Spearman correlation strengths were categorized as excellent (>0.7), excellent-good (0.61 to 0.70), good (0.4 to 0.6), or poor (<0.4). 16 Scores are presented as the means ± SD unless otherwise indicated and significance was set at P < .05.
Results
Preoperatively, the average NPRS, PROMIS PF, and PROMIS PI scores were 3.6 (SD, 2.9; range, 0-10), 41.9 (SD, 8.8; range, 19.1-73.4), and 59.5 (SD, 7.3; range, 38.6-80.7), respectively (Table 2). Postoperatively, the average NPRS, PROMIS PF, and PROMIS scores were 1.8 (SD, 2.2; range, 0-10), 42.3 (SD, 8.5; range, 19.1-70.0), and 55.2 (SD, 8.6; range, 38.7-79.0), respectively (Table 2). The mean follow-up was 14.4 months (range, 6.0-34.0 months) (Table 2).
For all patients, preoperative PROMIS PI demonstrated an excellent-good correlation with preoperative PROMIS PF (ρ = −0.66, P < .01) (Table 3). The same excellent-good correlation was found between postoperative PROMIS PI and postoperative PROMIS PF (ρ = −0.69, P < .01). In contrast, poor correlations were found between preoperative NPRS and PROMIS PF (ρ = −0.32, P < .01) and postoperative NPRS and PROMIS PF (ρ = −0.33, P < .01). Good correlations were found between NPRS and PROMIS PI preoperative scores (ρ = 0.46, P < .01) and postoperative scores (ρ = 0.51, P < .01). Taken together, the PROMIS PF domain demonstrated a stronger correlation to PROMIS PI in the preoperative and postoperative setting than NPRS. Similar findings were noted when patients were grouped by CPT code (Table 4). For each individual CPT code, preoperative PROMIS PI demonstrated a stronger correlation with PROMIS PF than NPRS. Likewise, with the exception of CPT code 27870 (ankle arthrodesis), preoperative PROMIS PI was more strongly correlated to postoperative scores than NPRS.
Spearman Rho (ρ) for Selected Patient-Reported Outcome Combinations.
Abbreviations: NPRS, Numeric Pain Rating Scale; PF, Physical Function; PI, Pain Interference.
All correlations were significant (P < .05).
Spearman Rho (ρ) for Selected Patient-Reported Outcome Combinations by CPT Code.
Abbreviations: NPRS, Numeric Pain Rating Scale; PF, Physical Function; PI, Pain Interference.
All scales except PROMIS PF demonstrated floor effects; however, ceiling effects were much lower. The proportion of lowest scores for NPRS was 23.5% preoperatively and 43.6% postoperatively for the entire sample (Table 5). Specific CPT codes demonstrated high floor effects preoperatively (range, 11.9%-33.6%) and postoperatively (range, 31.8%-55.6%) compared with PROMIS PI (preoperatively, <3.5%; postoperatively, < 16.7%) (Table 6). Neither NPRS nor PROMIS PI showed ceiling effects higher than 12.2%. PROMIS PF showed negligible floor and ceiling effects.
Frequency of Low and High Scores on Patient-Reported Outcome Scales for All Patients.
Abbreviations: PROMIS, Patient-Reported Outcomes Measurement Information System.
Percentage of Patients Observed With the Lowest Score and Highest Score per Current Procedural Terminology Code for Each Scale.
Abbreviations: NPRS, Numeric Pain Rating Scale; PF, Physical Function; PI, Pain Interference.
Discussion
This work continues the necessary research aimed at evaluating legacy PRO instruments (eg, NPRS) compared with newer, universal PRO tools (eg, PROMIS). Ultimately, narrowing down the multitude of PRO questionnaires to a complete, yet concise set will offer comprehensive coverage of valuable patient insight, while also decreasing questionnaire fatigue. Overall, PROs not only allow patients the ability to assess whether a treatment was a success, but also inform surgeons of possible areas of care improvement and allow shared decision-making. With the Orthopaedic Foot and Ankle Outcomes Research (OFAR) network using PROMIS as one of the standardized outcome measures, 8 the findings in the current study provide additional value. While previous work has evaluated other legacy foot and ankle–specific PRO tools compared with PROMIS,5,11 this is the first study that utilizes the commonly used NPRS in a cohort of patients undergoing common elective foot and ankle procedures.
PROMIS PI demonstrated a strong-moderate correlation both preoperatively and postoperatively with PROMIS PF. A number of prior studies have found a similar relationship between PROMIS PF and PI across the orthopedic surgery spectrum. In one study of 1299 patients presenting to an upper extremity orthopedic clinic, there was a strong-moderate correlation (Pearson correlation [r] = −0.60, P < .05) between PROMIS PF and PI. 9 Similarly, Overbeek et al concluded that there was a moderate correlation between PROMIS PF and PI (r = −0.51, P < .001) in a cohort study of 93 patients with confirmed upper extremity illness. 12 Moreover, these results are consistent with those obtained in the spine literature, as Kendall et al found a strong correlation between PROMIS PF and PI (r = −0.72, P < .05) among 1900 patients presenting to a spine clinic. 10
In agreement with the current study, Nixon et al revealed a strong PROMIS PF and PI correlation (r = −0.76, P < .001) in a cohort of 85 patients with hallux valgus. 11 In addition, the current study broadens these findings by the inclusion of multiple foot and ankle procedures, all of which demonstrated a similar relationship between PROMIS PI and PF when combined into a single cohort and when analyzed by individual CPT code. The stronger correlation of PROMIS PF with PI compared with NPRS may occur because of the focus on pain interference rather than pain intensity, or the IRT and CAT modeling used in the PROMIS PI instrument. Nevertheless, these findings suggest that the relationship between PROMIS PF and PI is likely generalizable across common elective foot and ankle procedures and potentially to other procedures outside of the foot and ankle realm.
These data show that PROMIS PI may differentiate lower levels of pain better than NPRS clinically. For the overall sample, 23.5% patients reported no pain (ie, score of 0) on the NPRS scale preoperatively and 43.6% postoperatively (Table 5). The comparative values for PROMIS are less than 1% preoperatively and postoperatively except for PROMIS PI, which was 11.9% postoperatively. PROMIS scales did not show floor effects above 3.5% for any procedures. These results are similar to those of a previous study that also showed low floor and ceiling effects in PROMIS PF and PI in foot and ankle surgical patients. 7 The PROMIS scales were able to scale lower levels of pain interference better than the NPRS. Essentially, no pain on the NPRS (ie, score of 0) is not equivalent to no pain interfering with daily activities or low physical abilities. Postoperative floor effects for NPRS are high (up to 55.6% of patients), while equivalent values are less than 16.7% for PROMIS PI and negligible for PROMIS PF. This suggests that PROMIS scales may detect variation in patients with low pain and physical function better than the NPRS.
This study does have its limitations. There may be the presence of selection bias based on our inclusion criteria, as those who recovered prior to 6 months may have opted not to return to the clinic. Additionally, while the PROMIS questionnaires were given to all patients present in clinic, completing them remained optional. Furthermore, at our institution, PROMIS was only administered in English and Spanish. Completing the PROMIS questionnaires might have been difficult for patients whose primary language differed from English or Spanish. While our combined sample size was over 500 patients, a strategy employed in this study was to allow a variable sample size per PRO pairs of interest. This increased our sample representation up to 27.7% for some variables and optimized stratifying by CPT code. Nevertheless, a uniform representation of the sample and larger samples per CPT code may have impacted the statistical analysis. Larger, more comprehensive studies may be required to further verify the ability to generalize our results. Finally, the current study presents correlations of outcome measures in both the preoperative and postoperative setting.
Conclusion
In a sample of patients undergoing elective foot and ankle procedures, both NPRS and PROMIS PI had concurrent validity with PROMIS PF; however, the PROMIS PI domain demonstrated stronger concurrent validity with PROMIS PF than NPRS. In addition, PROMIS PI showed the ability to detect lower levels of pain compared with NPRS. PROMIS PF and PI provided a superior and comprehensive assessment of pre- and postoperative physical function and pain interference. The administration of PROMIS PI may be utilized to assess a patient’s preoperative level of pain and its impact on function and well-being. Our findings may be able to aid surgeons in guiding postoperative patient expectations regarding physical function and pain anticipation in the preoperative setting prior to undergoing elective foot and ankle surgery.
Supplemental Material
FAI803314-ICMJE – Supplemental material for PROMIS Pain Interference Is Superior vs Numeric Pain Rating Scale for Pain Assessment in Foot and Ankle Patients
Supplemental material, FAI803314-ICMJE for PROMIS Pain Interference Is Superior vs Numeric Pain Rating Scale for Pain Assessment in Foot and Ankle Patients by David N. Bernstein, Meghan Kelly, Jeffrey R. Houck, John P. Ketz, A. Samuel Flemister, Benedict F. DiGiovanni, Judith F. Baumhauer and Irvin Oh in Foot & Ankle International
Footnotes
Authorship Statement
We have included 8 authors on this manuscript, as this was a joint effort between the University of Rochester’s Foot & Ankle Division, a foot and ankle fellow, and a current medical student. All authors have contributed significantly to the design of the study, have contributed to data collection, and have participated in the writing of the manuscript. All authors assume full responsibility for the content of the manuscript.
Ethical Review Committee Statement
This project was approved by the University of Rochester Medical Center (IRB 00062579).
Informed Consent Statement
This is an internal database review. We have IRB approval for review of the data in the foot and ankle repository, but individual patient consent is not required.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Judith F. Baumhauer, MD, MPH, is the vice president of PROMIS Health Organization. ICMJE forms for all authors are available online.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
