Abstract
Background:
The Patient-Reported Outcomes Measurement Information System (PROMIS) is a powerful set of patient-reported outcome measures (PROMs) that are gaining popularity throughout orthopaedic surgery. The use of both adult and pediatric PROMIS questionnaires in orthopaedic sports medicine limits the value of the PROMIS in routine sports medicine clinical care, research, and quality improvement. Because orthopaedic sports medicine surgeons see patients across a wide age range, simplifying the collection of PROMIS computer adaptive tests (CATs) to a single set of questionnaires, regardless of age, is of notable value.
Purpose/Hypothesis:
The purpose was to determine the strength of the correlation between the pediatric and adult PROMIS questionnaires. We hypothesized that there would be a high correlation between the adult and pediatric versions for each PROMIS domain, thereby justifying the use of only the adult version for most sports medicine providers, regardless of patient age.
Study Design:
Cohort study (Diagnosis); Level of evidence, 2.
Methods:
Between December 2018 and December 2019, all pediatric sports medicine patients presenting to a single, academic, orthopaedic sports medicine clinic were asked to participate in the present study with their parents’ consent. Patients were asked to complete a set of adult PROMIS domains (Physical Function and/or Upper Extremity, Pain Interference, and Depression) as well as a set of pediatric PROMIS domains (Mobility and/or Upper Extremity, Pain Interference, and Depressive Symptoms). Concurrent validity was assessed using Pearson correlation coefficients (r). Ceiling and floor effects were determined.
Results:
A total of 188 patients met our inclusion criteria. The correlation between the adult and pediatric PROMIS Upper Extremity, Physical Function and Mobility, Pain Interference, and Depression and Depressive Symptoms forms were high-moderate (r = 0.68; P < .01), high-moderate (r = 0.69; P < .01), high (r = 0.78; P < .01), and high (r = 0.85; P < .01), respectively. Both adult and pediatric depression-related PROMIS domains demonstrated notable floor effects (adult: 38%; pediatric: 24%). The pediatric PROMIS Upper Extremity domain demonstrated a ceiling effect (20%).
Conclusion:
Adult PROMIS CATs may be used in an orthopaedic sports medicine clinic for both adult and pediatric patients. Our findings will help decrease the amount of resources needed for the implementation and use of PROMs for patient care, research, and quality improvement in orthopaedic sports medicine clinics.
Overall, 75% of American families with school-aged children have at least 1 child participating in organized athletics, which leads to approximately 45 million children playing sports at any one time. 20 Because of this significant level of youth sports participation, there has been increasing attention placed on pediatric sports medicine injuries. This is especially true because of the rising numbers of total injuries, with many related to higher rates of youth sports specialization and overuse.17,22,23 The main objectives of the treatment of injured pediatric sports medicine patients are to alleviate pain and return to baseline athletic function. To accurately evaluate pain and functional improvement after an intervention, patient-reported outcome measures (PROMs) have emerged as valuable tools.
One set of PROMs that continues to grow in popularity is the Patient-Reported Outcomes Measurement Information System (PROMIS). The PROMIS is a general set of PROMs normed to the population with a mean t score of 50 and SD of 10 and was developed through support from the National Institutes of Health; it utilizes item response theory as part of a computer adaptive test (CAT).7,8,15,18 Both adult and pediatric PROMIS questionnaires have been developed and validated in their respective populations as well as studied in adult 3 and pediatric 19 orthopaedic sports medicine populations.
In particular, orthopaedic sports medicine surgeons typically see both pediatric and adult patients on any given clinic day. This diverse patient population makes it a challenge to seamlessly switch between adult and pediatric PROMIS forms patient by patient in real time in a busy orthopaedic sports clinic. Moreover, the utilization of both pediatric and adult forms can reduce clinical research or quality improvement efficacy, as these forms cannot be easily aggregated and compared. Therefore, the primary goal of this study was to determine the strength of the correlation between pediatric and adult PROMIS Mobility/Physical Function, Upper Extremity, Pain Interference, and Depressive Symptoms/Depression assessments in pediatric sports medicine patients. We hypothesized that there would be a high-moderate to high correlation between the adult and pediatric versions for each domain, thereby justifying the use of only the adult version for most sports medicine providers, regardless of patient age.
Methods
This study was approved by the appropriate hospital’s institutional review board. Between December 2018 and December 2019, pediatric patients presenting to 1 of 2 board-certified orthopaedic sports medicine surgeon’s (V.M. and E.C.M.) clinic were asked to participate in the present study with their parents’ consent. Patients were included if their age was between 8 and 17 years and they could complete the PROMIS questionnaires independently. Pediatric patients were instructed to complete both the adult and pediatric PROMIS sets by themselves. The adult PROMIS set consisted of the Pain Interference (V 1.1), Depression (V 1.0), and Upper Extremity (V 2.0) and/or Physical Function (V 2.0) domains, depending on the reason for visiting the clinic. The pediatric PROMIS set consisted of the Pain Interference (V 2.0), Depressive Symptoms (V 2.0), and Upper Extremity (V 2.0) and/or Mobility (V 2.0) domains, depending on the reason for visiting the clinic. Patients were allowed to complete both the Upper Extremity and Physical Function forms if they presented to the clinic with multiple concerns, with one focused on the upper extremity and one focused on another anatomic area (eg, knee). The PROMIS questionnaires are designed so that higher scores represent “more” of the domain being measured (eg, a higher PROMIS Upper Extremity score reflects greater upper extremity functional ability). After completing all pediatric PROMIS forms, the patients then completed all adult PROMIS forms. Only patients who completed all adult and pediatric PROMIS forms of interest were included in the analyses.
In addition to completing the PROMIS questionnaires, patients also completed a short form about athletic participation (sports played [if any], years of active participation, and months per year of active participation) and basic demographic information (age, sex, and race). All surveys were distributed and stored using Research Electronic Data Capture, a web-based Health Insurance Portability and Accountability Act–compliant data management and collection application maintained by Vanderbilt University. 14
Descriptive statistics were calculated and reported. Pearson correlation coefficients (r) were calculated between adult and pediatric PROMIS instruments. A subgroup correlation analysis by sex was also performed. Similar to previous studies, correlation strengths were categorized as the following: high (≥0.70), high-moderate (0.61-0.69), moderate (0.40-0.60), moderate-weak (0.31-0.39), or weak (≤0.30). 13 In addition, ceiling and floor effects for both adult and pediatric PROMIS forms were determined by calculating the number of patients who had PROM scores toward the maximum and minimum scores possible and dividing it by the total number of patients in the study sample. 2 Consistent with the literature, a true floor or ceiling effect was considered present if ≥15% of patients were at either of these 2 score extremes. 9
For all analyses, significance was set a priori at P < .05. Statistical analyses were conducted using Stata/SE 14.2 for Mac (StataCorp).
Results
Of the 202 patients who were considered for inclusion, a total of 188 pediatric patients (93%) fit our inclusion criteria. The 14 patients (7%) excluded had incomplete PROMIS questionnaires. The patient characteristics of those included in the present study were similar to those patients who were excluded (Appendix Table A1, available in the online version of this article).
Of those included, the mean age was 15 years (range, 8-17 years), and a majority of patients were male (63%) and White (58%) (Table 1). Overall, 90% of our patient sample played organized sports, with football being the most popular (9%). Nearly half of all patients (46%) participated in athletics 10 to 12 months out of each year. The most common primary reason for seeking care was a knee injury (59%).
Patient Characteristics (n = 188) a
Data are shown as n (%) unless otherwise indicated.
The mean PROMIS Upper Extremity, Physical Function or Mobility, Pain Interference, and Depression or Depressive Symptoms scores are reported in Table 2.
Adult and Pediatric PROMIS Scores
The correlation between the adult and pediatric PROMIS Upper Extremity, Physical Function and Mobility, Pain Interference, and Depression and Depressive Symptoms forms were high-moderate (r = 0.68; P < .01), high-moderate (r = 0.69; P < .01), high (r = 0.78; P < .01), and high (r = 0.85; P < .01), respectively (Table 3). When analyzed by sex, the correlation between the adult and pediatric PROMIS Upper Extremity, Physical Function and Mobility, Pain Interference, and Depression and Depressive Symptoms forms in male patients were high (r = 0.70; P < .01), high-moderate (r = 0.63; P < .01), high (r = 0.78; P < .01), and high (r = 0.86; P < .01), respectively (Table 4). In addition, the correlation between the adult and pediatric Physical Function and Mobility, Pain Interference, and Depression and Depressive Symptoms forms in female patients were high (r = 0.75; P < .01), high (r = 0.76; P < .01), and high (r = 0.82; P < .01), respectively (Table 4). Of note, the correlation between adult and pediatric PROMIS Upper Extremity forms could not be determined in the female patient subgroup, given the limited sample size (n = 6).
Pearson Correlation Coefficients (r) Between Adult and Pediatric PROMIS Forms a
Dashes indicate that a Pearson correlation coefficient (r) could not be calculated.
Pearson Correlation Coefficients (r) Between Adult and Pediatric PROMIS Forms by Sex a
Dashes indicate that a Pearson correlation coefficient (r) could not be calculated.
Excluded because n = 6.
For both the PROMIS Depression (adult) and PROMIS Depressive Symptoms (pediatric) forms, there were large floor effects (38% and 24% of patient sample, respectively) (Table 5). In addition, the pediatric PROMIS Upper Extremity form demonstrated a ceiling effect (20% of patient sample). No other adult or pediatric PROMIS domain demonstrated a ceiling or floor effect when using ≥15% of patients as the cutoff.
PROMIS Domain Ceiling and Floor Effects
Discussion
The results of this study indicate that among pediatric sports medicine patients, the adult version of the PROMIS CAT forms shows high-moderate to high correlation with the pediatric versions across numerous health domains. Moreover, the adult PROMIS CAT versions for the Physical Function/Mobility and Pain Interference domains showed no significant floor or ceiling effects in the pediatric population, whereas the pediatric Upper Extremity form demonstrated a substantial ceiling effect. These key findings justify consideration for using adult PROMIS CAT forms in pediatric patients, thus simplifying the PROM collection process for clinical and research purposes.
Within orthopaedic surgery, there is a growing call to increase the use of PROMs, especially the PROMIS, in day-to-day clinical practice and shared decision making.1,6 However, the collection of PROMIS questionnaires as part of routine clinical care requires resource allocation (eg, monetary and personnel), including investments in technology if CATs are to be used. With both validated sets of adult and pediatric PROMIS forms, this process may be more burdensome and challenging in orthopaedic sports medicine clinics where both pediatric and adult patients are treated. Because the collection of PROMs is not yet fully widespread, ensuring as easy an implementation as possible for clinics not yet collecting PROMs is crucial. We believe that our findings support an easier implementation model because only 1 set of PROMIS domains needs to be implemented in sports medicine clinics.
Studies evaluating the PROMIS in clinical settings continue to increase, with many seeking to determine the correlation strength of the PROMIS to legacy instruments. The goal of such studies is to evaluate whether the PROMIS may be acceptable for outcome measurements in a number of diverse patient populations. In pediatric and adult spine surgery, researchers found that many PROMIS domains demonstrated high-moderate to high correlation with similar legacy PROM domains, including those of the Scoliosis Research Society questionnaires.5,9 Thus, these results suggest that PROMIS domains may be acceptable PROM alternatives for spine patients. In the present study, we did not compare the PROMIS with legacy instruments; instead, we compared 2 different PROMIS sets—adult and pediatric—in a similar way. The goal was to see if only 1 set would be acceptable across the age spectrum of pediatric orthopaedic sports medicine patients. We found high-moderate to high correlation between the 2 sets of PROMIS questionnaires overall and within sex subgroups, suggesting concurrent validity (ie, similar constructs being measured by both sets of the PROMIS forms). However, differences do exist, and orthopaedic sports medicine surgeons should consider them if they elect to utilize only 1 set of PROMIS questionnaires in their clinic. For example, the reference population for the adult forms is adults, while the reference population for the pediatric forms is children. 16 Nonetheless, given the robust design of the instruments, we believe that the PROMIS scores from either set of questionnaires can provide a general view of the health status of a pediatric patient, and the change in PROMIS scores over time, if the same set of PROMIS forms is used, can be valuable in monitoring clinical well-being. While our findings imply that adult PROMIS forms may be acceptable in a pediatric orthopaedic sports medicine population, the pediatric forms should be used if resources allow it. However, if resource limitations exist, we believe that our findings demonstrate that the insights gained using adult PROMIS forms in the pediatric orthopaedic sports medicine setting are of value and better than no insight.
It is also important to evaluate the ceiling and floor effects of PROMs to determine the ability of the instruments to differentiate between varying levels of self-reported symptoms. Lower ceiling and floor effects reflect an improved ability to differentiate patients. Within a pediatric sports medicine patient population, we found a high floor effect for both the pediatric (24%) and the adult (38%) PROMIS Depressive Symptoms/Depression forms. This is consistent with numerous reports in the orthopaedic literature.4,10-12 These high floor effects are likely multifactorial in nature, and patients may be hesitant or unwilling to disclose true mental health concerns in such a manner. This could result in a high proportion of responses indicating an absence of depressive symptoms (ie, high floor effect). However, future research is warranted to better make this determination. In our study, there was a limited difference between the severity of the floor effect between the 2 sets of questionnaires (ie, adult and pediatric). Thus, no recommendation on whether to use the adult or pediatric PROMIS questionnaires can be derived from this finding. However, the pediatric PROMIS Upper Extremity domain demonstrated a notable ceiling effect, while the adult PROMIS Upper Extremity domain did not. This suggests that the adult version of the PROMIS Upper Extremity domain may better differentiate pediatric sports medicine patients; therefore, this favors the use of the adult PROMIS Upper Extremity form. However, future research will be necessary to see if this remains evident as updated versions of the questionnaire set are developed. Last, it is important to note that both the adult and the pediatric PROMIS Physical Function/Mobility and Pain Interference questionnaires performed well with no notable floor or ceiling effects. Thus, our findings suggest that either set of PROMIS forms would be appropriate for use in an orthopaedic sports medicine clinic.
Our study has a number of limitations that should be considered. First, our findings are from a single, urban, academic medical center with the participation of 2 fellowship-trained orthopaedic sports medicine surgeons. Therefore, our results may not be generalizable to other sports medicine clinics in more rural settings or across orthopaedic subspecialties. Second, while we instructed pediatric patients to complete the questionnaires themselves, it is possible that parents assisted their children in completing the questionnaires. However, observational evidence suggests that this concern was limited. Last, our study strictly measured the concurrent validity of adult and pediatric PROMIS forms at a single visit. We did not evaluate the responsiveness of both sets of questionnaires, nor did we demonstrate the exact agreement between the 2 sets of PROMIS forms. Therefore, it is important that readers not take our findings as a suggestion to freely move back and forth between adult and pediatric PROMIS questionnaires. For example, it would be inappropriate based on this study to evaluate the change in the t score if a patient’s functional status was measured with the PROMIS Mobility questionnaire at one clinic visit and then the PROMIS Physical Function form at the next clinic visit. However, our findings do support the consistent use of a single set of PROMIS questionnaires (eg, adult) across all patients over time. Further, future work should seek to determine if the responsiveness of the adult and pediatric questionnaires is consistent in a pediatric orthopaedic sports medicine population over time (eg, preoperative to postoperative). This is especially important given that Mulcahey et al 21 demonstrated that pediatric PROMIS measures did not detect changes in pediatric patients with cerebral palsy undergoing a surgical intervention.
Overall, our study demonstrates that the adult PROMIS Upper Extremity, Physical Function, Pain Interference, and Depression domains captured similar outcomes to the pediatric PROMIS Upper Extremity, Mobility, Pain Interference, and Depressive Symptoms forms in a pediatric orthopaedic sports medicine patient population. While we recommend using the pediatric forms in the pediatric population whenever possible, our results can be used to help ease the introduction of routine PROMIS CAT collection into sports clinics by allowing surgeons to utilize only 1 set of PROMIS questionnaires for all patients when resources are limited. In such cases, we suggest the use of the adult PROMIS questionnaires. However, as PROMs, especially the PROMIS, become more widespread and a critical element in direct patient care, we expect that investments will be made to ensure that pediatric patients receive only pediatric questionnaires and adult patients receive only adult questionnaires. Additionally, we suspect that continued work will take place to further develop the adult and pediatric PROMIS questionnaires to ensure even better performance with limited floor and ceiling effects.
Supplemental Material
DS_10.1177_0363546520966034 – Supplemental material for Pediatric PROMIS Computer Adaptive Tests Are Highly Correlated With Adult PROMIS Computer Adaptive Tests in Pediatric Sports Medicine Patients
Supplemental material, DS_10.1177_0363546520966034 for Pediatric PROMIS Computer Adaptive Tests Are Highly Correlated With Adult PROMIS Computer Adaptive Tests in Pediatric Sports Medicine Patients by David N. Bernstein, Sreten Franovic, D. Grace Smith, Luke Hessburg, Nikhil Yedulla, Vasilios Moutzouros and Eric C. Makhni in The American Journal of Sports Medicine
Footnotes
Submitted April 3, 2020; accepted July 23, 2020.
One or more of the authors has declared the following potential conflict of interest or source of funding: V.M. has received hospitality payments from Smith & Nephew and Stryker. E.C.M. has received consulting fees from Smith & Nephew; hospitality payments from Smith & Nephew, Stryker, Pinnacle, and Arthrex; and education payments from Pinnacle and Arthrex. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
