Abstract
Control over symptoms postconcussion is central to an active self-directed recovery process. Therefore, assessing a patient’s confidence in controlling their symptoms and facilitating their concussion recovery is an important component of treatment. Previously, no measures existed to assess symptom-specific self-efficacy (SE) in pediatric concussion recovery. SE is an individual’s belief or confidence in their capabilities to execute action plans necessary to perform certain behaviors. Based on this definition, we developed the Progressive Activities of Controlled Exertion–Self-Efficacy (PACE-SE) scale to measure a patient’s SE related to pediatric concussion recovery-specific activities. The aim of this article is to present the psychometric characteristics (evidence of reliability, validity) of the PACE-SE scale. The 17-item PACE-SE was administered to children and adolescents, 10–18 years of age, recovering from a diagnosed concussion as part of a standard clinical evaluation. Results revealed a four-factor structure producing the following scales: Managing My Stress, Managing My Activity, Seeking Adult Assistance, and Maintaining Positive Outlook. The PACE-SE scores indicated excellent internal consistency reliability with reasonable test–retest reliability over time. Evidence for the association between recovery status and greater confidence and control over recovery-related activities as measured by the PACE-SE was supported by: (1) an inverse association with symptom status reflecting lower confidence for managing recovery with higher symptom load, (2) greater reported problems with school performance associated with lower SE, (3) positive change in SE ratings across two clinic visits associated with symptom improvement, and (4) a significant difference in SE ratings evident between recovered and nonrecovered patients. The psychometric evidence supporting the PACE-SE scale provides the clinician with a measure to understand the child/adolescent patient’s self-confidence in facilitating their concussion recovery.
Introduction
Evidence to assist the diagnosis and management of concussions has increased substantially over the past 20 years. 1,2 Active treatment targeting key clinical symptom profiles 3 is now favored over a passive “rest” model 4 with the patient’s active participation essential for navigating the challenges of the recovery process. The patient’s knowledge of their injury presentation and perception of their control over their symptoms is central to this treatment model and composes their sense of recovery-related self-efficacy (SE). To date, however, no measure of SE exists specific to concussion recovery in children and adolescents. This article introduces a new assessment/tracking measure that helps to define the patient’s SE, that is, their confidence in managing the challenges of recovery from their concussion. We describe the development of the Progressive Activities of Controlled Exertion–Self-Efficacy (PACE-SE) scale and present psychometric evidence regarding its reliability and validity in support of its use in the assessment and treatment of concussion.
Bandura 5 defines SE as one’s belief in their capabilities to execute action plans necessary to perform a given behavior. These beliefs or perceptions influence one’s thoughts, emotions, and choices that are made to engage in activities. 6 The construct of SE has a rich history of application across health conditions such as diabetes, 7 cancer, 8 and asthma 9 as well as sport performance 10 and academic learning. 11 We extend these concepts to pediatric concussion recovery. Active engagement in one’s treatment plan is facilitated by the patient feeling capable and confident in their abilities to exert control over challenges, problems, and obstacles related to their recovery. This SE belief system is not a global trait but is instead a set of beliefs linked to distinct realms of functioning, 12 in this case concussion recovery. As such, there is no general, all-purpose measure of perceived SE. Instead, one’s confidence in performing concussion recovery-specific behaviors requires a unique assessment measure that captures the inherent challenges, symptom-specific problems, and obstacles encountered in recovery. SE for concussion recovery is therefore conceptualized as a health-specific realm of self-confidence influencing the individual’s control over recovery activities.
SE has been explored previously with adults who have sustained traumatic brain injury. Cicerone and Azulay 13 found that perceived SE for managing cognitive symptoms best predicted global life satisfaction postinjury. SE beliefs affect cognitive, affective, and motivational processes influencing rehabilitation outcomes. 14 Gagnon 15 found decreased SE for performance of physical activities in children with concussion compared with their own preinjury confidence and that of matched uninjured children. In two separate studies of adults with concussions, Belanger and colleagues found that knowledge, SE, and attributions are correlated with postconcussion symptom severity, 16 while higher SE predicted a positive response to cognitive rehabilitation in military service members. 17
This study presents the development and psychometric examination of a pediatric concussion-specific measure of SE that is based on an active treatment framework called the PACE rehabilitation model. 18 The PACE model is an active concussion rehabilitation approach that incorporates the gradual reintroduction of activities within cognitive/school, physical/recreational, and social domains. 3 The rehabilitation model teaches the patient and family an active, constructive approach to managing their recovery: maintaining a positive outlook toward recovery and strategic management of daily activities. The PACE model emphasizes the injured child’s progressive, systematic return to key life activities with four key features: (1) setting the positive foundation for recovery, (2) defining the parameters of the child’s activities and exertional response across the day, (3) teaching activity management and symptom monitoring skills, and (4) reinforcing progress toward recovery. The PACE-SE scale was developed 19,20 to better define and track the patient’s perceived state of control over their injury and recovery, based on the conceptual background of the PACE rehabilitation model. Items were derived by experienced concussion clinicians to reflect the four key features listed above. Preliminary psychometric examination of the measure was conducted with an initial sample by Burns et al. in 2016. 19 The present article expands upon that early study with a more comprehensive psychometric analysis. In addition, the relationship of the PACE-SE measure to preinjury anxiety/depression history, an aspect of test validity, was explored in 2020. 20
The aim of this study was to examine multiple sources of psychometric evidence supporting the reliability/precision and validity/accuracy of the PACE-SE scale scores as a measure of patient SE. Specific psychometric support of the PACE-SE scale was based on determining whether items conceptually factored together and whether these items clusters matched the proposed rehabilitation model (i.e., validity of the construct/internal structure). We expected that items within the scales would tend to be endorsed similarly (i.e., internally consistent) and demonstrate moderate stability in an individual’s pattern of responses over time (i.e., test–retest reliability). Based on an expected progression of symptoms and increased SE due to active intervention, some attenuation of this stability was also expected due to variability amongst individuals’ recovery. It was also hypothesized that SE scores should correlate with other measures of outcome including overall symptom burden, exertional symptom responses, and school stress and difficulties. The scores were also hypothesized to be sensitivity to change in recovery, with increased SE as the person improves overall. Relatedly, we hypothesized differences in SE ratings between individuals who were recovered and not recovered. Last, we expected to see patterns of response on PACE-SE scale that were relatively similar across age groups, yet with some differences based on preinjury background such as sex, anxiety, or attention-deficit/hyperactivity disorder (ADHD). No differences were expected based on acute injury characteristics.
Method
Participants
Participants included 397 children and adolescents (10–18 years of age) diagnosed with a concussion during an initial outpatient clinical evaluation (visit 1; V1) at a large regional hospital, with 199 of those participants returning for a second outpatient visit (V2) that was based on their clinical needs for ongoing care. Concussion diagnosis was confirmed by the treating clinician, using criteria from the 2013 American Academy of Neurology guidelines. Table 1 presents the sample demographic information for the two visits (e.g., V1: 53.1% female, 63.2% White, 18.1% Black). Study inclusion criteria included first clinic visit (V1) within 90 days of the injury (M = 18.1, standard deviation [SD] = 15.3, median = 14) to capture the diverse range of postinjury cases, while the mean (SD) V1–V2 interval was 16.2 (10.3) days (median = 14). Injury characteristics are reported in Table 2 indicating the most common cause of injury being sports/recreational activities (64.7%), followed by falls (11.1%) and motor vehicle-related injuries (7.3%). Acute injury characteristics indicate 9.9% of the sample with loss of consciousness, 14.4% with retrograde amnesia, and 17.9% with anterograde amnesia.
Demographic Characteristics
p < 0.05.
SD, standard deviation.
Injury Characteristics
Preinjury diagnoses revealed anxiety disorders more frequent for girls (31.9%) than boys (22.0%, p < 0.03) with no significant boy/girl differences (p > 0.05) in rates of depression (12.8%), ADHD (18.9%), or learning disabilities (LD; 7.8%).
Data were collected under IRB approval with informed consent obtained from parents and 18-year-old participants, and assent from children and adolescents.
Measures
Progressive activities of controlled exertion–self-efficacy scale
The PACE-SE is a measure of SE, for patients ages 10–18 years, reflecting capabilities believed to be important to influence recovery following concussion. 20 As guided by Albert Bandura, 6 all items are framed from the perspective of perceived SE to accomplish the task with the initial stem “I am confident that I can…” Response choices followed a unidirectional, Guttman-type scale, ranging from 0 “Not confident that I can do it” to 10 “Highly confident that I can do it.” To become familiarized with the 0–10 response scale, the patient is first provided two practice items (i.e., confidence in lifting a 10-pound weight and confidence in lifting a 250-pound weight) prior to completion of the full scale.
In the initial development of the PACE-SE, 50 items were generated by a team of experienced concussion clinicians with reference to the four key features indicated in the PACE rehabilitation model. 18 This initial scale was piloted with 91 patients to examine their initial psychometric properties and led to the current and final set of 17 items that are based on reasonable item response distribution, exploratory factor analysis (EFA), and appropriate item-scale membership via internal consistency reliability coefficients. 19
Postconcussion symptom inventory, 2nd edition. 21
The Postconcussion Symptom Inventory, 2nd Edition (PCSI-2) is a developmentally validated symptom rating scale of postconcussion symptoms, with separate scales for parents of children ages 5–18 years and for self-report for ages 8–12 and 13–18 years. Scores for the three forms of the PCSI-2 have previously been examined for item content, factor structure, and appropriate reliability coefficients 22 (internal consistency, rater concordance, and test–retest), 21 yielding four symptom domains (physical, cognitive, emotional, sleep/fatigue). Internal consistency for the total PCSI-2 and subscales is strong for both parent and self-report, with moderate test–retest reliability. To control for preinjury problems with these nonspecific symptoms, retrospective ratings are collected with the current postinjury ratings, generating a retrospective adjusted postinjury difference (RAPID) score. 22 The RAPID scores of the self-report PCSI-2 were used in this study.
Children’s exertional effects rating scale-4 item. 23
The Children’s Exertional Effects Rating Scale-4 item (ChEERS4) is a momentary symptom rating scale that assesses the child’s perceived change in symptoms associated with an activity (cognitive, physical). The child is asked to provide a momentary rating of four symptoms (headache, fatigue, difficulties concentrating, irritability) on a scale from 0 to 10 accompanied by relevant emotive faces, before the beginning of the activity and again at the end of the activity. In this case, the patient was asked to rate these symptoms before and after cognitive testing conducted in clinic over an ∼45-min session. The exertional effects index (EEI) score is calculated as the sum of the increases in rating for each of the four symptoms. The psychometric properties of the ChEERS4 measure have been established in Sady, Vaughan, and Gioia, 23 establishing that this dynamic symptom response to activity can be differentiated from the PCSI-2 symptom report.
Concussion learning assessment and school survey, 3rd edition. 24
The Concussion Learning Assessment and School Survey, 3rd Edition (CLASS-3) is a self-report and parent-report measure of academic challenges following concussion. The CLASS-3 measures four areas pertaining to school performance: General Academic Concern, Academic Problems, School Stresses, and Academic Subjects. Each scale asks the patient to rate new or worsening academic problems/stresses since the concussion. A summed score is generated for each of the scales with each individual item on a scale of 0–3 for the first three domains and 0–4 for the fourth domain (academic subjects), generating total scale scores as follows: General Academic Concern (0–3), Academic Problems (0–42), School Stresses (0–18), and Academic Subjects (0–16). Psychometric evidence in support of the reliability/precision of the scores and in support of the validity/accuracy of the interpretation of the CLASS-3 scale scores has been established, 24 indicating appropriate item-scale membership, factor structure, stability over time, relationship to similar and dissimilar measures, and sensitivity to change over recovery.
Procedure
Participants completed the PACE-SE and the measures listed above as part of a standard clinical evaluation for concussion. The PACE-SE was administered during the initial visit (V1) following clinical interview, but prior to any specific psychoeducation related to concussion and prior to treatment planning. This timing allowed us to measure patient SE prior to any educational, rehabilitation, or behavioral intervention provided by the concussion specialists seen during their appointment. In the follow-up visit, the PACE-SE scale was administered during the testing/data collection phase at the beginning of the visit and prior to provision of any additional treatment guidance. Participants were given standardized administration instructions, including review of the practice items with the administrator to familiarize with the 0–10 scaling. The majority of the patients completed the rating scales on a computer, entering them directly into a research database (REDCap) 25 though some patients completed questionnaires on paper due to availability of computers or light sensitivity on screens with the data entered manually into REDCap later by clinic staff.
Statistical analyses
The psychometric properties were examined for the 17-item PACE-SE starting with the definition of the scale’s internal structure via EFA as a method supporting the valid interpretation of the scale. EFA was performed using Principal Axis Factoring with Promax rotation with Kaiser Normalization, while Kaiser–Meyer–Olkin measure was examined for sampling adequacy. Evidence for the reliability of the PACE-SE total score and scale scores was examined via Cronbach’s Alpha for internal consistency and with the intraclass correlation coefficient (ICC) as well as a Pearson correlation coefficient to assess the scores’ reliability over time. Valid interpretation of the PACE-SE scale scores required further exploration of their relationship with demographic factors, preinjury and injury characteristics, and other clinical measures (symptom scales, academic functioning) and was performed using Pearson correlations. As the PACE-SE was constructed to assist recovery management over time, the scale scores’ sensitivity to change over the two visits was examined using repeated-measures Analysis of Variance (ANOVA). Preliminary analyses were conducted comparing the sample of participants who completed only the V1 with the participants who completed the V2 for any important differences in demographic, history, or injury characteristics. A binary logistic regression analysis was performed to examine the predictive relations of the PACE-SE total and four-scale scores to a criterion, recovery status, as a source of evidence contributing to the validity of interpretation of the PACE-SE scores. All data analyses were conducted using IBM SPSS Statistics software, version 26, and JASP 0.18.3 for the EFA.
Results
Definition of the progressive activities of controlled exertion–self-efficacy scale structure
Evidence based on internal structure
To examine the internal structure of the PACE-SE, an EFA with oblique (Promax) rotation of the 17 PACE-SE items collected at the V1 was conducted. This scale structure sets the foundation for the examination of other evidence of the reliability and validity of its scores. Selection of the numbers of factors was determined by both conceptual and statistical methods, using parallel analysis. A four-factor solution was determined to be the best estimate with factor loadings set at >0.3 (see Table 3) accounting for 62.2% of the variance. The first factor (Seeking Adult Assistance) comprised items that reflect receiving adult assistance (parents, teachers) in managing recovery. Item 12 (“I can ask an adult to help me find things that make me feel better”), with similar loadings on Factors 2 and 1, was selected for Factor 2 due to its better conceptual fit (and with an item-total correlation = 0.63). The second factor (Managing My Activity) comprised six items that reflect confidence in managing one’s activities. Five of the six items loaded uniquely on this factor. Item 8 (“I can block out times during the day when I need to take rest breaks”), though with a weaker factor loading, was added to this scale based on the conceptual theme of this factor. The significant item-total correlation (0.57) of item eight with factor 1 supported its placement. Factor 3 (Maintaining a Positive Outlook) was defined by four items that reflect the person’s ability to remain positive during their recovery. Factor 4 (Managing My Stress) reflects items that pertain to managing stress during recovery. Intercorrelations between the four scales reveal moderate relationships, ranging from 0.474 to 0.660, demonstrating an expected association within the scale and still a unique identity across each factor.
Exploratory Factor Analysis of the PACE-SE, Factor Loadings, and Correlations from the 4-Factor Solution
Principal Axis Factoring with Promax rotation with Kaiser Normalization; Kaiser–Meyer–Olkin Measure of Sampling Adequacy = 0.920; Total Variance Accounted for = 62.2%.
Items chosen for each factor are indicated by bold text.
Factor Names: 1 = Seeking Adult Assistance, 2 = Managing My Activity, 3 = Maintaining Positive Attitude, 4 = Managing My Stress.
p < 0.001.
PACE-SE, Progressive Activities of Controlled Exertion–Self-Efficacy.
Sources of evidence for reliability of progressive activities of controlled exertion–self-efficacy scale scores
Based on the definition of the four-scale structure, evaluation of the scale scores and total score’s reliability was conducted, including examination of internal consistency coefficients and test–retest reliability coefficients.
Internal consistency
Internal consistency coefficients reflect the strength of the association among the items of a single scale, which speaks to the unity of the scale. Cronbach’s alpha 26 was calculated for the PACE-SE total score and the four scales. As indicated in Table 4, coefficients for the total score were strong for the full sample at V1 (α = 0.91). Coefficients for the four scales were reasonable, ranging from 0.71 (Maintaining Positive Outlook) to 0.84 (Managing My Activity), with some attenuation likely due to the small number of items in the scales. Examination of the item-total correlations within each of the scales indicates appropriate scale membership for each of the component items. The overall results indicate appropriate cohesiveness within each of the scales and the total score.
PACE-SE Reliability Coefficients
Retest interval: M = 16.2, SD = 10.3 (median = 14).
Intraclass correlation coefficient.
Test–retest reliability
Test–retest reliability coefficients are intended to reflect the stability of the scores in a measure over time. Reliability of PACE-SE total scale and four-scale scores were examined from V1 to V2 using Pearson’s r and the ICC for the two age groups and the total sample. The average retest interval was 16.2 days (SD = 10.3) between the two visits. Although the results presented in Table 4 demonstrate reasonable stability, we recognize that these correlations are affected to some degree by the fact that the measure is being completed by actively recovering patients, each of whom has their own unique recovery trajectory. This introduces additional variability unrelated to the stability of the measure itself in the statistical calculation, potentially reducing the test–retest coefficient. Examination of these statistics does, however, provide useful information regarding the consistency in ratings between these two-time points. Stability of the total scale scores for the total sample is appropriate (Pearson r = 0.68, ICC = 0.81, p < 0.001) as it is for most of the scales with the three-item Managing My Stress scale exhibiting the lowest coefficient for each age group. T-tests revealed no significant differences in the stability of the scores were found between the two age groups or between males and females.
Sources of evidence for valid interpretation of the progressive activities of controlled exertion–self-efficacy scale scores
Evidence based on relationship to demographic, preinjury, and injury characteristics
Examining the relationship between the PACE-SE and demographic, preinjury history factors and injury characteristics provides important context for the interpretation of the scores. At V1, no association was found between the age of the child/adolescent and the PACE-SE total score or any of the four scales using Pearson correlation coefficients. Neither parent education level (mother or father) nor racial background were found to be significantly related to PACE-SE scores, examined with ANOVA. With respect to sex of the rater, ANOVAs revealed that girls reported significantly lower confidence than boys on two of the subscales—Managing My Stress (p < 0.001, partial eta2 = 0.034) and Managing My Activity (p < 0.01, partial eta2 = 0.021)—and the PACE-SE total score (p < 0.01, partial eta2 = 0.017), with further analysis revealing the effect to be specific to the adolescent (13–18 year) age group. No significant relationships were found between acute injury characteristics (i.e., presence of loss of consciousness, retrograde/anterograde amnesia) and SE ratings on crosstab analysis. A small, significant positive Pearson correlation was found with time since injury to V1 and the SE ratings, indicating that the further away the V1 was from the injury, the higher degree of self-confidence (but with <2% of the variance accounted for in this relationship). Examination of preinjury history factors using t-tests revealed no significant difference in SE ratings for children with and without ADHD or LD at the V1. In contrast, children diagnosed with anxiety had significantly lower ratings of perceived SE on the PACE-SE total scale and four scales (e.g., total score, p < 0.01, partial eta2= 0.030) at V1 demonstrated through an ANOVA. Children diagnosed with depression also reported lower SE on the PACE-SE total scale (p < 0.01, partial eta2 = 0.022) and two of the scales—Managing My Stress (p < 0.001, partial eta2 = 0.038) and Seeking Adult Assistance (p < 0.01, partial eta2 = 0.028) at V1.
Evidence based on relationship with other measures
Validity evidence in support of the interpretation of the PACE-SE scores can also be examined via evidence based on its relationships with other measures, such as existing postconcussion symptom measures (i.e., PCSI-2, ChEERS4) and other functional outcomes (i.e., CLASS-3). Stronger correlations (Pearson correlation coefficients) demonstrate greater similarity in the constructs (convergent evidence), whereas lower associations demonstrate less similarity (discriminant evidence). Table 5 presents the correlations between the PACE-SE total score and the two symptom measures, PCSI-2 and the ChEERS4, for each age group at V1. Results indicate significant negative correlations between the PACE-SE total score and the total PCSI-2 RAPID score and the PCSI-2 Physical RAPID score for both age groups. Correlations with the PCSI-2 Cognitive, Emotional, and Sleep/Fatigue scales were significant only for the 13- to 18-year-old age group.
Relationship of the PACE-SE Total Score to Other Measures at V1 Using Pearson Correlations
p < 0.05.
p < 0.01.
p < 0.001.
CLASS-3, Concussion Learning Assessment and School Survey, 3rd Edition; ChEERS4, Children’s Exertional Effects Rating Scale-4 item;PCSI-2, Postconcussion Symptom Inventory, 2nd Edition.
Both age groups exhibited significant negative relationships between the PACE-SE total score and the dynamic symptom change measured by the EEI of the ChEERS4. The negative correlations indicate that the higher the level of symptom rating increase, the lower the SE in managing recovery. The greatest relationship was with the physical symptoms, indicating their relatively greater influence on SE, across the full age range. Younger children did not reflect such a relationship with cognitive, emotional, or sleep/fatigue symptoms. Both groups indicated a significant negative relationship between SE ratings and dynamic symptom change on the ChEERS4, indicating that the greater the symptoms increased with activity, the lower the confidence in exerting control over recovery. Overall, these findings point to important relationships between the child’s perceived confidence in controlling aspects of their recovery and the overall symptom burden and dynamic symptom response to activities.
SE and its relationship to school functioning, specifically the postconcussion challenges experienced in the academic setting, were examined with the CLASS-3 across the entire sample, as well as by age group. It should be noted that sample sizes for the CLASS-3 School Stresses scale were relatively low due to a change in the response scaling during data collection. Table 4 presents the correlations between these measures. Significant negative correlations with the PACE-SE are demonstrated for all aspects of academic experience for the adolescent group and for the Academic Problems and Academic Classes scales in the younger group. The findings highlight that the greater the number of academic problems experienced by the student, and the greater number of classes in which they are having difficulties, the lower the perceived SE for managing their concussion recovery. These findings indicate an important relationship between success in school and how confident the student feels in managing aspects of their recovery.
Sensitivity to change
A critical component of concussion assessment and management is the detection and monitoring of change during recovery. It is important for a clinical measure to reliably detect change in the student’s functioning consistent with their overall improvements in recovery. We examined the sensitivity of the PACE-SE to change over time, defined as from V1 to V2, applying a within-subjects repeated-measures ANOVA. The equivalence of the sample characteristics was examined for those returning for a second visit (V2 n = 199) relative to those who came for only one visit (V1 n = 197). No significant differences were found between the groups for mean age or percentages of race membership, parent educational level, or diagnoses of anxiety, depression, or ADHD/LD (p > 0.05). Additionally, no differences between the groups were found (p > 0.05) for the initial injury characteristics (loss of consciousness, retrograde amnesia, anterograde amnesia). Males were more likely, however, to have been only seen in clinic once (55%) compared with females (44%), while females were more likely to have had two clinic visits (56%) compared with males (45%) (p = 0.04). Time to V1 and the time interval between V1 and V2 were examined as they could possibly be associated with concussion outcomes. 21 Males and females did not differ in the time to V1 or the interval between the two visits (p > 0.05). For age, the sample was divided into children (ages 10–12 years) and adolescents (ages 13–18 years); time to V1 was shorter in the 10- to 12-year age group (M = 13.2 days) than the 13- to 18-year group (M = 18.7 days, p < 0.05). There were no differences by age in the V1–V2 interval (p > 0.05).
Table 6 reports the means and effect sizes associated with change over the period from V1 and V2 for the PACE-SE total score and the four scales. A significant increase in the total score was found from the first to the second visit (p < 0.001, partial eta2 = 0.243), accounting for a significant and substantial amount of variance. Relatively speaking, this degree of change is significantly greater than that seen merely by the factor of time since injury, as previously reported, which accounted for <2% of the variance. Similarly, the four scales each demonstrated significant, reliable change across the two visits, also exhibiting strong effect sizes. To establish that the significant improvement in PACE-SE scores is associated with recovery, the relationship of the change in the PACE-SE total score over the two visits was examined relative to the change in symptom status on the PCSI-2, and revealed significant correlations for the parent and 13- to 18-year-old samples (parent r = −0.24, p = 0.001 [n = 187]; 13–18, r = −0.42, p < 0.001 [n = 159]) with only a marginally significant relationship found for the smaller subsample (n = 36) of 10- to 12-year-old children (r = −0.28, p = 0.10). Overall, these findings indicate that as symptoms decrease, a higher level of confidence in exerting control over recovery is reported.
PACE-SE Scale Sensitivity to Change
p < 0.001.
n = 199.
Earlier analyses noted differences between males and females in their ratings of SE at V1 on the Managing My Stress and Managing My Activity scales. As a result, differences between males and females in the PACE-SE sensitivity to change were investigated from the first to second visit. No differences were found in the rate of change on three of the four scales-Managing My Activity, Seeking Adult Assistance, Maintaining Positive Outlook-between males and females, as they both improved at the same rate. A small but significant difference was evident, however, between males and females in the rate of change on the Managing My Stress scale (Time × Sex interaction, p < 0.04, partial eta2= 0.021) with a faster rate of change in females. Further examination of the data indicates that girls start with a lower degree of confidence on this scale at V1 but improve significantly to approach the level of confidence of males at V2. Patients with histories of anxiety also exhibited differences in their SE ratings at the V1 prompting the question as to whether they differ in their rate of change over time from patients without an anxiety history. No difference was found in the rate of change in SE for children with anxiety over the two visits compared with children without anxiety, and further examination of this rate of change in SE by sex found no differences between males and females. That is, persons with anxiety, whether male or female, exhibited the same trajectories of improved confidence as those without anxiety. Taken together, the PACE-SE and its scales demonstrate evidence of sensitivity to change over time.
The predictive relations of the PACE-SE total and four-scale scores to a criterion, recovery status, were examined as a source of evidence contributing to the validity of interpretation of the PACE-SE scores. A binary logistic regression was performed to discover the effects of PACE-SE scores on the likelihood that participants have recovered. Sex and history of anxiety of the participant were also added to the model as possible predictors due to their univariate relation to SE in this study as well as their previously established association with recovery outcome. The overall logistic regression model for the four PACE-SE scales as predictors was significant (X2 = 87.6, p < 0.001), accounting for 29.5% of the variance in recovery status and correctly classifying 80% of the cases. Further examination revealed no significant relation of sex to the model prediction and only marginally significant contribution of anxiety diagnosis to the model (p < 0.10). Examination of the four scales revealed that the Managing My Activity scale (Wald = 6.1, p = 0.014) and Maintaining a Positive Outlook scale (Wald = 11.9, p = 0.001) were significant predictors of recovery, while the Manage my Stress scale exhibited a marginally significant relationship to recovery outcome (p = 0.10). These results indicate that the lower the scores on the PACE-SE scales, the less likely the participant was deemed recovered.
Discussion
Concussion can produce a variety of postinjury clinical profiles and trajectories of recovery with broad similarities yet unique individual differences in outcomes. A constant in this varying recovery process is the need for injured persons to participate actively in their treatment, requiring a positive, informed stance to engage carefully and constructively in their daily activities, while managing their symptoms. As clinicians, we recognized variability in patients’ and families’ understanding of injury factors and confidence in navigating their recovery, motivating us to explore this issue further. As with other medical conditions, we posited that the individual’s SE plays an influential role in the process of managing, and ultimately facilitating their recovery. That is, the patient must feel capable and confident in their ability to exert control over the challenges, problems, and obstacles that recovery from their concussion presents. To better understand the individual patient’s capabilities to cope with their injury and contribute productively to recovery activities, the PACE-SE scale was developed. This study provides psychometric evidence in support of the PACE-SE scale’s reliability and validity with respect to the measure’s interpretation. Early expert clinician input, review of the SE literature, and the 4-stage PACE model 18 provided the backdrop to the generation and piloting of the preliminary scale item content, with trimming from the initial 50 items to the current 17-item measure.
Analysis of the internal structure of the 17-item PACE-SE revealed four factors (scales) based on combined statistical and conceptual grounds—Managing My Stress, Managing My Activity, Seeking Adult Assistance, and Maintaining Positive Outlook. Evidence for reliability of the PACE-SE total score indicated excellent internal consistency and good item-scale membership for the four scales. Test–retest reliability coefficients with this sample of clinically recovering patients demonstrate reasonable stability over time in the context of additional variability in scores due to individual recovery trajectories that likely attenuated the coefficients. The self-selection of patients seen for a second visit, either because they were recovered or lost to follow-up for other reasons, probably further attenuated this effect. It was notable that Managing My Stress had the lowest test–retest coefficient, likely in part due to the direct benefit of psychoeducation provided during the treatment section of the initial visit.
Evidence is presented for the validity and accuracy in the interpretation of the PACE-SE total score and the four scales. The items were generated by a team of expert concussion clinicians providing an initial level of content validity with significant and meaningful relationships found between the PACE-SE total and subscale scores and symptom status as indicated with the PCSI-2 and ChEERS4 symptom measures. These relationships demonstrate predictable inverse associations reflecting lower confidence in SE for managing recovery at the first clinic visit with higher symptom load and exertional effects. Further, greater challenges with academic learning and performance on the CLASS-3 measure were similarly associated with lower SE. Although these correlations should not infer causation at the present time, the association is important to take note of in future treatment studies. The interpretation of the PACE-SE is clarified in that the scores are not correlated with age of the patient, number of prior concussions, acute injury factors, parent education or racial background. Scores also are not associated with ADHD or LD status but do demonstrate an association with anxiety and depression history. This latter relationship makes conceptual sense in demonstrating an inverse relationship between the psychological resiliency of positive SE and the emotional states of anxiety and depression. Clinically, these findings suggest the importance of taking the emotional history of the recovering patient into account during treatment as additional support may be necessary to promote SE during recovery. Previous work has demonstrated a relationship between SE and anxiety and depression. 20,27,28 Examining patient SE ratings across two clinic visits indicates a strong sensitivity of the measure to positive change across recovery, which was further associated with symptom improvement. Notably, the effect size of this change between clinic/treatment visits was substantially larger than the small effect associated with the passage of time since injury. Finally, the significant difference in SE ratings between recovered and nonrecovered patients provides further evidence that the PACE-SE is measuring improved confidence in recovery activities.
With the reported evidence in support of the reliability and valid interpretation of the PACE-SE scale scores, its clinical utility can be considered. In the service of preparing the patient for active participation in their treatment, we believe that, in addition to the standard examination of the neurological system, symptom domains, balance/vestibular/ocular-motor functions, and cognition, the concussion evaluation should also assess the patient’s SE for the recovery process. That is, the following questions should be asked “How confident are you in taking charge of your recovery? More specifically, are you confident you can manage your stress as it relates to your concussion’s effects? Do you have confidence that you can exert control over your daily activity schedule? Are you confident in accessing the assistance of parents and teachers as you navigate the demands of your day? And finally, how confident are you in maintaining a positive outlook related to the challenges of recovery?” The PACE-SE scales provide an avenue to assess these issues in the recovering patient, speaking to the issues of patient resilience and readiness to navigate the challenges, problems and obstacles posed by the concussive injury.
Contributing to patient SE, the PACE rehabilitation model seeks to teach the patient critical background knowledge of the injury and build confidence in taking on recovery activities gradually and one step at a time, starting with a reduced schedule of activities, and slowly increasing their activity level as their symptom recovery allows. The PACE-SE measure can provide a window into the patient’s current readiness to engage with confidence in their progressive rehabilitation plan. The robustness of the four-scale structure, good internal consistency of the scales, reasonable stability over time, and relationship to symptom status and academic performance suggest these SE dimensions are valid indicators of the person’s “beliefs about their capabilities to produce designated levels of performance that exercise influence over events that affect their lives.” 12 The PACE-SE measure illuminates the patient’s confidence to perform the paced activity management and stress management necessary to promote their concussion recovery with a positive outlook.
A promising finding in this study was the improved SE scores as the PACE recovery process moves forward. While the current study design does not allow us to draw a causal connection between the treatment process embedded in the PACE rehabilitation model and increased SE, this finding is suggestive and worthy of further investigation in a controlled treatment trial similar to that of Belanger. 17 Other findings such as the lower initial SE ratings of adolescent females relative to males and persons with anxiety and depressive disorders are worth investigating further as they may be important factors to consider in the clinical treatment process. This initial effect with adolescent females did not appear to be related to their anxiety status or to their level of symptoms, and thus requires further study. Bandura’s 12 statement that SE relates to perceived capabilities points to the importance of believing in one’s “recovery-related capabilities” to produce a positive outcome. Thus, the concussion evaluation and treatment process should consider the potential role of the patient’s SE related to their injury and their capabilities to influence their own recovery activities. The PACE-SE measures offer a measure to identify these issues and incorporate them into treatment.
While this work demonstrates evidence in support of appropriate reliability and validity, several limitations should be mentioned. First, as previously noted, it is not possible to make causative statements about the role of positive SE directly impacting concussion recovery improvements. While it is a logical conclusion that confidently engaging in pro-recovery activities, reducing one’s stress, asking for assistance from adults when needed, and keeping a positive view of recovery would proffer benefits to recovery, any true causal statements require a more controlled treatment study. Second, the PACE-SE measure was developed for late elementary, middle school, and high school age groups, begging the question as to the role of, and means to assess, SE in younger children <10 years of age. It would be reasonable to predict that such a relationship exists between SE and recovery, but further study is needed with developmentally sensitive methods. Thirdly, there is recognized method variance inherent in asking an individual child or parent to report on a variety of topics related to their injury and functioning. Finally, the current study included a reasonable sample of White and Black children; however, children of Hispanic background and other cultures were not significantly represented in this sample. The possible effects of linguistic and cultural differences on SE of concussion recovery efforts in general, and with the PACE-SE measure, should be further explored. Bandura’s model of SE and behavior change expressly discusses the important role of the environmental context, which would incorporate different cultures. Klassen 29 discusses important considerations as they examine differences between Asian, West, and Eastern European cultures with respect to SE beliefs. Finally, the psychometric analysis of the PACE-SE measure takes place within the context of clinical care and is specifically intended to influence the ratings on these scales. The stability of one’s ratings might, therefore, be affected, in part, by the clinical intervention and limiting the measure’s true stability.
Conclusion
The PACE-SE measure exhibits reasonably strong psychometric properties supporting its use as a measure to assess and monitor the child and adolescent’s SE during their recovery from concussion. In addition to providing clinically useful information to the clinician in their support of the concussed patient, the PACE-SE can also advance pediatric concussion research evidence, similar to studies by Belanger 17 with adults, by offering an outcome measure for quantifying an important aspect of postconcussion recovery in children that may relate to treatment response.
Transparency, rigor, and reproducibility summary
The analysis plan for this study was not formally preregistered as the data were collected as part of a Institutional Review Board (IRB)-approved de-identified clinical database. Data were collected in the course of the clinic visit. The analysis plan was not formally preregistered, but the team member with primary responsibility for the analysis (lead author) certifies that the psychometric analysis plan was prespecified. Data collection were performed by investigators who were aware of relevant participant characteristics. Data were collected with subjects in the active concussion recovery state. De-identified data from this study are not available in a public archive. De-identified data from this study will be made available (as allowable according to institutional IRB standards) by emailing the corresponding author. Analytic code used to conduct the analyses presented in this study is not available in a public repository. They may be available by emailing the corresponding author. The authors agree to provide the full content of the article on request by contacting the corresponding author.
Footnotes
Authors’ Contributions
G.A.G.: Conceptualization (lead), investigation (equal), writing—original draft (equal), formal analysis (equal), and writing—review and editing (equal). C.C.V.: Conceptualization (equal), writing—original draft (equal), investigation (equal), data curation (lead), formal analysis (equal), and writing—review and editing (equal). M.D.S.: Conceptualization (equal), data curation (equal), investigation (equal), writing—original draft (equal), formal analysis (equal), and writing—review and editing (equal). E.G.: Investigation (equal), formal analysis (equal), writing—original draft (equal), and writing—review and editing (equal). A.B.: Investigation (equal), formal analysis (equal), and writing—review and editing (equal). M.Z.: Investigation (equal) and writing—review and editing (equal).
Author Disclosure Statement
The authors have no disclosures.
Funding Information
The authors have no funding to disclose.
Supplementary Material
Supplementary Table S1
Appendix A. Progressive Activities of Controlled Exertion–Self-Efficacy (PACE-SE) Scale.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
