Assessment of skin disease in scleroderma: Practices and opinions of investigators studying scleroderma

Introduction

The modified Rodnan skin score (MRSS) is a feasible, reliable, and valid outcome measure incorporated for assessing skin thickness in systemic sclerosis (SSc, scleroderma), and in early disease, correlates with mortality and internal organ disease involvement.^1–4 The MRSS is commonly used as a primary or key secondary outcome measure in clinical trials in SSc. ⁵ However, the MRSS is limited by inter-rater variability.^2,6 To address variability and improve technique, formal MRSS training has been proposed for investigators prior to participating in clinical trials for SSc. ² However, it is unknown how often physicians caring for patients with SSc perform the MRSS in clinical practice or what technical approach to performing the MRSS is most often used. Specifically, three approaches have been previously described: “maximizing,” “averaging,” or recording the score “most representative” for each MRSS site. ² Using the “maximizing” approach, the examiner scores each site according to the most severe local involvement. Using the “averaging” approach, the examiner assigns a score based upon the average of the local skin involvement. Using “most representative” approach, the examiner scores each site by the score that best represents each individual area. The purpose of this study was to assess MRSS training, perceptions, and practices among individuals studying scleroderma.

Methods

An invitation with an online survey link was delivered electronically to 282 individuals within the Scleroderma Clinical Trials Consortium (SCTC). The SCTC is an international organization that includes clinicians and researchers with expertise in SSc. Participating SCTC centers are located in North and South America, Europe, Asia, and Australia. The 46-item anonymous survey (available in Supplemental Material) included three sections: participant demographics, MRSS background and training, and MRSS assessment practices. The survey was accessible for 5 weeks between October and November 2019. A reminder email was sent 1 week prior to the survey closing date. Response frequencies and descriptive data were analyzed. Comparisons were made using chi-square test, and p value of <0.05 was considered significant. Appropriate ethical approval was obtained from the Hospital for Special Surgery Institutional Review Board (Study Number 2019-1711).

Results

Of 282 SCTC members who received the survey invitation, 116 (41%) participated. The most common participant role was physician (95%), followed by research assistant (3%), nurse (< 1%), and physician assistant (<1%; Table 1). The majority of participants exclusively care for adult patients (77%), practice at a scleroderma center (93%) within an academic institution (90%), and are located in North America (41%) or Europe (40%). A total of 49% of participants have practiced for >20 years, and 44% evaluate 11–20 patients with SSc weekly. The vast majority of participants have performed the MRSS at least one time in any clinical setting (>99%). Regarding importance of the MRSS in caring for patients with SSc, 43% of participants feel that the MRSS is “somewhat important,” and 56% of participants feel that the MRSS is “essential.”

OPEN IN VIEWER

Table 1.

Survey participant demographics.

Characteristic	Total (N = 116)
Sex, female	59 (51%)
Age (years)
≤40	27 (23%)
41–60	66 (57%)
>60	23 (20%)
Geographical region
North America	47 (41%)
Europe	46 (40%)
Asia	9 (8%)
South America	6 (5%)
Australia/Oceania	6 (5%)
Middle East/North Africa	1 (<1%)
Central America/Caribbean	1 (<1%)
Years in practice
1–10	29 (25%)
11–20	30 (26%)
21–30	31 (27%)
>30	26 (22%)
Number of patients with SSc seen weekly
0–10	41 (35%)
11–20	44 (38%)
21–30	17 (15%)
>30	14 (12%)
Physician	110 (95%)
Academic center	104 (90%)
Scleroderma center	108 (93%)
Participates in clinical trials	97 (84%)
Adults-only practice	90 (77%)
Performs the MRSS	115 (99%)

MRSS: modified Rodnan skin score; SSc: systemic sclerosis.

Most participants (91%) reported previous MRSS training; 72% of those with previous MRSS training were trained in the context of a clinical trial. Outside of clinical trials, training was received through the SCTC, American College of Rheumatology, European League Against Rheumatism, or European Scleroderma Trials and Research Group workshops, and/or rheumatology fellowship/mentor teaching. Of participants who have not received MRSS training, 60% were interested in receiving training to perform the MRSS, 10% felt that the MRSS is “not important for care, but is interesting to assess,” 50% felt that the MRSS is “somewhat important for care,” and 40% felt that the MRSS is “essential for care.” Among those who received training, the majority (58%) felt that the MRSS is “essential for care.”

In the full study cohort, 39% of participants felt either “uncomfortable” or only “somewhat comfortable” performing the MRSS. Among the 76 participants who received training in the context of a clinical trial, 22 (29%) feel “uncomfortable” or only “somewhat comfortable” performing the MRSS (Supplemental Figure 1). Among the 38 participants who attended ⩾3 training sessions in the context of a clinical trial, 8 (21%) feel “uncomfortable” or only “somewhat comfortable.” There were no significant differences in the proportion of participants who feel either “uncomfortable” or only “somewhat comfortable” (vs “very comfortable”) among participants ⩽40 versus >40 years old (40% vs 36%, p = 0.26), participants who see ⩽10 versus >10 patients with SSc weekly (48% vs 33%, p = 0.10), or participants with ⩽20 versus >20 years in practice (41% vs 37%, p = 0.67).

The practice pattern regarding frequency of performing the MRSS varied by disease subtype (diffuse vs limited cutaneous SSc). More participants perform the MRSS at least one time for patients with diffuse cutaneous SSc (95%) compared to limited cutaneous SSc (88%). Fifty percent of participants perform the MRSS for patients with diffuse cutaneous SSc at every visit. In contrast, for patients with limited cutaneous SSc, the most common practice was to perform the MRSS initially, then periodically (43%).

The technical method for measuring each skin site in the MRSS varied among participants: 44% used “maximizing,” 28% used “averaging,” and 18% used “most representative.” No other MRSS approach was used by participants. There was similar variability when approach was stratified by the two most common geographical regions, the number of MRSS training sessions, and participant comfort level with performing the MRSS (Figure 1). “Maximizing” remained the most common approach when stratifying by comfort level; however, those who have received ⩾3 training sessions in the context of clinical trials most often used the “averaging” approach (47%). Among nine individuals who perform the MRSS but have not received formal training, one-third (3) used “averaging,” one-third (3) used “maximizing,” and one-third (3) used the “most representative” approach.

OPEN IN VIEWER

Figure 1.

Modified Rodnan skin score (MRSS) technical approach. The technical approach to the MRSS, stratified by (a) geographical region, (b) comfort level in performing the MRSS, and (c) number of MRSS training sessions received in the context of a clinical trial (among 76 participants who had received training in the context of clinical trial). Maximum indicates “maximizing,” average indicates “averaging,” and representative indicates “most representative” approach).

Participants with greater experience (>20 vs ⩽ 20 years in practice) were more likely to have received MRSS training in the context of a clinical trial (79% vs 53%, respectively, p < 0.001) (Supplemental Table 1). However, there were no significant differences in the MRSS approach, comfort level, or practice pattern according to years of practice.

The most common additional skin assessment methods utilized by participants were physician global assessment of disease severity (31%) and patient global assessment (30%). No other approach to assessment of SSc skin was commonly reported by participants (Table 2).

OPEN IN VIEWER

Table 2.

Spectrum of skin assessment methods for systemic sclerosis among 116 participants in the Scleroderma Clinical Trials Consortium.

Scleroderma skin assessment methods	Total (N = 116)
Modified Rodnan skin score a	115 (99%)
Physician global assessment a	36 (31%)
Patient global assessment a	35 (30%)
Patient assessment of pruritus	8 (6.9%)
Skin ultrasound a	6 (5.2%)
Health Assessment Questionnaire Disability Index	4 (3.4%)
Oral aperture assessment	4 (3.4%)
Edema assessment	4 (3.4%)
Patient assessment of change compared to last visit	3 (2.6%)
Patient assessment of pigment change	3 (2.6%)
Telangiectasia assessment	2 (1.7%)
Tendon friction rub count	2 (1.7%)
Patient assessment of hot skin	1 (<1%)
Finger palm distance	1 (<1%)
Assessment of ulcers	1 (<1%)
Assessment of calcinosis	1 (<1%)
Assessment of skin fold changes	1 (<1%)
Assessment of dryness	1 (<1%)
Assessment of photosensitivity	1 (<1%)

a

Methods that were assessed using a direct survey question. All other responses/methods were written-in by participants.

Discussion

For many years, the MRSS has been the primary means by which skin disease in SSc is quantified in clinical practice and, especially, in clinical trials. Despite concerns raised about the ability of the use of MRSS to demonstrate change in clinical trials, ⁷ the results of this study demonstrate that the MRSS remains highly endorsed by investigators as the primary method for assessing skin involvement in patients with SSc.

This study documented significant variation among scleroderma experts in the technical approach to performing the MRSS with none of the three approaches (“maximizing,” “averaging,” or “most representative”) endorsed by >50% of investigators. Participants most commonly used the “maximizing” approach. Although there is a theoretical concern that “maximizing” may limit MRSS responsiveness to change, ² a recent analysis from a pilot study showed that all three methods (“maximizing,” “averaging,” or “most representative”) had similar responsiveness to change at 6 and 12 months. ⁸ The SCTC and the World Scleroderma Foundation training programs specifically address these issues by using one method to assess MRSS in individual clinical trials and ensuring that the same observer measures MRSS longitudinally for individual patients. ² The approach to MRSS taught commonly in clinical trial training sessions is “averaging.” The current study demonstrates that individuals who received ⩾3 training sessions in the context of a clinical trial most commonly use the “averaging” approach, while those who received <3 training sessions most commonly use the “maximizing” approach. However, participants were not asked to differentiate their approach between trials and clinical practice. Among nine individuals who perform the MRSS but have not received formal training, there was an equal distribution between the three approaches. These findings suggest that training may impact one’s approach. A unified approach taught consistently across training sessions and used for all clinical trials may improve reliability and investigator comfort with performing the MRSS.

The majority of those who have not previously attended a training session for performing the MRSS were interested in receiving training. Feeling “uncomfortable” or only “somewhat comfortable” was not isolated to investigators with less experience (⩽40 years old, see ⩽10 patients with SSc weekly, and/or with ⩽20 years in practice). One potential explanation for this finding relates to the intrinsic technical challenges and inter-rater variability of the MRSS which may prevent even expert scleroderma researchers from feeling “very comfortable” performing the MRSS. Nevertheless, these results support ongoing efforts to increase opportunities for MRSS training sessions, even among experienced SSc investigators.

Although the MRSS is by far the most commonly used method of assessing skin disease in SSc (>99%), study participants also reported using additional methods to assess skin in patients with SSc, including physician global assessment (31%), patient global assessment (30%), pruritis assessment (7%), skin ultrasound (5%), the Health Assessment Questionnaire Disability Index (HAQ-DI) (3%), among others. However, no other method was used by a significant number of investigators. This reinforces the continued importance of the MRSS for assessing skin in SSc.

Among this study’s strengths are that the number of participants is large for a study of SSc and the geographic and experience diversity adds to the generalizability of the results. This study also has limitations to consider. First, all participants were members of the SCTC (an organization of individuals with expertise in SSc care and research) which limits generalizability. However, it is for these individuals that the MRSS is most relevant. Nonetheless, interest in MRSS training and patterns of MRSS use would likely vary between the study participants and physicians who see fewer patients with SSc and who do not participate in clinical trials in SSc. Also, despite the significant number of participants, response rate was 41% and may not represent the opinions and practices of the full target population of SSc investigators. Individuals who did not reply may have received less training in performing the MRSS, be less active in clinical trials in SSc, and/or use varying approaches to performing the MRSS. Thus, these results may have some limited generalizability. In addition, a history of video training or other non-in-person training methods was not assessed.

In conclusion, these results indicate continued strong support for using MRSS for care and research in SSc and an interest in MRSS training. These results also highlight a potential need to harmonize the technical approach to performing the MRSS. Future work in multiple centers evaluating the responsiveness to change and reliability of the three technical approaches may help to identify a unified method for performing MRSS in clinical trial and patient care settings.

Supplemental Material

Supplemental Material

Supplemental Material