Performance of SLEDAI-2K to detect a clinically meaningful change in SLE disease activity: a 36–month prospective cohort study of 334 patients

Abstract

Objective

The objective of this paper is to evaluate the performance of the Systemic Lupus Erythematosus Disease Activity Index (SLEDAI-2K) in detecting clinically meaningful changes in SLE disease activity.

Methods

A longitudinal cohort study was conducted of 334 SLE patients during a 36-month follow-up. At each outpatient visit, disease activity was scored using the Physician Global Assessment (PGA) and SLEDAI-2K. Correlations between PGA and SLEDAI-2K were assessed. A clinically meaningful change in SLE disease activity was defined as a ΔPGA ≥ 0.3 points from baseline. Performance of SLEDAI-2K in detecting a clinically meaningful worsening or improvement was tested using receiver operating characteristic (ROC) analysis.

Results

Adjusted mean PGA and SLEDAI-2K scores presented a high correlation (rho = 0.824, p < 0.0005). In ROC analysis, a SLEDAI-2K variation presented an area under the curve (AUC) of 0.697 (95% confidence interval (CI) (0.628–0.766), p < 0.0005) to detect a clinically meaningful improvement, with a sensitivity of 28.8% for a SLEDAI-2K ≥ 4 reduction. The AUC to detect a clinically meaningful worsening was 0.877 (95% CI (0.822–0.932), p < 0.0005), with a sensitivity of 35.3%.

Conclusions

SLEDAI-2K has a limited ability to detect clinically meaningful changes in SLE disease activity, failing to identify almost two-thirds of cases judged as having a clinically meaningful improvement or worsening. There is a need for more sensitive SLE disease activity measures in clinical practice and research.

Keywords

Disease activity index systemic lupus erythematosus SLEDAI SRI

Introduction

The assessment of disease activity has a central role in clinical daily practice, research and clinical trials involving patients with systemic lupus erythematosus (SLE). Multiple disease monitoring instruments have been developed, but all present important limitations.

The SLE Disease Activity Index (SLEDAI), a global score index that includes the sum of 24 weighted clinical and laboratory variables, with its SLEDAI-2K and Safety of Estrogens in Lupus Erythematosus National Assessment (SELENA)-SLEDAI versions, is the most widely used SLE disease activity measure.^1–3 SLEDAI is used worldwide in daily clinical practice. Moreover, it is the core determinant of the SLE Responder Index (SRI4) used in clinical trials and also to define flares, remission and low disease activity states.^4–7

However, the inability to demonstrate clinically meaningful improvements in recent trials, despite a positive clinical impression of efficacy, has raised the debate about the performance of SLEDAI in detecting clinically meaningful changes in SLE disease activity.⁸ Each item of the SLEDAI is scored dichotomically, resulting in the attribution of the same numerical weight, irrespective of the severity of change observed. Additionally, potentially severe lupus manifestations, such as hemolytic anemia, pneumonitis or gastrointestinal activity are not scored in SLEDAI. These aspects can be expected to hinder its ability to identify partial, albeit clinically meaningful, improvement or worsening. Such a limitation, if proven, would question the application of this instrument not only in target-driven management of SLE but also in the clinical evaluation of new medications.

The aim of this study is to evaluate the performance of SLEDAI-2K in detecting a clinically meaningful change in SLE disease activity in a clinical practice setting.

Materials and methods

Study design

A longitudinal cohort study was conducted from January 2014 to December 2016, at a tertiary care lupus clinic. Consecutive patients fulfilling the American College of Rheumatology 1997 and/or the Systemic Lupus Erythematosus International Collaborating Clinics 2012 classification criteria were included.^9,10 This project adheres to the principles of Declaration of Helsinki, was approved by the local ethics committee, and informed consent was obtained from all patients.

Data collection

At each outpatient visit, disease activity during the previous 30 days was scored according to the Physician Global Assessment (PGA) (0–3 points, on a 10-cm scale) and SLEDAI-2K (0–105), by the same senior rheumatologist (L.S.I.), with extensive experience in SLE management. PGA and SLEDAI-2K were scored blinded for the previous assessments. A difference in PGA ≥ 0.3 points at follow-up visits compared with study baseline was considered a clinically meaningful change in SLE disease activity, both for improvement and worsening.⁴

Statistical analysis

Descriptive analysis was performed for demographic and disease activity data at baseline. Correlation between PGA and SLEDAI-2K values was assessed through Spearman's test. SLEDAI-2K– and PGA-adjusted means were determined as the area under the curve (AUC) of its serial values at each visit, divided by the total time length.¹¹ We also assessed the Spearman's correlation between changes from baseline, observed in SLEDAI-2K and in PGA, at the visits with a clinically meaningful worsening or improvement.

Performance of SLEDAI-2K in predicting clinically meaningful changes in SLE disease activity was assessed through receiver operating characteristic (ROC) curve analysis. For patients with a clinically meaningful change, differences in SLEDAI-2K were calculated from baseline to the first visit at which this change occurred. For patients with clinically stable disease according to PGA over the entire follow-up, the SLEDAI-2K variation between baseline and last visit was used. Sensitivity, specificity, positive and negative predictive values (PPV, NPV) of cut-off variations in SLEDAI-2K ≥ 1 and ≥4 points in identifying a clinically meaningful change in SLE disease activity were calculated. Statistical significance was set at p < 0.05. Analyses were performed with IBM SPSS Statistics, V.24, and with R, V.3.1.2.¹²

Results

Demographic and clinical data at baseline

We included 334 patients (87.1% female, mean age of 44.8 ± 14.5 years, mean disease duration of 10.6 ± 8.8 years). During the 36-month follow-up, the included patients had at least two visits per year, no more than six months apart, the median time follow-up was 29 months (range, 4–36 months), and 2129 visits were performed. Table 1 shows baseline characteristics of the included patients.

Table 1

Baseline characteristics of the patients included

Baseline characteristics (n = 332)
Gender (female), n (%)	291 (87.1)
Age (years), mean ± SD	44.8 ± 14.5
Duration of SLE (years), mean ± SD	10.6 ± 8.8
PGA (0-3), mean ± SD	0.3 ± 0.4
SLEDAI-2K (0-105), mean ± SD	3.3 ± 2.9
Active lupus manifestations
Arthritis, n (%)	22 (6.6)
Mucocutaneous activity, n (%)	32 (9.6)
Serositis, n (%)	1 (0.3)
Lupus nephritis, n (%)	21 (6.3)
Hematological, n (%)	34 (10.2)
Immunological, n (%)	268 (80.2)
Neuropsychiatric, n (%)	3 (0.9)
Vasculitis, n (%)	1 (0.3)
Lupus treatment
Antimalarials, n (%)	311 (93.1)
Azathioprine, n (%)	53 (15.9)
Cyclosporine A, n (%)	9 (2.7)
Methotrexate, n (%)	19 (5.7)
Mycophenolate, n (%)	28 (8.4)
Cyclophosphamide, n (%)	5 (1.5)
Belimumab, n (%)	8 (2.4)
Rituximab, n (%)	3 (0.9)
Prednisone dose, mean ± SD	3.5 ± 7.9

SLEDAI-2K: Systemic Lupus Erythematosus Disease Activity Index 2000; PGA: Physician Global Assessment.

Correlation between PGA and SLEDAI-2K

At baseline, PGA and SLEDAI-2K scores were strongly correlated (rho = 0.766, p < 0.0005). The adjusted means of PGA and SLEDAI-2K over the 36-month follow-up were also strongly correlated (rho = 0.824, p < 0.0005). During follow-up, 80 and 51 patients had a clinically meaningful improvement and worsening, respectively, based on a difference in PGA ≥ 0.3 points in follow-up visits compared to study baseline. The clinically meaningful improvement and worsening occurred at a mean time of follow-up of 6.9 ± 7.3 and 14.3 ± 10.7 months, respectively. The disease activity manifestations of the patients with a clinically meaningful change, at baseline and at the clinically meaningful change visit, can be found in Supplementary Tables 1 and 2. The correlation of the variation in SLEDAI-2K and PGA between the baseline and the visit with a clinically meaningful improvement and worsening was 0.398 (p < 0.0005) and 0.463 (p < 0.0001), respectively.

Performance of change in SLEDAI-2K

The ROC curve analysis of the performance of SLEDAI-2K in identifying clinically meaningful changes in disease activity revealed an AUC of 0.697 (95% confidence interval (CI) = 0.628–0.766, p < 0.0005) for improvement, and of 0.877 (95% CI = 0.822–0.932, p < 0.0005) for worsening (Figure 1). Estimated sensitivities, specificities, PPV and NPV are presented in Table 2 and Supplementary Tables 3 and 4, using two different cut-offs of change in SLEDAI-2K: ≥1 and ≥4 points.

Figure 1

ROC Curve analysis of the performance of the SLEDAI-2K to detect a clinically meaningful improvement (a) and worsening (b).

Table 2

Performance of SLEDAI-2K to detect a clinically meaningful change in PGA

	Δ SLEDAI-2K ≥ 1				Δ SLEDAI-2K ≥ 4
	Sens	Spec	PPV	NPV	Sens	Spec	PPV	NPV
Improvement in PGA ≥ -0.3	70.0	57.1	39.7	82.5	28.8	92.9	62.2	76.3
Worsening in PGA ≥ +0.3	72.5	90.3	62.7	93.6	35.3	99.6	94.7	87.3

Sens: Sensitivity (%); Spec: Specificity (%); PPV: Positive predictive value (%); NPV: Negative predictive value (%).

Discussion

We found that SLEDAI-2K scores and their changes over time are generally in good agreement with the clinical estimation of disease activity, as described by PGA. This is in agreement with previous reports.^13,14 However, the performance of SLEDAI-2K in identifying cases of significant improvement or aggravation is inadequate for use in clinical practice and research. In fact, a variation in SLEDAI-2K ≥ 4 presents low sensitivity in identifying clinically meaningful changes in disease activity, failing to capture most events of either improvement or worsening. As expected, sensitivity is increased by using a threshold of one point difference, but with an unacceptable decrease in specificity and, especially, PPV.

Previous studies on the performance of SLEDAI have yielded heterogeneous results, depending on the analytical approach, the cut-offs of change and the comparator used for defining clinically meaningful change.^13,15–19 To define meaningful change in SLEDAI and PGA, we used the cut-offs adopted in SRI4 (4 and 0.3 points, respectively), an outcome measure that has been extensively used as a primary endpoint in SLE clinical trials.⁴ Although a variation in PGA ≥ 0.3 is not a validated cut-off, it is considered to be a clinically meaningful change in SLE disease activity, not only for worsening (as it is incorporated in the SRI), but also for improvement.^4,20,21 Importantly, a detailed analysis of the lupus disease activity items in the patients that in our study were identified by this PGA-based approach as presenting an improvement or worsening showed that in all cases there was a corresponding change in objective activity items (Supplementary Tables 1 and 2). Moreover, recently our group published a new SLE-Disease Activity Score (SLE-DAS) and used a PGA variation ≥0.3 points to identify the optimal discriminative variation of SLE-DAS to detect a clinically meaningful change.²¹ That SLE-DAS cut-off presented high performance in identifying clinically significant changes in disease activity in an external validation cohort. An alternative approach to identify a clinically meaningful change could be based on treatment change (stop/decrease and start/increase) compared with a previous visit. However, this approach is not without limitations, because changes in treatment also reflect the treating physician’s disease activity judgement, similarly to the PGA. Furthermore, recommended treatment changes do not always reflect an actual disease activity variation as it happens, in the lupus nephritis induction regimen, with the corticosteroid standard tapering protocol, usually started before improvement.²²

The PGA is an inherently subjective measure, but not to the point of being discretionary. Our results support its validity by demonstrating a high correlation between SLEDAI-2K and PGA at each visit, as well as between the mean time-adjusted SLEDAI-2K and PGA. The fact that both were scored always by the same experienced rheumatologist may have added to this consistency. The use of the PGA to assess lupus disease activity in research has received renewed interest and support, not only with its inclusion in SRI4, but also with the development of the Lupus Foundation of America-Rapid Evaluation of Activity, an index based on the assessment of a PGA for each organ system.²³ The SRI4 has been applied in some clinical trials using the SELENA-SLEDAI version, while in others the SLEDAI-2K was included; the major difference is that SLEDAI-2K always scores proteinuria >500 mg per day because of active lupus nephritis, while SELENA-SLEDAI does not score persistent proteinuria, resulting in an inappropriate four-point reduction in cases with persistently active lupus nephritis.

The limited sensitivity of SLEDAI to change seems to be an unavoidable consequence of scoring each disease activity item dichotomically, thus ignoring the severity of abnormalities. For example, thrombocytopenia of 90,000 or 5000/mm³ is equally scored (one point); lupus arthritis with 2 or 20 active joints is equally scored (four points); and the reduction of proteinuria due to lupus nephritis from 2500 mg per day to 600 mg per day is not reflected in SLEDAI-2K (four points). This obviously hinders the use of SLEDAI to monitor and guide treatment in the individual patient, as well as its validity as a tool to evaluate the efficacy of novel treatment agents or strategies.

The results of our study have major implications in the interpretation of clinical trials applying SRI4 as the primary endpoint. SRI4 defines efficacy as an improvement of ≥4 points in SLEDAI from baseline. The other components (British Isles Lupus Assessment Group and PGA) are used only to validate a patient as a responder, as they are required only to not worsen significantly, in an attempt to increase specificity. Given the low sensitivity of a change in SLEDAI-2K ≥4 to detect clinically meaningful change, it is likely that treatment effects are largely underestimated by SRI4.

The inability of SLEDAI-2K to capture partial responses in disease activity led to the development of the SLEDAI-2K Responder Index 50 (SRI-50), which can capture a ≥50% improvement in disease among lupus patients.²⁴ Its construct validity, reliability and ability to identify responders have been shown in several studies.^24–28 However, the SRI-50 cannot identify worsening of disease activity.

Limitations of our study include its monocentric setting. The use of the PGA as the gold standard for clinically meaningful change in disease activity may be exposed to criticism, but the absence of a more objective alternative, its strong correlation with SLEDAI and its wide acceptance by the clinical and research communities support its use.

The prospective assessment of data at each clinical visit with consistently scored SLEDAI-2K and PGA, the large cohort with long duration of follow-up, and the real-life setting are important strengths of our study. To the best of our knowledge, this study is the first evaluating the performance of SLEDAI in a real-life clinical setting.

In conclusion, we demonstrated that SLEDAI-2K has a low sensitivity in identifying clinically meaningful changes in SLE disease activity. This is a major limitation both in clinical trials and daily practice. Research is needed to develop SLE disease activity measures with improved sensitivity to change.

Supplemental Material

Supplemental material for Performance of SLEDAI-2K to detect a clinically meaningful change in SLE disease activity: a 36–month prospective cohort study of 334 patients

Supplemental Material for Performance of SLEDAI-2K to detect a clinically meaningful change in SLE disease activity: a 36–month prospective cohort study of 334 patients by D Jesus, M Rodrigues, A Matos, C Henriques, J A Pereira da Silva and L S Inês in Lupus

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Bombardier

Gladman

Urowitz

Caron

Chang

. Derivation of the SLEDAI. A disease activity index for lupus patients. The Committee on Prognosis Studies in SLE. Arthritis Rheum 1992; 35: 630–640.

Gladman

Ibañez

Urowitz

. Systemic Lupus Erythematosus Disease Activity Index 2000. J Rheumatol 2002; 29: 288–291.

Petri

Kim

Kalunian

et al.

Combined oral contraceptives in women with systemic lupus erythematosus. N Engl J Med 2005; 353: 2550–2558.

Furie

Petri

Wallace

et al.

Novel evidence-based Systemic Lupus Erythematosus Responder Index. Arthritis Rheum 2009; 61: 1143–1151.

van Vollenhoven

Voskuyl

Bertsias

et al.

A framework for remission in SLE: Consensus findings from a large international task force on definitions of remission in SLE (DORIS). Ann Rheum Dis 2017; 76: 554–561.

Franklyn

Lau

Navarra

et al.

Definition and initial validation of a Lupus Low Disease Activity State (LLDAS). Ann Rheum Dis 2016; 75: 1615–1621.

Zen

Iaccarino

Gatto

et al.

Prolonged remission in Caucasian patients with SLE: Prevalence and outcomes. Ann Rheum Dis 2015; 74: 2117–2122.

Kalunian

Urowitz

Isenberg

et al.

Clinical trial parameters that influence outcomes in lupus trials that use the Systemic Lupus Erythematosus Responder Index. Rheumatology (Oxford) 2018; 57: 125–133.

Hochberg

. Updating the American College of Rheumatology revised criteria for the classification of systemic lupus erythematosus. Arthritis Rheum 1997; 40: 1725–1725.

10.

Petri

Orbai

Alarcón

et al.

Derivation and validation of the Systemic Lupus International Collaborating Clinics classification criteria for systemic lupus erythematosus. Arthritis Rheum 2012; 64: 2677–2686.

11.

Ibañez

Urowitz

Gladman

. Summarizing disease features over time: I. Adjusted mean SLEDAI derivation and application to an index of disease activity in lupus. J Rheumatol 2003; 30: 1977–1982.

12.

R Development Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2010.

13.

Fatemi

Raeisi

Sayedbonakdar

Smiley

. Sensitivity analyses of four systemic lupus erythematosus disease activity indices in predicting the treatment changes in consecutive visits: A longitudinal study. Clin Rheumatol 2018; 37: 955–962.

14.

Uribe

Vilá

McGwin

Jr Sanchez

Reveille

Alarcón

. The Systemic Lupus Activity Measure-revised, the Mexican Systemic Lupus Erythematosus Disease Activity Index (SLEDAI), and a modified SLEDAI-2K are adequate instruments to measure disease activity in systemic lupus erythematosus. J Rheumatol 2004; 31: 1934–1940.

15.

Ward

Marx

Barry

. Comparison of the validity and sensitivity to change of 5 activity indices in systemic lupus erythematosus. J Rheumatol 2000; 27: 664–670.

16.

Gladman

Goldsmith

Urowitz

et al.

Sensitivity to change of 3 systemic lupus erythematosus disease activity indices: International validation. J Rheumatol 1994; 21: 1468–1471.

17.

Yee

Isenberg

Prabu

et al.

BILAG-2004 index captures systemic lupus erythematosus disease activity better than SLEDAI-2000. Ann Rheum Dis 2008; 67: 873–876.

18.

Chang

Abrahamowicz

Ferland

Fortin

. CaNIOS Investigators. Comparison of the responsiveness of lupus disease activity measures to changes in systemic lupus erythematosus activity relevant to patients and physicians. J Clin Epidemiol 2002; 55: 488–497.

19.

Fortin

Abrahamowicz

Clarke

et al.

Do lupus disease activity measures detect clinically important change?

J Rheumatol 2000; 27: 1421–1428.

20.

Devilliers

Bonithon-Kopp

Jolly

. The Lupus Impact Tracker is responsive to changes in clinical activity measured by the Systemic Lupus Erythematosus Responder Index. Lupus 2017; 26: 396–402.

21.

Jesus

Matos

Henriques

et al.

Derivation and validation of the SLE Disease Activity Score (SLE-DAS): A new SLE continuous measure with high sensitivity for changes in disease activity. Ann Rheum Dis 2019; 78: 365–371.

22.

Hahn

McMahon

Wilkinson

et al.

American College of Rheumatology guidelines for screening, treatment, and management of lupus nephritis. Arthritis Care Res (Hoboken) 2012; 64: 797–808.

23.

Askanase

Pong

et al.

Preliminary test of the LFA rapid evaluation of activity in lupus (LFA-REAL): An efficient outcome measure correlates with validated instruments. Lupus Sci Med 2015; 2: e000075–e000075.

24.

Touma

Gladman

Ibañez

Urowitz

. Development and initial validation of the Systemic Lupus Erythematosus Disease Activity Index 2000 Responder Index 50. J Rheumatol 2011; 38: 275–284.

25.

Touma

Gladman

Ibañez

Taghavi-Zadeh

Urowitz

. Systemic Lupus Erythematosus Disease Activity Index 2000 Responder Index-50 enhances the ability of SLE Responder Index to identify responders in clinical trials. J Rheumatol 2011; 38: 2395–2399.

26.

Touma

Gladman

Ibañez

Urowitz

. SLEDAI-2K Responder Index 50 captures 50% improvement in disease activity over 10 years. Lupus 2012; 21: 1305–1311.

27.

Touma

Urowitz

Fortin

et al.

Systemic Lupus Erythematosus Disease Activity Index 2000 Responder Index-50: A reliable index for measuring improvement in disease activity. J Rheumatol 2011; 38: 868–873.

28.

Touma

Urowitz

Taghavi-Zadeh

Ibañez

Gladman

. Systemic Lupus Erythematosus Disease Activity Index 2000 Responder Index 50: Sensitivity to response at 6 and 12 months. Rheumatology (Oxford) 2012; 51: 1814–1819.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.12 MB