Patient-reported outcome measures in systemic lupus erythematosus by a web-based application: A randomized,crossover,agreement study

Abstract

Objectives

Patient-reported outcome measures (PROMs) are evaluated in randomized controlled trials (RCTs) in patients with systemic lupus erythematosus (SLE), but not widely used in clinical practice. However, interest in incorporating PROMs into the management of SLE is increasing as PROMs provide a unique insight into the patient’s perception of lupus disease activity. The objective was to assess agreement in PROMs answered using a web app versus an outpatient touchscreen among patients with SLE.

Methods

In a crossover RCT, SLE patients answered the following PROMs in a random order using the web app and the outpatient touchscreen: Systemic Lupus Erythematosus Activity Questionnaire (SLAQ) Global Health, SLAQ Symptom, SLAQ Total, SLAQ Worsening, Pain Visual Analog Scale (VAS), Fatigue VAS, Patient Global Health VAS, Health Assessment Questionnaire Disability Index (HAQ-DI), Patient Acceptable Symptom State (PASS), and an Anchoring Question. Equivalence between the two device types was demonstrated if the 95% confidence interval (95% CI) of the difference in PROM scores was within the prespecified equivalence margin. Agreement between the two device types was assessed using mixed linear models.

Results

Thirty-four patients with SLE were included. Equivalence was demonstrated between the two device types for SLAQ Global Health with a difference of −0.21 (95% CI: −0.65 to 0.23). Moreover, equivalence was also found for HAQ-DI, Pain VAS, and Fatigue VAS whereas only comparability within the limits of the Minimal Clinically Important Difference (MCID) was demonstrated for VAS Patient Global Health. Statistical comparability was demonstrated for SLAQ Total, SLAQ Worsening, PASS, and Anchoring Question (no predefined MCID/equivalence margins available). However, a statistically significant difference between device types was observed for the SLAQ Symptom of −0.56 (95% CI: −1.10 to −0.01). The difference was, however, very small when considering the scale range of 0–24; thus, it was not judged to be of clinical relevance. Preference for the web app was very high (91.2%).

Conclusion

For the first time ever, equivalence and comparability between two electronic device types for various PROMs were demonstrated among patients with SLE. Implementation of the device is expected to improve the management of SLE.

Keywords

Systemic lupus erythematosus patient-reported outcome measures smartphone application minimally clinically important difference equivalence crossover study

Trial registration: Clinicaltrials.gov, NCT04411407, https://clinicaltrials.gov/ct2/show/NCT04411407?cond=SLE&cntry=DK&city=Aalborg&draw=2&rank=1.

Introduction

Systemic lupus erythematosus (SLE) is an autoimmune disease characterized by formation of autoantibodies and multisystem organ involvement with a wide range of specific and non-specific organ symptoms that vary with disease activity and among patients.^1,2

Patient-reported outcome measures (PROMs) are a pivotal tool to complement the physician’s evaluation and more completely reflect the patient’s perception of SLE symptoms. However, PROMs are mostly assessed in clinical trials and not frequently in clinical practice.³ Both disease-specific and generic PROMs have been evaluated and used in randomized controlled trials (RCTs) in SLE. ^4–7 Currently, there is no international consensus about which PROMs are most preferable or essential in the management of patients with SLE.

The Systemic Lupus Erythematosus Activity Questionnaire (SLAQ) ⁸ is one of the more frequently used PROMs in SLE as it captures the impact of global health and disease activity, both considered essential outcomes when assessing patients with SLE.⁹ The validated SLAQ questionnaire ^8,10 consists of 4 separate domains:

SLAQ Global Health: the patient’s evaluation of lupus disease activity on an NRS scale from 0 to 10.

SLAQ Symptom: the patient’s report of presence of lupus-specific symptoms, range 0–24.

SLAQ Total: the patient’s evaluation of severity of lupus disease activity by a weighted score of the 24 lupus-specific symptoms, range 0–44.

SLAQ Worsening: the patient’s evaluation of the presence/severity of a lupus flare on a transitional scale, 0 (no flare), 1 (mild flare), 2 (moderate flare), or 3 (severe flare).

As technology has evolved, the assessment of PROMs has progressed from data collection on paper forms to electronic data capturing with digital solutions.¹¹ Thus, the outpatient touchscreen used in Denmark has previously demonstrated agreement with paper forms among patients with various rheumatic diseases.^12,13 Electronic PROM data capture has many advantages over paper-based forms as secondary data entry errors are avoided, and more exact and complete data are captured with minimal administrative burden.¹¹ Thus, assessment of electronic PROM data captured on an outpatient touchscreen has been implemented in the management of various diseases worldwide during the past decades. However in recent years, interest in electronic reporting of PROMs through web applications (apps) has increased. It allows patients to access and respond to PROMs on their own device at their own convenience; thereby, possibly minimizing recall bias¹¹ and allowing reporting symptoms of flares as they occur. In addition, home access with an app or a website has the following advantages over an outpatient touchscreen: no queue, no hygiene problems, and discrete and comfortable data entry without disturbances.

Previously, the Danish Rheumatology Database (DANBIO) app as well as a web-based from home solution have proven to be comparable to the outpatient touchscreen among Danish patients with inflammatory arthritis.^14,15 However, to our knowledge PROM agreement between two electronic device types among patients with SLE has not yet been evaluated. Throughout this paper, comparability and agreement refers to absence of a clinically relevant difference between device types whereas equivalence refers to an insignificant difference between device types, that is, within the limits of half of the Minimal Clinically Important Difference (MCID). Thus, the objective of this trial was to explore if electronic registration of PROMs with SLAQ Global Health as a primary outcome measure through the DANBIO web app is comparable, or even equivalent, to the outpatient touchscreen among patients with SLE.

Methods

Patient and public involvement

A patient research partner (PRP) acknowledged the purpose of this trial, refined and approved the trial patient information material, and approved the trial protocol. However, the PRP was not involved in inclusion of participants nor in the conduct or analyses of the trial. Research personnel will disseminate trial results to participants after publication.

Study design and participants

This trial was designed as a randomized, within-participants, crossover, agreement trial conducted at Aalborg University Hospital, Denmark. Patients ≥18 years old diagnosed with SLE in accordance with the 2019 European Alliance of Associations for Rheumatology (EULAR)/American College of Rheumatology (ACR) SLE Classification Criteria or judged to have a defined SLE diagnosis by the rheumatologist were eligible for participating. Furthermore, experience with the PROM questionnaires in DANBIO (≥1 previous assessment) was required. Exclusion criteria included language barriers (PROMs were only available in Danish) and device barriers (participants with no access to smartphones or tablets).

Procedures

Eligible patients diagnosed with SLE were enrolled by a screening telephone contact from research personnel; thereafter, written informed consent was obtained. Participants were randomized in a 1:1 ratio to:

Group web app → touchscreen (WA → TS): participants completed PROMs on the DANBIO web app and after a “washout period,” the same PROMs were answered on the outpatient touchscreen.

Group touchscreen → web app (TS → WA): participants completed PROMs on the outpatient touchscreen and after a “washout period,” the same PROMs were answered on the DANBIO web app.

A “washout period” of 1 to 2 days between the two PROM completions was prespecified to minimize potential recall bias as a short time period (e.g., 5 min) increase the risk of a biased over-agreement. Furthermore, the prespecified “washout period” was judged to be short enough to ensure that no major change in disease activity would occur.

Participants accessed the PROM questionnaire on the outpatient touchscreen with their personal civil registration number as usual practice. To access the DANBIO web app, the participants had to log on with their personal identification number (NemID) on their own smartphone or tablet from home or wherever convenient.

Patient-reported outcome measures

The participants answered the following PROMs on the two device types:

SLAQ Global Health: described previously, MCID calculated to ±1.495 (as described in ‘power and sample size calculation’), equivalence margins: ±0.75.

SLAQ Symptom: described previously, MCID and equivalence margins not defined.

SLAQ Total: described previously, MCID and equivalence margins not defined.

SLAQ Worsening: described previously, MCID and equivalence margins not defined

Pain Visual Analog Scale (VAS): the patient’s evaluation of pain within the last week on a 100-mm horizontal scale, range 0–100. MCID: ± 10,¹⁶ equivalence margins: ±5.

Fatigue VAS: the patient’s evaluation of fatigue within the last week on a 100-mm horizontal scale, range 0–100. MCID: ± 10,¹⁶ equivalence margins: ± 5.

Patient Global Health VAS: the patient’s evaluation of disease activity within the last week on a 100-mm horizontal scale, range 0–100. MCID: ± 10,¹⁶ equivalence margins: ±5.

Health Assessment Questionnaire Disability Index (HAQ-DI): the patient’s evaluation of physical function, range 0–3. MCID: ± 0.22,¹⁶ equivalence margins: ± 0.11.

Patient Acceptable Symptom State (PASS): the patient’s evaluation of whether the current symptom state is acceptable, answered with “yes” or “no”. MCID and equivalence margins are not defined.

Anchoring Question: the patient’s evaluation of change in disease activity since last visit, range: −3 (much worse), −2 (worse), −1 (slightly worse), 0 (unchanged), 1 (slightly better), 2 (better), and 3 (much better). MCID and equivalence margins are not defined.

Objectives and outcomes

The primary objective was to assess if electronic registration of SLAQ Global Health by patients with SLE using the DANBIO web app was comparable, or even equivalent, to the outpatient touchscreen.

Secondary outcomes were comparatively assessed between the two device types for the following PROMs: SLAQ Symptom, SLAQ Total, SLAQ Worsening, Pain VAS, Fatigue VAS, Patient Global Health VAS, HAQ-DI, PASS, and an Anchoring Question. Finally, device preference was assessed.

Collected variables

The DANBIO database and the electronic medical journal were assessed to obtain the following baseline characteristics: sex, age, diagnostic criteria, disease duration, current SLE treatment, anti-nuclear antibodies (ANA), anti–double-stranded deoxyribonucleic acid (anti-dsDNA), C-reactive protein (CRP), most recent (mean 8.5 months) physician assessed Systemic Lupus Erythematosus Disease Activity Index (SLEDAI) score, and Systemic Lupus International Collaborating Clinics (SLICC) damage index. Furthermore, the most recent report of the following PROMs was obtained from the DANBIO database with the following to the word: as baseline characteristics: SLAQ scores, VAS scores, HAQ-DI, PASS, and Anchoring Question. However, these data were also included in the statistical analysis of primary and secondary outcomes as the mixed linear model was adjusted for PROM-level at inclusion, as described in the Statistical Analysis section below.

All study data were collected in a dedicated electronic case report form (e-CRF) in the Research Electronic Data Capture (REDCap) system hosted by the North Denmark region.^17,18

Power and sample size calculation

The sample size calculation was performed in accordance with the recommendations in the Difference Elicitation in TriAls (DELTA²) guideline.¹⁹ To our knowledge, no MCID for SLAQ Global Health is defined; however, in a recent paper by Katz et al., improvement/worsening in SLE disease activity on a scale from 0 (not active) to 10 (very active) was defined as ± 1.5 points.⁶ We estimated a similar minimal important difference for SLAQ Global Health from 1000 random SLAQ Global Health bootstrap samples from 44 patients with SLE from Aalborg University Hospital. The estimated median standard deviation (SD) was 2.99 (range: 2.25–3.62); thus, the MCID for SLAQ Global Health was: 0.5 × 2.99 = 1.495. Equivalence was defined as half of the effect that is considered a clinically relevant reduction, that is, half of the MCID; therefore, the equivalence margins for SLAQ Global Health was ± 0.75 (i.e., 0.5 × 1.495). However, for the sample size calculation, the more conservative estimate of the SLAQ Global Health equivalence margins that only allows minimal variance between two interventions was used, that is, based on the minimum value of the SD range of 2.25. This approach yields a conservative SLAQ Global Health equivalence margin of ± 0.56 (i.e., 0.5 × 2.25 × 0.5).

As specified in the Trial Protocol (Supplementary Table S1) and the Statistical Analysis Plan (SAP) (Supplementary Table S2), a two one-sided tests analysis for additive equivalence of paired means with bounds −0.56 and +0.56 for the mean difference in SLAQ Global Health with a common standard deviation of 3.0 SLAQ Global Health points and correlation 0.95 between measures (i.e., a high degree of anticipated correlation between the two device types), a sample size of 33 patients was required to obtain a power of 90%. Thus, it was decided to aim for inclusion of 34 patients in total, that is, 17 patients in each intervention group.

Randomization: Allocation concealment and implementation

Senior biostatistician Robin Christensen, with no clinical involvement in the trial, made the computer-generated randomization sequence in SAS PROC PLAN before the start of the inclusion period. The randomization sequence used permuted blocks and an allocation ratio of 1:1 to distribute participants in one of the two intervention groups. Thereafter, the independent data manager Johanne Hovgaard Winther (JHW) entered the randomization sequence into the dedicated e-CRF in REDCap.

Statistical analyses

Analyses of trial data were done in accordance with the prespecified SAP, the Consolidated Standards of Reporting Trials (CONSORT) statements, ^20,21 and the Enhancing the QuAlity and Transparency Of health Research (EQUATOR) network recommendation.²² Baseline characteristics were summarized as number and percentages, or mean and standard deviations, or median and interquartile range.

Primary and secondary outcomes were analyzed using the intent-to-treat (ITT) population, that is, all patients who were randomized and completed the first PROM registration independent of subsequent protocol violations. A mixed linear model with both fixed effects (first or second registration; device [web app or touchscreen]), and the interaction between them, as well as random effects factor (patient identification number) adjusted for the assessed PROM-level at inclusion was applied. If the 95% CI of the difference in PROMs between the two device types was within the prespecified equivalence margins, agreement was considered satisfied.²³ Agreement between the two device types for the primary outcome SLAQ Global Health was demonstrated with a Bland–Altman plot.²⁴ McNemar’s (paired) test was used to evaluate agreement for the two device types for PASS and SLAQ worsening. Thus, ordinal scores for all PROMs collected with the two device types were compared without any tests for associations with other outcome measures, that is, an anchor-based approach was not applied.

Analyses were conducted using STATA (version 16) and SAS studio software (version 9.4).

Results

The inclusion period started at July 15, 2020, and ended with enrollment of the last patient on November 24, 2020.

Among 66 patients contacted for a telephone screening, 34 patients (51.5%) were enrolled after informed consent was obtained as illustrated in Figure 1. Main reasons for non-enrollment included the following: did not own a smartphone or tablet (10.6%), unable to visit the outpatient clinic in the study period (9.1%), and interest in participating but did not send the signed informed consent form back to the investigator (9.1%). Enrolled patients were younger than excluded patients as illustrated in Supplementary Table S1, but no significant differences in gender, disease duration, SLAQ global health, or SLEDAI were observed.

Figure 1.

Flow diagram of patient recruitment.

Most baseline characteristics were similar between the two intervention groups as presented in Table 1. However, a significant age difference was present as the group WA → TS was slightly younger than the group TS → WA; that is, mean age difference was −9.8 years (95% CI: −19.3 to −0.2) and a p-value of 0.046 analyzed with a two-sample t-test. As expected, most patients were women (79.4%), ANA-positive (97.1%), and fulfilled the 2019 EULAR/ACR SLE classification criteria ¹ (97.1%). One patient was ANA-negative and, thus, did not fulfill the 2019 classification criteria. However, the patient was diagnosed with renal biopsy lupus nephritis together with joint involvement, malar rash, and hypocomplementemia; thus, the SLE diagnosis was not questioned.

Table 1.

Patient demographics and clinical characteristics.

Variable	WA → TS n = 17	TS → WA n = 17	All n = 34	p-value
General characteristics
Women, n (%)	14 (82.4)	13 (76.5)	27 (79.4)	1.00^a
Age (years), mean (SD)	41.6 (13.4)	51.4 (13.9)	46.5 (14.4)	0.046^b
Fulfill 2019 SLE Classification Criteria, n (%)	16 (94.1)	17 (100)	33 (97.1)	1.00^a
Disease duration (years), median (IQR)	7.7 (1.7–13.2)	5.6 (2.7–16.1)	6.4 (2.0–13.0)	0.959^c
SLEDAI (0–105), median (IQR)	6.0 (0.0–8.0)	1.0 (0.0–4.5)	2.0 (0.0–6.0)	0.087^c
SLICC Damage Index (0–47), median (IQR)	0.0 (0.0–1.0)	0.0 (0.0–0.0)	0.0 (0.0–1.0)	0.079^c
ANA positive, n (%)	16 (94.1)	17 (100)	33 (97.1)	1.00^a
Anti-dsDNA^d, median (IQR)	9.6 (3.4–41.0)	3.6 (0.9–14.5)	7.5 (1.0–15.8)	0.469^c
CRP (mg/L)^e, median (IQR)	3.4 (1.3–7.2)	1.6 (1.0–4.4)	2.4 (1.2–5.1)	0.196^c
Current medication for SLE
None, n (%)	4 (23.5)	2 (11.8)	6 (17.7)	0.66^a
Hydroxychloroquine, n (%)	11 (64.7)	13 (76.5)	24 (70.6)	0.71^a
Azathioprine, n (%)	2 (11.8)	3 (17.7)	5 (14.7)	1.00^a
Prednisolone, n (%)	5 (29.4)	5 (29.4)	10 (29.4)	1.00^a
PROMs
SLAQ global health (0–10), median (IQR)	3.0 (1.0–6.0)	4.0 (1.0–5.5)	3.0 (1.0–5.8)	0.985^c
SLAQ symptom (0–24), median (IQR)	10.0 (7.0–12.0)	11.0 (6.5–17.0)	10.5 (7.0–14.0)	0.334^c
SLAQ total (0–44), median (IQR)	9.3 (5.8–15.3)	11.0 (5.6–16.4)	10.3 (5.8–16.2)	0.597^c
SLAQ worsening (0–3), median (IQR)	0.0 (−2.0-0.0)	0.0 (0.0–0.0)	0.0 (−1.0-0.0)	0.146^c
HAQ-DI (0–3), median (IQR)	0.1 (0.0–0.7)	0.3 (0.1–1.1)	0.3 (0.0–0.8)	0.153^c
Pain VAS (0–100 mm), median (IQR)	17.0 (5.5–53.0)	24.0 (6.0–53.5)	18.5 (5.8–53.3)	0.782^c
Fatigue VAS (0–100 mm), median (IQR)	30.0 (15.5–71.0)	54.0 (32.0–75.5)	48.0 (17.8–74.3)	0.263^c
Patient global health VAS (0–100 mm), median (IQR)	46.0 (8.5–60.0)	50.0 (11.0–67.0)	48.0 (8.8–65.0)	0.459^c
PASS “yes”, n (%)	13 (81.3)	13 (76.5)	26 (78.8)	1.00^a
Anchoring question (−3 to 3), median (IQR)	0.0 (−0.8-0.0)	0.0 (−0.5-0.0)	0.0 (−0.5-0.0)	0.886c

WA: web app, TS: touchscreen, n: number, SD: standard deviation, SLE: systemic lupus erythematosus, IQR: interquartile range, SLEDAI: Systemic Lupus Erythematosus Disease Activity Index, SLICC: Systemic Lupus International Collaborating Clinics, ANA: anti-nuclear antibodies, anti-dsDNA: anti-double stranded deoxyribonucleic acid, CRP: C-reactive protein, mg: milligram, L: liter, SLAQ: Systemic Lupus Activity Questionnaire, HAQ-DI: Health Assessment Questionnaire disability index, VAS: Visual Analog Scale, PASS: Patient Acceptable Symptom State.

^a: Analyzed using Fisher’s exact test.

^b: Analyzed using two-sample t-test.

^c: Analyzed using Wilcoxon rank-sum test (Mann–Whitney U test).

^d: Reference value <10 kilounits/L.

^e: Reference value <8 mg/L.

Missing values: Group WA → TS: SLEDAI (n = 2), SLAQ global health (n = 2), SLAQ symptom (n = 2), SLAQ total (n = 2), SLAQ worsening (n = 2), PASS (n = 1), and anchoring question (n = 1).

A Bland–Altman plot showed excellent reliability and good agreement between the two device types for SLAQ Global Health (Figure 2).

Figure 2.

Bland–Altman plot for SLAQ Global Health assessed on the two device types. (a) SLAQ Global Health web app scores against touchscreen scores, that is, x against y plot. Solid line: line of equality. (b) SLAQ Global Health difference (i.e., web app minus touchscreen) against mean SLAQ Global Health score. Solid horizontal line: mean difference, dashed horizontal lines: 95% limits of agreement. Note: SLAQ: systemic lupus erythematosus activity questionnaire

As presented in Table 2, equivalence was demonstrated for SLAQ Global Health between the two device types with a mean difference of −0.21 (95% CI: −0.65 to 0.23) as the 95% CI was well within the prespecified equivalence margin of ± 0.75. Furthermore, equivalence was also shown for all other PROMs with prespecified equivalence margins (i.e., HAQ-DI, Pain VAS, and Fatigue VAS) except Patient Global Health VAS as the 95% CI of −1.18 to 6.53 exceeded the prespecified equivalence margin of ± 5.0. However, as the difference in the Patient Global Health VAS score between the two device types was within the MCID of ± 10.0, comparability was proven, and the observed difference was not considered clinically relevant. Comparability with no statistically significant difference between the two device types was also demonstrated for SLAQ Total, SLAQ Worsening, PASS, and Anchoring Question as presented in Table 2. However, a statistically significant difference between device types was observed for SLAQ Symptom of −0.56 (95% CI: −1.10 to −0.01), p-value 0.045. The observed difference was, however, very small when considering the scale range of 0–24; thus, it was not judged to be of clinical relevance.

Table 2.

Comparison between intervention groups for all PROMs and clinical outcomes.

	Group WA → TS				Group TS → WA					Difference
Outcome	WA		TS		TS		WA			WA − TS
Primary outcome	Mean	SE	Mean	SE	Mean	SE	Mean	SE	Mean difference	95% CI	p-value (superiority test)
SLAQ global health (0–10)	2.9	0.7	3.3	0.7	3.4	0.7	3.3	0.7	−0.21	−0.65 to 0.23	0.348
Secondary outcomes
SLAQ symptom (0–24)	9.3	1.3	9.7	1.3	11.5	1.3	10.8	1.3	−0.56	−1.10 to −0.01	0.045
SLAQ total (0–44)	10.2	1.8	10.0	1.8	12.2	1.8	10.8	1.8	−0.61	−1.45 to 0.22	0.145
SLAQ worsening, % (n/N)	64.7%^a	(11/17)^a	64.7%^a	(11/17)^a	70.6%^a	(12/17)^a	64.7%^a	(11/17)^a	17.7% (6/34)^b	6.8% to 34.5%^c	0.564^d
HAQ-DI (0–3)	0.41	0.08	0.41	0.08	0.38	0.08	0.42	0.08	0.02	−0.01 to 0.05	0.255
Pain VAS (0–100 mm)	19.7	4.5	19.3	4.5	26.6	4.5	30.1	4.5	2.0	−0.85 to 4.85	0.163
Fatigue VAS (0–100 mm)	51.9	4.6	52.8	4.6	41.4	4.6	41.7	4.6	−0.3	−3.80 to 3.15	0.851
Patient global VAS (0–100 mm)	33.3	4.1	26.0	4.1	30.6	4.1	28.7	4.1	2.7	−1.18 to 6.53	0.167
PASS, % (n/N)	70.6%^e	(12/17)^e	88.2%^e	(15/17)^e	70.6%^e	(12/17)^e	82.4%^e	(14/17)^e	26.5% (9/34)^f	12.9% to 44.4%^g	0.739^d
Anchoring question (−3 to 3)	−0.06	0.24	0.06	0.24	0.00	0.24	−0.24	0.24	−0.18	−0.46 to 0.11	0.211

WA: web app, TS: touchscreen, SE: standard error, CI: confidence interval, SLAQ: Systemic Lupus Activity Questionnaire, n: number, HAQ-DI: Health Assessment Questionnaire disability index, VAS: Visual Analog Scale. Missing values: none.

Data are analyzed using mixed linear model unless otherwise stated.

^aPercentage and number of patients who answered “No Flare” to SLAQ worsening.

^bPercentage and number of patients with a difference in SLAQ worsening score between the two device types, that is, a difference in SLAQ worsening >0.

^c95% CI for the proportion of patients with a difference in SLAQ worsening score between the two device types.

^dAnalyzed using McNemar’s test to assess if a significant difference between the two device types existed.

^ePercentage and number of patients who answered “Yes” to PASS.

^fPercentage and number of patients with a difference in PASS score between the two device types, that is, a difference in PASS >0.

^g95% CI for the proportion of patients with a difference in PASS score between the two device types.

As presented in Table 3, no significant differences in gender or age groups were found when exploring subgroup analyses by SLAQ Global Health.

Table 3.

Subgroup analyses on SLAQ Global Health.

Subgroups	Subgroup “Yes”			Subgroup “No”			Difference between subgroups
Subgroups	n	Mean	SD	n	Mean	SD	Mean	95% CI	p-value
Yes = female sex No = male sex	27	−0.1	1.3	7	−0.6	1.0	0.5	−0.6 to 1.5	0.39
Yes = the younger half^a No = the older half	17	0.0	0.6	17	−0.4	1.7	0.4	−0.5 to 1.3	0.34
Yes = age <65 years No = age ≥65 years	31	−0.1	1.3	3	−1.0	1.0	0.9	−0.7 to 2.4	0.26

n: number, SD: standard deviation, CI: confidence interval.

Missing values: none.

^aYounger half = age ≤47 years.

Patient preference for the DANBIO web app was very high 91.2% (31 patients), only 1 patient preferred the outpatient touchscreen (2.9%) and 2 patients had no device preference (5.9%).

Discussion

To our knowledge, agreement between two electronic device types has not previously been demonstrated for reporting of PROMs by patients with SLE. Equivalence was shown for all PROMs with prespecified equivalence margins (i.e., HAQ-DI, Pain VAS, and Fatigue VAS) except Patient Global Health VAS, where only comparability was proven. However, the observed difference in Patient Global Health VAS was within the limits of the MCID and was not judged to have any clinical relevance. Comparability with no statistically significant difference between device types was also demonstrated for SLAQ Total, SLAQ Worsening, PASS, and Anchoring Question; however, a statistically significant difference between device types was observed for SLAQ Symptom of −0.56 (95% CI: −1.10 to −0.01). The observed difference was, however, very small when considering the scale range of 0–24; thus, it was not judged to be of clinical relevance.

A recent systematic literature review (SLR) on mobile health apps for patients with SLE found a major lack of high-quality studies evaluating app efficacy.²⁵ No apps used validated PROMs to evaluate symptoms, and the educational content was labeled low. The SLR only identified two RCTs; the first explored cellular text message reminders with educational content to increase adherence to outpatient visits among young adults with SLE and demonstrated improved adherence to clinic visits but not hydroxychloroquine treatment.²⁶ The second study was a telephone-based weight management program for people with physical disabilities and included only one patient with SLE (1.1%).²⁷ Thus, the SLR concluded that use of mobile health apps for patients with SLE is a relatively new and unexplored topic.²⁵

The findings in this trial are in line with previous studies in inflammatory arthritis that demonstrated agreement within the limits of defined MCIDs for various PROMs between two electronic device types.^14,15 Secher et al. showed agreement in HAQ, Pain VAS, Fatigue VAS, Patient Global Health VAS, Bath Ankylosing Spondylitis Disease Activity Index (BASDAI), and Bath Ankylosing Spondylitis Functional Index (BASFI) between a web-based solution using participants' own computer/tablet and an outpatient touchscreen by patients with rheumatoid arthritis (RA) or axial spondyloarthritis (axSpA).¹⁴ Recently, agreement was demonstrated for HAQ-DI, Pain VAS, Fatigue VAS, Patient Global Health VAS, BASDAI, BASFI, PASS, and Anchoring Question score between an app on the participants’ own device and an outpatient touchscreen by patients with RA, axSpA, and psoriatic arthritis (PsA).¹⁵

Significant strengths to this study are the rate of responses by patients to all PROMs on the two device types, resulting in no missing data in both intervention groups; the study design, that is, randomized, crossover trial for which the internal validity is considered very high and the study population consisted mostly of females, judged to be representative for an outpatient SLE cohort.

In this trial, patients highly preferred the DANBIO web app over the outpatient touchscreen (91.2%). The preference was higher than previously reported: 78.3% of patients with RA, axSpA, or PsA preferred the DANBIO app over the outpatient touchscreen, ¹⁵ and 50% of patients with RA or axSpA preferred a from-home web-based solution on a computer/tablet over the outpatient touchscreen.¹⁴ A possible explanation for the high web app preference could be sampling bias as a significant difference in age was observed between excluded patients (i.e., patients who were not enrolled) and included patients. Another possible explanation for the high web app preference could be easier access as the web app does not require being downloaded on participants' own devices in contrast to the previous DANBIO app. Moreover, in the study by Secher et al., participants were to answer PROMs on the web-based solution from home and not when and where it suited them as in this study.

Limitations to consider in this trial include the significant difference in age between excluded and included patients, as well as between intervention groups. This could be a result of sampling bias, that is, that older patients with less app experience/knowledge did not wish to participate in the trial. However, only 31.3% of excluded patients did not have access to or lacked smartphone/tablet technical skills. Age differences between the intervention groups were not considered to have influenced the trial results as each participant was in his/her own control. Another possible limitation is that most participants had low disease activity (i.e., low SLEDAI, ds-DNA, and CRP); thus, the PROM variance for the trial population would be less than in a population with high disease activity. However, 21.9% of the participants (7/32) had an SLEDAI >6 corresponding to high disease activity ²⁸ and 37.5% (12/32) were not in SLEDAI low disease activity (i.e., SLEDAI >4²⁹). Moreover, it is unlikely that any variance in PROMs due to disease activity would affect the trial results in a significant way as the 95% CIs for the PROM differences are well within the limits of the prespecified MCID. Another limitation could be insufficient power for the subgroup analyses on SLAQ Global Health; to minimize risk of reaching a false conclusion, it was decided that only a p-value <0.10 would be considered potentially important. No significant differences in subgroup analyses were demonstrated using this significant level. Finally, a limitation to consider is that three patients in the WA → TS group answered PROMs on the two device types without a washout period of 1 to 2 days; that is, the assessments were performed on two consecutive days, resulting in a “washout period” under 24 h. The reason for the deviation was simply that the patients forgot the scheduled PROM registration from home despite a text message reminder. However, as the PROM registrations was done on two consecutive dates, the washout period was still longer than reported in similar studies.^{12–14,30–32}

Overall, the results of this trial, that is, agreement between two electronic device types for various PROMs by SLE patients, are considered reliable; thus, the web app is an acceptable alternative for reporting PROMs in clinical practice. Moreover, a recent SLR by Byrom et al. found equivalence in PROM reporting when migrating from paper to electronic format and between different electronic device types.³³ Thus, the results of this study can be generalized when implementing apps for PROM reporting in SLE management around the world and possibly also in diseases with similar symptom patterns, for example, vasculitis or mixed connective tissue disease. However, not all healthcare systems worldwide have access to the required technology to implement a web app; therefore, paper-based forms are still relevant.

Previous research has shown that patients with SLE do not feel understood by their healthcare providers and have unvoiced concerns that are not met.³⁴ PROMs can be a valuable tool to provide the physician with a better understanding of the patient’s perception of disease state, and a web app allows patients to easily and conveniently report changes in disease activity as a flare occurs, thereby minimizing the time delay from disease worsening to consultation with the treating physician. Implementation of this web app in clinical practice is expected to improve healthcare in SLE patients.

Conclusion

For the first time, equivalence and comparability using two electronic device types were demonstrated in various PROMs reported by patients with SLE. Moreover, patients strongly preferred use of the web app over the outpatient touchscreen. Implementation of this web app in clinical practice is expected to optimize the management of SLE patients.

Supplemental Material

Supplemental Material - Patient-reported outcome measures in systemic lupus erythematosus by a web-based application: A randomized, crossover, agreement study

Supplemental Material for Patient-reported outcome measures in systemic lupus erythematosus by a web-based application: A randomized, crossover, agreement study by Line Uhrenholt, Simone Høstgaard, Julie F Pedersen, Robin Christensen, Lene Dreyer, Henrik C B Leffers, Peter C Taylor, Vibeke Strand, Søren Jacobsen, Anne Voss, Jon W Gregersen and SalomeKristensen in Lupus

Footnotes

Acknowledgements

The authors thank patient research partner TAI for refining and approving the participant information material and approving the trial protocol. Furthermore, the authors thank data manager JHW for uploading the concealed allocation sequence in REDCap and MSc Christian Cato Holm for data management. PCT thanks the National Institute of Health Research for financial support to The NIHR Biomedical Research Centre in Musculoskeletal Disease, Oxford University Hospitals NHS Trust and University of Oxford.

Authors' contribution

LU, SK, LD, and RC conceived the trial hypothesis and designed the trial. LU is the sponsor–investigator. LU, SH, and JFP enrolled patients and via REDCap assigned participants to intervention groups. LU, SK, SH, JFP, LD, and RC were responsible for the conduction of the trial, analyses, and publication of trial data. LU wrote this article, and all co-authors contributed with editing and approved the final draft.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: LD: Grants (BMS), Speakers bureau (Galderma, Eli Lilly, Janssen), PCT: Grants (Celgene, Eli Lilly, Galapagos, Gilead), Consultant (AbbVie, Biogen, Eli Lilly, Fresenius, Galapagos, Gilead, GlaxoSmithKline, Janssen, Nordic Pharma, Bristol-Myers Squibb, Pfizer, Roche, Sanofi), Speakers bureau (Biogen). VS: Consultant (Abbvie, Amgen, AstraZeneca, BMS, EMD Serono, Equilium, Genentech/Roche, GSK, Horizon, Janssen, Lilly, Novartis, Pfizer, Rheos, Sanofi, Sun Pharma, UCB). LU, SH, JFP, RC, HBL, SJ, AV, JWG, and SK: Declares no conflict of interest.

Funding

The author(s) disclosed the receipt of the following financial support for the research, authorship, and/or publication of this article: The trial received financial support from the Danish Rheumatism Association (R192-A6737) and the Health Innovation Fond of North Denmark Region (2020–035538). The DANBIO web application is developed and owned by Zitelab ApS. The Oak Foundation supports the Parker Institute with a core grant (OCAY-18-774-OFIL). However, no funding parties nor Zitelab ApS was involved in the trial design or conduct, nor analyses or writing and submitting this article.

Ethics approval

The local Ethics Committee and the Danish Data Protection Agency declared that approval was not required. However, the study was approved as a quality control trial by the Head of Hospital Management, Aalborg University Hospital. The trial was registered at (NCT04411407) prior to participant inclusion (May 28, 2020).

Informed Consent

Written informed consent was obtained from all participants before inclusion. The trial was carried out in accordance with the Helsinki Declaration, Good Clinical Practice (GCP) and the study protocol.

Data availability

This article and the supplementary information files include all analyzed data from this study.

ORCID iD

Line Uhrenholt

Supplement Material

Supplemental material for this article is available online.

References

Aringer

Costenbader

Daikh

, et al. European league against rheumatism/american college of rheumatology classification criteria for systemic lupus erythematosus. Arthritis Rheumatol 2019; 71: 1400–1412.

Leffers

HCB

Troldborg

Voss

, et al. Smoking associates with distinct clinical phenotypes in patients with systemic lupus erythematosus: a nationwide Danish cross-sectional study. Lupus Sci Med 2021; 8: e000474.

Annapureddy

Devilliers

Jolly

. Patient-reported outcomes in lupus clinical trials with biologics. Lupus 2016; 25: 1111–1121.

Mahieu

Yount

Ramsey-Goldman

. Patient-reported outcomes in systemic lupus erythematosus. Rheum Dis Clin North America 2016; 42: 253–263.

Romero-Diaz

Isenberg

Ramsey-Goldman

. Measures of adult systemic lupus erythematosus: updated version of British Isles Lupus Assessment Group (BILAG 2004), European Consensus Lupus Activity Measurements (ECLAM), Systemic Lupus Activity Measure, Revised (SLAM-R), Systemic Lupus Activity Questionnaire for Population Studies (SLAQ), Systemic Lupus Erythematosus Disease Activity Index 2000 (SLEDAI-2K), and Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index (SDI). Arthritis Care Res 2011; 63: S37–46.

Katz

Pedro

Alemao

, et al. Estimates of responsiveness, minimally important differences, and patient acceptable symptom state in five patient‐reported outcomes measurement information system short forms in systemic lupus erythematosus. ACR Open Rheumatol 2020; 2: 53–60.

Izadi

Gandrup

Katz

, et al. Patient-reported outcome measures for use in clinical trials of SLE: a review. Lupus Sci Med 2018; 5: e000279.

Karlson

Daltroy

Rivest

, et al. Validation of a systemic lupus activity questionnaire (SLAQ) for population studies. Lupus 2003; 12: 280–286.

Strand

Gladman

Isenberg

, et al. Endpoints: consensus recommendations from OMERACT IV. Lupus 2000; 9: 322–327.

10.

Yazdany

Yelin

Panopalis

, et al. Validation of the systemic lupus erythematosus activity questionnaire in a large observational cohort. Arthritis Rheum 2008; 59: 136–143.

11.

Coons

Eremenco

Lundy

, et al. Capturing Patient-Reported Outcome (PRO) data electronically: the past, present, and promise of ePRO measurement in clinical trials. Patient-Patient-Centered Outcomes Res 2015; 8: 301–309.

12.

Schefte

Hetland

. An open-source, self-explanatory touch screen in routine care. Validity of filling in the bath measures on ankylosing spondylitis disease activity index, function index, the health assessment questionnaire and visual analogue scales in comparison with paper versions. Rheumatology 2010; 49: 99–104.

13.

Wæhrens

Amris

Bartels

, et al. Agreement between touch-screen and paper-based patient-reported outcomes for patients with fibromyalgia: a randomized cross-over reproducibility study. Scand J Rheumatol 2015; 44: 503–510.

14.

Secher

Glintborg

Gudbergsen

, et al. Comparing patient-reported outcomes entered at home versus at hospital, and testing touch screens for initial recruitment to scientific trials in arthritis patients. Scand J Rheumatol 2019; 48: 178–184.

15.

Uhrenholt

Christensen

Dreyer

, et al. Using a novel smartphone application for capturing of patient-reported outcome measures among patients with inflammatory arthritis: A randomized, crossover, agreement study. Scand J Rheumatol 2021; 1–9. DOI: 10.1080/03009742.2021.1907925.

16.

Strand

Boers

Idzerda

, et al. It’s good to feel better but it’s better to feel good and even better to feel good as soon as possible for as long as possible. Response criteria and the importance of change at OMERACT 10. J Rheumatol 2011; 38: 1720–1727.

17.

Harris

Taylor

Thielke

, et al. Research electronic data capture (REDCap)-A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009; 42: 377–381.

18.

Harris

Taylor

Minor

, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 2019; 95: 103208.

19.

Cook

Julious

Sones

, et al. DELTA2 guidance on choosing the target difference and undertaking and reporting the sample size calculation for a randomised controlled trial. BMJ 2018; 363: k3750.

20.

Moher

Hopewell

Schulz

, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010; 340: c869.

21.

Dwan

Altman

, et al. CONSORT 2010 statement: extension to randomised crossover trials. BMJ 2019; 366: l4378.

22.

Christensen

Bliddal

Henriksen

. Enhancing the reporting and transparency of rheumatology research: a guide to reporting guidelines. Arthritis Res Ther 2013; 15: 109.

23.

Piaggio

Elbourne

Pocock

, et al.; CONSORT Group. Reporting of noninferiority and equivalence randomized trials. JAMA 2012; 308: 2594–2604.

24.

Bland

Altman

. Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet 1986; 327: 307–310.

25.

Dantas

Weber

Osani

, et al. Mobile health technologies for the management of systemic lupus erythematosus: a systematic review. Lupus 2020; 29: 144–156.

26.

Ting

Kudalkar

Nelson

, et al. Usefulness of cellular text messaging for improving adherence among adolescents and young adults with systemic lupus erythematosus. J Rheumatol 2012; 39: 174–179.

27.

Rimmer

Wang

Pellegrini

, et al. Telehealth weight management intervention for adults with physical disabilities. Am J Phys Med Rehabil 2013; 92: 1084–1094.

28.

Polachek

Gladman

, et al. Defining low disease activity in systemic lupus erythematosus. Arthritis Care Res 2017; 69: 997–1003.

29.

Gladman

Ibañez

Urowitz

. Systemic lupus erythematosus disease activity index 2000. J Rheumatology 2002; 29: 288–291.

30.

Epis

Casu

Belloli

, et al. Pixel or paper? validation of a mobile technology for collecting patient-reported outcomes in rheumatoid arthritis. JMIR Res Protoc 2016; 5: e219.

31.

Richter

Becker

Koch

, et al. Self-assessments of patients via Tablet PC in routine patient care: comparison with standardised paper questionnaires. Ann Rheum Dis 2008; 67: 1739–1741.

32.

Pincus

Castrejon

RiadObreja

, et al. Reliability, feasibility, and patient acceptance of an electronic version of a multidimensional health assessment questionnaire for routine rheumatology care: validation and patient preference study. JMIR Formative Res 2020; 4: e15815.

33.

Byrom

Gwaltney

Slagle

, et al. Measurement equivalence of patient-reported outcome measures migrated to electronic formats: a review of evidence and recommendations for clinical trials and bring your own device. Ther Innovation Regul Sci 2019; 53: 426–430.

34.

Hale

Treharne

Lyons

, et al. “Joining the dots” for patients with systemic lupus erythematosus: Personal perspectives of health care from a qualitative study. Ann Rheum Dis 2006; 65: 585–589.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.32 MB