The Edmonton Symptom Assessment System: Poor performance as screener for major depression in patients with incurable cancer

Abstract

Background:

Depressive symptoms are prevalent in patients with advanced cancer, sometimes of a severity that fulfil the criteria for a major depressive episode.

Aim:

The aim of this study was to investigate how the item on depression in the Edmonton Symptom Assessment System with a 0–10 Numerical Rating Scale performed as a screener for major depressive episode. A possible improved performance by adding the Edmonton Symptom Assessment System-Anxiety item was also examined.

Design:

An international cross-sectional study including patients with incurable cancer was conducted. The Edmonton Symptom Assessment System score was compared against major depressive episode as assessed by the Patient Health Questionnaire-9. Screening performance was examined by sensitivity, specificity and the kappa coefficient.

Setting:

Patients with incurable cancer (n = 969), median age 63 years and from eight nationalities provided report. Median Karnofsky Performance Status was 70. Median survival was 229 days (205–255 days).

Results:

Patient Health Questionnaire-9 major depressive episode was present in 133 of 969 patients (13.7%). Edmonton Symptom Assessment System-Depression screening ability for Patient Health Questionnaire-9 major depressive episode was limited. Area under the receiver operating characteristic curve was 0.71 (0.66–0.76). Valid detection or exclusion of Patient Health Questionnaire-9 major depressive episode could not be concluded at any Edmonton Symptom Assessment System-Depression cut-off; by the cut-off Numerical Rating Scale ⩾ 2, sensitivity was 0.69 and specificity was 0.60. By the cut-off Numerical Rating Scale ⩾ 4, sensitivity was 0.51 and specificity was 0.82. Combined mean ratings by Edmonton Symptom Assessment System-Depression and Edmonton Symptom Assessment System-Anxiety revealed similar limited screening ability.

Conclusion:

The depression and anxiety items of the Edmonton Symptom Assessment System, a frequently used assessment tool in palliative care settings, seem to measure a construct other than major depressive episode as assessed by the Patient Health Questionnaire-9 instrument.

Keywords

Palliative care neoplasms depression depressive disorder symptom assessment

What is already known about the topic?

The depression item of the Edmonton Symptom Assessment System (ESAS) is a frequently used symptom assessment tool, but has no standard guide for how to interpret patients’ scorings into clinical decisions.

About 15% of patients with advanced cancer have a major depression which is frequently overlooked.

ESAS-Depression has shown promising results as a good screener for depression in patients early in the cancer disease trajectory or in a curative setting.

What this paper adds?

ESAS-Depression shows limited screening ability for major depression as assessed by the Patient Health Questionnaire-9 (PHQ-9) instrument in patients with incurable cancer (area under the receiver operating characteristic curve of about 0.7).

Adding the ESAS-Anxiety item provides similar limited screening ability.

The ESAS-Depression item measures a construct other than major depression as assessed by the PHQ-9 instrument in patients with incurable cancer.

Implications for practice, theory or policy

ESAS-Depression is of questionable value for screening of major depression in patients with advanced cancer.

At the present time, ESAS-Depression does not substitute for PHQ-9 screening in patients with incurable cancer.

Given the poor agreement between the ESAS-Depression and the PHQ-9, either one or both of the instruments perform poorly in patients with incurable cancer.

Introduction

Depression in patients with advanced cancer

Patients with advanced cancer experience multiple symptoms such as pain, fatigue and anxiety, with depressive symptoms being among the commonest.^1,2 The term ‘depression’ expresses a spectre from the transient feeling of sadness to major depression disorder (MDD).^3–5 Assessed by different questionnaires, depression estimates vary around 30%–50% of patients with advanced cancer.^1,6 Major depressive disorder is defined by the Diagnostic and Statistical Manual of Mental Disorders (DSM) and International Classification of Diseases (ICD) psychiatric classification systems as one or more major depressive episodes (MDEs).^7,8 According to the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5), a MDE is characterized by the persistent presence of at least five of nine depressive symptoms that cause significant distress or functional impairment for 2 weeks or longer. A systematic review estimated the point prevalence of MDE in palliative care cancer populations to 16.5% (13.1%–20.3%)² as opposed to estimations of 0.9%–4.6% in general populations.⁹ The high prevalence underlines depression as an important concern in palliative care. This is further accentuated by the fact that major depression can be relieved by treatment, making identification of major depression an integral part of palliative care.^10–12

Depression in symptom assessment

In palliative care cancer programmes, symptom assessment tools are used to assess symptom intensity, prevalence and course; to guide clinical consultations; and to monitor treatment effects.^13–15 Patients’ self-report of symptoms, also known as patient-reported outcomes (PROs), is a core assessment method in palliative care given this care’s central focus on symptoms and quality of life.^15–17 A plethora of instruments exists, ranging from single-item assessment tools to comprehensive assessment tools for multiple symptoms each often assessed by multiple items per dimension. Brief methods are advocated for monitoring the many symptoms in progressively diseased patients.^18–20 Most tools assessing multiple symptoms include one or more items on depression symptoms. The second most applied assessment of depression in palliative care research is the Edmonton Symptom Assessment System (ESAS) that includes nine simple items on frequent symptoms.^21,22

Screening for depression

Screening for depression by single items has been investigated in patients with advanced cancer. Screening by use of the MDE main criterion on lowered mood was originally promising.²³ The single item ‘Are you depressed?’ performed poorly in three later studies among patients with advanced cancer.^24–26 In contrast to these findings, Taylor et al.²⁷ found good screening capabilities by use of the main criterion on lowered mood for MDE. The question ‘Have you felt depressed most of the day, nearly every day, for two or more weeks?’ identified MDD assessed by a structured interview with a sensitivity of 0.80 and a specificity of 0.85.²⁷ This finding indicates that simple items are possible screeners for MDE in patients with advanced cancer.

The ESAS-Depression item

The ESAS assesses nine symptoms that are common among patients with advanced cancer, on 0–10 Numerical Rating Scales (NRS). The ESAS includes one item of the following symptoms: pain, fatigue, nausea, depression, anxiety, drowsiness, appetite, general well-being and shortness of breath.²¹ The ESAS was developed for a pragmatic day-to-day assessment of symptoms.^21,28,29 The ESAS has successfully been implemented in daily symptom monitoring in palliative care worldwide^28,30 and is frequently applied in palliative care research assessment.²² A recently developed extended version of the ESAS, the European Association for Palliative Care (EAPC) Basic Dataset, adds items on sleep, constipation and vomiting.³¹ There is no definitive guide how to interpret patient scorings of the emotional items ESAS-Depression and ESAS-Anxiety.^28,29 In validation studies, the ESAS-Depression item has been compared to the Hospital Anxiety and Depression Rating Scale (HADS),^32–37 the Rotterdam Symptom Checklist^38–40 and later the Patient Health Questionnaire-9 (PHQ-9) numerical sumscore;^41,42 PHQ-9 can be analysed by sumscore or by the MDE diagnostic concept. Palliative care guidelines recommend psychiatric classification as the standard reference for depression.^12,14 Interpretation of ESAS-Depression scorings into clinical decisions would be facilitated if ESAS-Depression could be applied as a screening instrument for MDE.

ESAS-Depression – a valid indicator of depression?

Previous studies have examined the screening abilities of ESAS-Depression. Vignaroli et al.³³ examined a mixed cancer population and compared ESAS-Depression with HADS-Depression (HADS-D ⩾ 11). With acceptable sensitivity (0.83) and low specificity (0.47), Vignaroli et al. proposed ESAS-Depression score of ⩾2 as a cut-off and proposed further research on cut-off values for severe depression. Bagha et al.⁴² examined cancer outpatients. They found that ESAS-Depression performed well compared to PHQ-9 sumscore (⩾10 of 27) and proposed ESAS-Depression as an initial screening instrument to exclude non-depressed patients before screening with a more extended instrument. An ESAS-Depression score of ⩾2 was proposed as cut-off (sensitivity: 0.86, specificity: 0.72). Ripamonti et al.³⁷ found good screening performance of the ESAS-Depression item compared to HADS (cut-off ⩾ 11) in patients with non-advanced cancer. They proposed a score of ⩾4 as an optimal cut-point.

Documentation of the screening capabilities of the ESAS-Depression item in the palliative care context is limited. Studies have been small and depression has been conceptualised differently. Still, ESAS has been claimed to be a valid screener.^34,35,40

ESAS-Depression and ESAS-Anxiety

Anxiety and depression frequently co-occur in patients with advanced cancer.^2,34,43–45 The combined assessment of these symptoms is integrated in many instruments as a measure of psychological distress. The ESAS-Anxiety item also tapping an underlying depression construct is, therefore, a reasonable hypothesis that has not yet been investigated.

Aims of this study

The aims of this study were to test screening capabilities of ESAS-Depression alone and in combination with ESAS-Anxiety when compared to MDE as assessed by the PHQ-9 in patients with incurable cancer.

The specific research questions were as follows:

Does the ESAS-Depression item have adequate screening ability for a MDE assessed by the PHQ-9 instrument (PHQ-9-MDE) in patients with incurable cancer?

Does the additional assessment ESAS-Anxiety improve screening ability for MDE assessed by the PHQ-9 instrument (PHQ-9-MDE) in patients with incurable cancer?

Materials and methods

Design and sampling

An observational cross-sectional study was conducted as part of an international multicentre study, the Computer Symptom Assessment (CSA) study run by the European Palliative Care Research Collaborative (EPCRC).^16,46 The EPCRC project aimed to improve assessment and classification of depression, pain and cachexia in patients with advanced cancer.^{22,45,47–49} Inclusion criteria were patients with incurable cancer and aged 18 years or older. Exclusion criteria were obvious cognitive impairment, language problems or physical disability preventing participation. Patients were included in palliative care inpatient and outpatient units; hospices; and general oncology, surgical and medical wards.

The study was approved by the appropriate ethical authorities at each study site. All patients provided written informed consent.

Data collection

A total of 17 medical centres in eight countries participated from October 2008 to December 2009. The centres were as follows: Australia: Braeside Hospital NS, West Australian Centre for Cancer and Palliative Care, Curtin University, Southern Adelaide Palliative Services; Austria: Medical University of Graz, University of Calgary, Department of Family Medicine, Division of Palliative Medicine; United Kingdom; Ullevål & Norwegian Radium, Bergen, and Telmark Hospital Trust; Department of Internal Medicine, Division of Oncology; Canada: University of Alberta, Grey Nuns Community Hospital Division of Palliative Care Medicine, University of Calgary, Department of Family Medicine; United Kingdom: University of Bristol, Department of Palliative Medicine, Bristol Haematology and Oncology Centre, St George’s Hospital Medical School, St Georges University of London, Division of Palliative Medicine; Germany: University Hospital RWTH Aachen, Department of Palliative Medicine Aachen University; Italy: Rehabilitation and Palliative Care, National Cancer Institute of Milan, Unità Cure Palliative, Liguria, Genova; Norway: Ullevål & Norwegian Radium, Bergen, and Telmark Hospital Trust, Palliative Medicine Unit, Department of Oncology, St. Olav’s Hospital, Oslo University Hospital, Ullevål; Norwegian Radium Hospital, Sunniva Centre for Palliative Care, Haraldsplass Deaconess Hospital, Bergen; and Telemark Hospital Trust; Switzerland: Kantonsspital St. Gallen, Oncology and Palliative Medicine. Touch screen sensitive laptops were used for data collection; one section was filled in by patients and one section by health-care personnel. The native languages, Norwegian, English, German or Italian, were used in patient assessment. Each question had to be completed to move to the next question. A research nurse or study coordinator provided assistance if necessary.

Measurements

Health-care personnel provided demographic and medical data (Table 1). Several PROs,⁴⁶ among them the ESAS²¹ and the PHQ-9,⁴¹ were rated by the patients.

Table 1.

Patient characteristics.

		Sample patients	Non-sample patients^a
		N (%)	N (%)
Subjects		969 (100)	82 (100)
Sociodemographic data
Gender	Female	465 (48)	38 (46)
Gender	Male	504 (52)	44 (54)
Age (years)	Median	63	68
Age (years)	Range	18–91	28–98
Living situation	Spouse	643 (65.4)	52 (66.7)
Living situation	Living alone	254 (26.2)	21 (25.9)
Education (years)	⩽9	333 (34.5)	34 (41)
	10–12	338 (35.0)	32 (40)
	>12	295 (30.6)	5 (6.3)
Nationality	Norwegian	448 (50.4)	32 (39)
	Austrian	99 (10.2)	1 (1.2)
	Swiss	92 (9.5)	18 (22)
	Italian	88 (9.1)	3 (3.7)
	British	81 (8.4)	4 (4.9)
	Australian	61 (6.3)	9 (11)
	Canadian	31 (3.2)	3 (3.7)
	German	29 (3.0)	12 (14.6)
Medical data
Outpatients		420 (43.4)	31 (37.8)
Inpatients		547 (56.6)	51 (62.2)
Survival^b (days)	Median	229	52
Survival^b (days)	95% confidence interval	205–255	29–76
KPS	Mean	70.9	58.6
	Standard deviation	16.4	20.2
	<80	542 (56.6)	60 (73.2)
	⩽80	748 (78.2)	72 (87.9)
Major depression	PHQ-9-MDE^c	133 (13.7)
Depression	PHQ-9 sumscore ⩾ 10 of 27	307 (31.7)
Antidepressants^d	Total	131 (13.5)	10 (12.3)
	With PHQ-9-MDE^c	33
	Without PHQ-9-MDE^c	98
Cancer diagnosis	Gastrointestinal	257 (26.6)	17 (20.7)
	Breast	167 (17.3)	10 (12.2)
	Respiratory organs	161 (16.7)	16 (19.5)
	Urinary or male genital	159 (16.5)	15 (18.3)
	Gynaecological	26 (2.7)	6 (7.3)
	Other	199 (20.2)	18 (22.0)
Mean ESAS reports (standard deviation)
	Pain	2.2 (2.3)
	Tiredness	3.7 (2.5)
	Nausea	1.1 (2.5)
	Depression	1.9 (2.3)
	Anxiety	2.1 (2.3)
	Drowsiness	3.4 (2.5)
	Reduced appetite	3.3 (3.0)
	Feeling of well-being	3.4 (2.4)
	Shortness of breath	1.9 (2.4)

KPS: Karnofsky Performance Status; PRO: patient-reported outcome; PHQ-9: Patient Health Questionnaire-9; ESAS: Edmonton Symptom Assessment System.

Non-sample patients: patients who did not report PROs (N = 82; see Figure 1).

Survival: 71 sample patients missing data, 12 non-sample patients missing data.

MDE: major depressive episode according to PHQ-9 measurement (see text).

Antidepressant medication other than for pain treatment.

ESAS

The ESAS assesses nine symptoms: pain, tiredness, nausea, depression, anxiety, drowsiness, appetite, feeling of well-being and shortness of breath.^21,50 The items were introduced by ‘Please mark the number that best describes your situation right now:’ ‘Depression’ and ‘Anxiety’. The anchors of the scales were ‘Not depressed’, ‘Worst possible depression’, ‘Not anxious’ and ‘Worst possible anxiety’. The 0–10 NRS scoring by the ESAS-Depression item and the mean ESAS-Depression and ESAS-Anxiety scoring for each patient were examined.

PHQ-9

The PHQ-9 instrument is composed of 10 questions.⁵¹ The nine symptom criteria of MDE (DSM-5) are assessed: (1) little interest or pleasure in doing things; (2) feeling down, depressed or hopeless; (3) disrupted sleep; (4) feeling tired or having little energy; (5) appetite changes; (6) feeling bad about oneself or as a failure; (7) trouble concentrating; (8) slowness or fidgety, restlessness; and (9) thoughts of being better off dead or hurting oneself. The items are introduced by ‘Over the last two weeks, how often have you been bothered by …’. Each item is rated using four response options: 0, not at all; 1, several days; 2, more than half the days; and 3, nearly every day. Each symptom is regarded as present by rating 2 or 3. A 10th question asks for the symptoms’ influence on functioning but is not included in the standard analysis. The original English and the authorized Norwegian, German and Italian translations were used.^41,51 Standard scoring according to the DSM-5 diagnostic algorithm for MDE was applied as the main assessment (PHQ-9-MDE);^41,51 at least five of the nine symptoms should be present, and at least one of these must be the main symptom. The PHQ-9 numeric sumscore (0–27) was also used as a comparator.

Statistical analysis

Descriptive statistics

Standard descriptive statistics were applied with frequencies, mean (standard deviation (SD)) and median (range) of patient characteristics. Survival was calculated by the Kaplan–Meier method.⁵²

Sensitivity and specificity

Indication for clinical support for detection of major depression was evaluated according to suggestions by Löwe et al.:⁵³ a minimum specificity of 0.75 and a maximum sensitivity above the specificity value.⁵³

In a two-step screening procedure, sensitivity in the first step would be most important to not overlook depressed subjects; sensitivity of 0.85 was considered putative for this purpose.⁴²

Area under the receiver operating characteristic curve

Combined sensitivities and specificities were visualized in a receiver operating characteristic (ROC) curve. The area under the ROC curve (AUC) provides an estimate of overall discrimination and for evaluations of an appropriate cut-off value of the screening item. AUC of 0.5–0.7 indicates low accuracy, 0.7–0.9 indicates moderate accuracy and 0.9–1.0 indicates high accuracy.^54,55

Coefficient κ

Cohen’s⁵⁶ coefficient κ estimated strength of agreement between ESAS dichotomized by the different cut-offs and PHQ-9-MDE. Clinical usefulness was evaluated according to suggestions by Landis and Koch:⁵⁷ <0.00, poor; 0.00–0.20, slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; 0.81–1.0, almost perfect.

The PASW 21 statistical package (IBM SPSS, Armonk, NY, 2012) and an online statistical calculator (http://statpages.org/ctab2x2.html (18 March 2014) were used with the statistical analyses.

Results

Altogether 1070 patients completed the study (Figure 1). Information on 15 patients was incomplete because of technical failure, and they were omitted from analyses. Four patients withdrew their informed consent. Of the remaining 1051 patients, 26 did not complete ESAS-Depression and ESAS-Anxiety, and 56 did not complete the PHQ-9, leaving a sample of 969 patients (92.2%) for this study, 48% female and 52% male. Mean Karnofsky Performance Status (KPS) was 70.9 (SD: 16.4). Median age was 63 years (range: 18–91 years). Median survival was 229 days (95% confidence interval: 205–255 days). Gastrointestinal cancer was the commonest type of cancer (26.6%), 17.3% had breast cancer, 16.7% had cancer in the respiratory organs and 16.5% had urological cancer.

Half of the patient sample was recruited in Norway, the remaining patients being uniformly spread among the other nationalities (Table 1). The prevalence of a MDE as assessed by PHQ-9 (PHQ-9-MDE) was 13.7% (Table 1). Out of these 133 patients, 33 patients received antidepressant medication for other than pain. A total of 98 patients receiving antidepressants did not fulfil the PHQ-9-criteria for MDE. The mean score for ESAS-Depression was 1.9 (SD: 2.3), and mean ESAS-Anxiety was 2.1 (SD: 2.3). Patients who did not complete the PROs (non-sample patients) had lower KPS and shorter survival than those who did. About the same proportion of patients received antidepressants in the sample (13.5%) and the non-sample (12.3%) groups.

Sensitivity and specificity

The ROC curves (Figure 2) showed relatively low overall accuracies. The AUC was about 0.70 for all measures (Table 2) indicating that the ESAS-Depression item is a poor discriminator of MDE as defined by the PHQ-9-MDE. Similarly, the use of the combined ESAS-Depression and ESAS-Anxiety score added little. Also, by use of a cut-off on the PHQ-9 sumscore (⩾10 of 27) as an alternative to MDE, AUC was about the same (Table 2). A specificity of 0.75 combined with a sensitivity of 0.75 or higher was not reached by any cut-off (Tables 3 and 4).

Table 2.

Area under the receiver operating curve (AUC) with 95% confidence interval (CI).

Items	AUC	95% CI
ESAS-Depression	0.71	0.66–0.76
Mean ESAS-Depression and ESAS-Anxiety	0.71	0.65–0.76
ESAS-Anxiety	0.67	0.62–0.73
ESAS-Depression when compared to PHQ-9 sumscore (⩾10 of 27)	0.72	0.68–0.76

ESAS: Edmonton Symptom Assessment System; PHQ-9: Patient Health Questionnaire-9.

Table 3.

Psychometrics (95% confidence intervals) for ESAS-Depression.

Cut-offs on ESAS-Depression	Coefficient κ	Sensitivity	Specificity	True positives^a	False positives^a	False negatives^a	True negatives^a
No cut-off		1.00	0.00	133	0	0	836
Cut-off ⩾ 1	0.11 (0.07–0.14)	0.79 (0.71–0.85)	0.46 (0.45–0.47)	105	448	28	388
Cut-off ⩾ 2	0.15 (0.10–0.19)	0.69 (0.61–0.76)	0.60 (0.56–0.61)	92	336	41	500
Cut-off ⩾ 3	0.22 (0.15–0.27)	0.61 (0.53–0.69)	0.73 (0.71–0.74)	81	233	52	613
Cut-off ⩾ 4	0.26 (0.19–0.33)	0.51 (0.43–0.59)	0.82 (0.81–0.83)	68	150	65	686
Cut-off ⩾ 5	0.25 (0.17–0.34)	0.38 (0.31–0.46)	0.88 (0.87–0.90)	51	98	82	738
Cut-off ⩾ 6	0.25 (0.17–0.34)	0.29 (0.22–0.35)	0.93 (0.92–0.95)	38	55	95	781
Cut-off ⩾ 7	0.17 (0.09–0.26)	0.17 (0.12–0.23)	0.96 (0.95–0.97)	23	35	110	801
Cut-off ⩾ 8	0.13 (0.60–0.20)	0.11 (0.07–0.15)	0.98 (0.96–0.99)	14	16	119	820
Cut-off ⩾ 9	0.09 (0.04–0.15)	0.07 (0.04–0.10)	0.99 (0.99–0.99)	9	7	124	829
Cut-off = 10	0.06 (0.02–0.08)	0.04 (0.02–0.05)	1.00	5	1	128	835

ESAS: Edmonton Symptom Assessment System; PHQ-9: Patient Health Questionnaire-9.

Patients report on 0–10 Numerical Rating Scale (NRS).

True positives: ESAS rated above NRS cut-off point. Major depressive episode present according to the PHQ-9 instrument. False positives: ESAS rated above NRS cut-off point. Major depressive episode not present according to the PHQ-9 instrument. False negatives: ESAS rated below NRS cut-off point. Major depressive episode present according to the PHQ-9 instrument. True negatives: ESAS rated below NRS cut-off point. Major depressive episode not present according to the PHQ-9 instrument.

Table 4.

Psychometrics (95% confidence intervals) for mean values of ESAS-Depression and ESAS-Anxiety combined.

Cut-offs on mean ESAS-Depression and ESAS-Anxiety	Coefficient κ	Sensitivity	Specificity	True positives^a	False positives^a	False negatives^a	True negatives^a
No cut-off		1.00	0.00	133	0	0	836
Cut-off ⩾ 0.5	0.07 (0.04–0.09)	0.85 (0.78–0.90)	0.34 (0.33–0.34)	113	556	20	280
Cut-off ⩾ 1.0	0.09 (0.05–0.12)	0.80 (0.72–0.86)	0.42 (0.40–0.43)	106	488	27	348
Cut-off ⩾ 1.5	0.12 (0.08–0.16)	0.74 (0.66–0.81)	0.53 (0.53–0.51)	98	397	35	439
Cut-off ⩾ 2.0	0.15 (0.11–0.20)	0.69 (0.61–0.76)	0.60 (0.59–0.62)	92	331	41	505
Cut-off ⩾ 2.5	0.19 (0.13–0.24)	0.62 (0.54–0.70)	0.69 (0.68–0.70)	83	260	50	576
Cut-off ⩾ 3.0	0.23 (0.17–0.29)	0.58 (0.50–0.66)	0.76 (0.75–0.77)	77	202	56	634
Cut-off ⩾ 3.5	0.25 (0.18–0.32)	0.52 (0.43–0.59)	0.81 (0.80–0.82)	68	160	65	676
Cut-off ⩾ 4.0	0.26 (0.18–0.33)	0.48 (0.40–0.56)	0.83 (0.82–0.85)	64	139	69	697
Cut-off ⩾ 4.5	0.26 (0.18–0.34)	0.41 (0.33–0.48)	0.87 (0.86–0.89)	54	106	79	730
Cut-off ⩾ 5.0	0.27 (0.18–0.35)	0.37 (0.30–0.44)	0.90 (0.89–0.91)	49	85	84	751
Cut-off ⩾ 6.0	0.24 (0.16–0.33)	0.24 (0.18–0.30)	0.95 (0.94–0.96)	32	39	101	797
Cut-off ⩾ 7.0	0.18 (0.11–0.26)	0.15 (0.11–0.20)	0.98 (0.97–0.99)	20	18	113	818
Cut-off ⩾ 8.0	0.13 (0.06–0.19)	0.10 (0.06–0.13)	0.99 (0.98–0.99)	13	12	120	824
Cut-off ⩾ 9.0	0.09 (0.04–0.13)	0.06 (0.03–0.08)	1.00	8	4	125	832
Cut-off = 10	0.03 (0.00–0.03)	0.02 (0.00–0.02)	1.00	2	0	131	836

ESAS: Edmonton Symptom Assessment System; PHQ-9: Patient Health Questionnaire-9.

Patients report on 0–10 Numerical Rating Scale (NRS).

An adequate sensitivity of 0.85 was reached for mean ESAS-Depression and ESAS-Anxiety at the cut-off of NRS ⩾ 0.5. Specificity at this cut-off was low 0.34, which means that 66% of patients without MDE would be further evaluated for MDE (false positives).

At the cut-off of NRS ⩾ 2 both for the ESAS-Depression item and for mean ESAS-Depression and ESAS-Anxiety, the present sensitivity was 0.69 and specificity was 0.60 (Tables 3 and 4). At the cut-off of NRS ⩾ 4, sensitivity was 0.51 and specificity was 0.82 for ESAS-Depression; for mean ESAS-Depression and ESAS-Anxiety, sensitivity was 0.48 and specificity was 0.83 (Tables 3 and 4).

The highest sensitivity reached by the ESAS-Depression item was 0.79 with a specificity of 0.46 at the cut-off of NRS ⩾ 1. At this most liberal cut-off, 21% of the patients with PHQ-9-MDE would be overlooked and 54% of the patients without PHQ-9-MDE would be further evaluated for MDE.

Coefficient κ

Coefficient κ was in the range of 0.03–0.26 for all cut-offs of the ESAS items (Tables 3 and 4). This represents poor to slight agreement between the ESAS items and PHQ-9-MDE, for all cut-off points.

Discussion

ESAS-Depression had limited screening ability for MDE when MDE was assessed by the PHQ-9 instrument in patients with incurable cancer. AUC was 0.71, and either sensitivity or specificity was inadequate depending on the 0–10 NRS cut-off point. Combining an anxiety item with the depression item made no improvement. Using ROC curves, there were no satisfactory cut-off values for either detection or exclusion of PHQ-9-MDE. Assuming the PHQ-9 instrument assesses MDE adequately in the population, the ESAS-Depression item appears insufficient to screen for MDE.

Former suggestions of applying ESAS-Depression with a cut-off point of NRS ⩾ 2 or NRS ⩾ 4 provide little help for screening MDE according to this study. Applying the ESAS-Depression cut-off point of NRS ⩾ 2 as advocated by Vignaroli et al.³³ and Bagha et al.,⁴² this study revealed a sensitivity of 0.69 and a specificity of 0.60 which are inadequate both for exclusion and inclusion of MDE. The suggested cut-off point of NRS ⩾ 4 by Ripamonti et al.³⁷ provided an inadequate sensitivity of 0.51 and a specificity of 0.82 in this study (Table 3).

Some other studies on patients with advanced cancer have been published. Teunissen et al.³⁴ (n = 54) found limited screening ability comparing ESAS-Depression with HADS (HADS-D ⩾ 11). Delgado-Guay et al.³⁵ (n = 216) explored the association between ESAS-Depression and HADS-Depression and found a low Spearman correlation of 0.39. Carvajal et al.⁴⁰ (n = 90) found a weighted coefficient kappa of 0.32 between ESAS-Depression and Rotterdam Symptom Checklist (RSCL). The authors, though, concluded that the ESAS is valid for screening purposes.

Patients with advanced cancer describe the ESAS-Depression and ESAS-Anxiety items difficult to rate;^58,59 the terms ‘depression’ and ‘anxiety’ were perceived unspecific. An unclear perception of the term ‘depression’ might indicate more underlying concepts and different understandings in patients with incurable cancer. Resistance against reporting on psychiatric disorders is a general finding⁶⁰ and may influence scorings.⁵⁹ Selby et al.⁶¹ found a floor effect with discrepancy between low ratings of ESAS-Depression and ESAS-Anxiety and high emotional impact and burden. The ESAS-Depression and ESAS-Anxiety items were low rated in the actual study; mean ESAS-Depression was 1.9, and mean ESAS-Anxiety was 2.1. Noguera et al.³⁶ (n = 100) found a deflated floor effect and better screening performance by use of the more colloquial term ‘discouraged’ compared to ‘depressed’ in translating ESAS into Spanish. Refinement of wordings might have screening potentials not yet inquired in patients with advanced cancer.

Content of the terms ‘depression’ and ‘anxiety’ seems differently loaded in cultures.^14,36,62 The actual study mainly included European patients; half of them were Norwegian. National differences cannot be concluded from the actual study due to insufficient sample sizes; however, the study by Noguera et al.³⁶ points to a need of thorough considerations of further cultural adjustments of ESAS-Depression.

Adding the ESAS-Anxiety scoring to ESAS-Depression gave about the same limited screening ability. With the limited screening properties of ESAS-Depression, an eventual increased validity of adding anxiety measurement cannot be evaluated and should be further investigated. Another solution to increase accuracy of simple screening instruments might be the combined assessment with other key symptoms than anxiety. Payne et al.⁶³ increased accuracy significantly by questioning the second main depressive symptom ‘Loss of interest’ to the simple question ‘Are you depressed?’ in a study in the palliative care context (sensitivity 0.91, specificity 0.68 and reference criterion MDD by Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV) psychiatric interview). Questioning the two main depressive symptoms by the DSM complete symptom descriptions with a 0–10 NRS response has not been examined in patients with advanced cancer.

There are several adjusted versions of ESAS.²⁸ Inference from this study is limited to the ESAS version applied in the study. Regarding symptom time frame, the patients were asked about depression ‘just now’; the term ‘Depression’ was used, not ‘Depression = feeling sad’ as in the ESAS revised version.⁶⁴

A prerequisite for the inferences of the screening performance explored in this study is the adequacy of the PHQ-9 instrument as a reference standard for MDE. The revised DSM classification, DSM-5, promotes PHQ-9 to be investigated as an added severity measurement of MDE.⁷ The instrument is increasingly used in clinical studies, also in patients with cancer.^12,42,65,66 PHQ-9 is applied in clinical assessment throughout the cancer trajectory.⁴² Among the applied translated versions of PHQ-9, validity is only examined for the German version^51,53 and not for the Norwegian and Spanish versions. The PHQ-9 patients’ report has shown strong agreement with MDE assessed by psychiatric interviews as the gold standard of diagnosing in several patient populations;^53,67–69 however, some studies found limited detection performance of PHQ-9.^70,71 Heterogeneity of populations is one proposed condition to variations in performance of the PHQ-9.⁷² PHQ-9 is not validated in patients with advanced cancer. Luckett et al.⁷³ in a review commented on five of the nine PHQ-9 questions as problematic in a palliative care population: sleep, fatigue, appetite, concentration and restlessness. The problem with symptom overlap between depression and cancer affects the psychiatric classification system itself applied in patients with advanced cancer.^7,45,74 Palliative care guidelines define the psychiatric classification systems as the clinical standard.^11,14 The diagnostic algorithm might, however, reduce overestimation of MDE by the overlapping somatic depressive and cancer symptoms.⁷⁵ The prevalence rate of MDE measured by PHQ-9 in this study was 13.7%. This is in line with prevalence rates of major depression in patients with advanced cancer^2,6 and may indicate that the use of PHQ-9-MDE is acceptable in the study.

Strengths of the study include international multinational collaboration, inclusion of patients across countries and cultures, a large sample size of patients with advanced cancer and blinded reports due to computer-based assessments.

Limitations

Inference from this study can only be drawn for patients with incurable cancer. The inclusion was performed conveniently and not consecutively which might have skewed the sample and introduced a healthy bias. The frailest patients did not fill in the PROs supporting this interpretation. Still, the lowered performance status and the survival estimates indicate the sample is representative for patients with incurable cancer in an early palliative phase. About 50% of the patients were Norwegian, and the results might, therefore, not be generalizable for all countries. The wording of the item in Norwegian might not be optimal and need reconsideration. Another limitation is the use of PHQ-9 as the external criterion for MDE and not a diagnosis based on a psychiatric diagnostic interview. This was done for resource purposes. Still, the PHQ-9 asks for the DSM-5 criteria for MDE, and the direction of an eventual mode of administration effect is not possible to estimate.

Conclusion

ESAS-Depression seems to measure a construct other than PHQ-9-MDE in patients with advanced cancer. The underlying construct of ESAS-Depression should be further investigated in patients with advanced cancer as should the validity of the PHQ-9 as a determinant of MDE in this patient group. Cultural and translational considerations should be addressed.

Footnotes

Acknowledgements

The authors wish to thank Even Fyllingen for important assistance in the development of the computer programme and Turid Follestad for statistical assistance. We thank all participating centres for important contributions.

Core scientific group/work package leaders: Stein Kaasa (project coordinator), Frank Skorpen, Marianne Jensen Hjermstad and Jon Håvard Loge, Norwegian University of Science and Technology (NTNU); Geoffrey Hanks, University of Bristol; Augusto Caraceni and Franco De Conno, Fondazione IRCCS Istituto Nationale dei Tumori, Milan; Irene Higginson, King’s College London; Florian Strasser, Cantonal Hospital St Gallen; Lukas Radbruch, RWTH Aachen University; Kenneth Fearon, University of Edinburgh; Hellmut Samonigg, Medical University of Graz; Ketil Bø, Trollhetta AS, Norway; Michael Schaude, Bender MedSystems GMbH, Austria; Odd Erik Gundersen, Verdande Technology AS, Norway. Scientific advisory group: Neil Aaronson, The Netherlands Cancer Institute; Vickie Baracos and Robin Fainsinger, University of Alberta; Patrick C. Stone, St George’s University of London; Mari Lloyd Williams, University of Liverpool. Project management: Stein Kaasa, Ola Dale and Dagny F. Haugen, NTNU.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This study was supported by the European Commission’s Sixth Framework Programme (contract no LSHC-CT-2006-037777), PI Stein Kaasa, with the overall aim to improve treatment of pain, depression and fatigue through translation research and funding from Trondheim University Hospital, Norway. Elisabeth Brenne received a grant from the Norwegian Cancer Society.

References

Teunissen

Wesker

Kruitwagen

. Symptom prevalence in patients with incurable cancer: a systematic review. J Pain Symptom Manage 2007; 34(1): 94–104.

Mitchell

Chan

Bhatti

. Prevalence of depression, anxiety, and adjustment disorder in oncological, haematological, and palliative-care settings: a meta-analysis of 94 interview-based studies. Lancet Oncol 2011; 12(2): 160–174.

Slade

Andrews

. Latent structure of depression in a community sample: a taxometric analysis. Psychol Med 2005; 35(4): 489–497.

Solano

Gomes

Higginson

. A comparison of symptom prevalence in far advanced cancer, AIDS, heart disease, chronic obstructive pulmonary disease and renal disease. J Pain Symptom Manage 2006; 31(1): 58–69.

Andrews

Brugha

Thase

. Dimensionality and the category of major depressive episode. Int J Methods Psychiatr Res 2007; 16(Suppl. 1): S41–S51.

Hotopf

Chidgey

Addington-Hall

.Depression in advanced disease: a systematic review. Part 1. Prevalence and case finding. Palliat Med 2002; 16(2): 81–97.

American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition. Arlington, VA, American Psychiatric Association, 2013.

World Health Organization. The ICD-10 classification of mental and behavioral disorders: diagnostic criteria for research. Geneva: World Health Organization, 1993.

Andrade

Caraveo-Anduaga

Berglund

. The epidemiology of major depressive episodes: results from the International Consortium of Psychiatric Epidemiology (ICPE) Surveys. Int J Methods Psychiatr Res 2003; 12(1): 3–21.

10.

Rodin

Lloyd

Katz

. The treatment of depression in cancer patients: a systematic review. Support Care Cancer 2007; 15(2): 123–136.

11.

Rayner

Price

Hotopf

. The development of evidence-based European guidelines on the management of depression in palliative cancer care. Eur J Cancer 2011; 47(5): 702–712.

12.

Rayner

Lee

Price

. The clinical epidemiology of depression in palliative care and the predictive value of somatic symptoms: cross-sectional survey with four-week follow-up. Palliat Med 2011; 25(3): 229–241.

13.

Greenhalgh

Long

Flynn

. The use of patient reported outcome measures in routine clinical practice: lack of impact or lack of theory? Soc Sci Med 2005; 60(4): 833–843.

14.

National Comprehensive Cancer Network. National Comprehensive Cancer Network clinical practice guidelines in oncology: distress management, 2014, www.nccn.org/professionals/physician_gls/f_guidelines.asp (accessed 2015).

15.

Cleeland

Sloan

. Assessing the symptoms of cancer using patient-reported outcomes (ASCPRO): searching for standards. J Pain Symptom Manage 2010; 39(6): 1077–1085.

16.

Kaasa

Radbruch

. Palliative care research – priorities and the way forward. Eur J Cancer 2008; 44(8): 1175–1179.

17.

Fiscella

Ransom

Jean-Pierre

. Patient-reported outcome measures suitable to assessment of patient navigation. Cancer 2011; 117(15 Suppl.): 3603–3617.

18.

Jordhoy

Kaasa

Fayers

. Challenges in palliative care research; recruitment, attrition and compliance: experience from a randomized controlled trial. Palliat Med 1999; 13(4): 299–310.

19.

Mitchell

. Are one or two simple questions sufficient to detect depression in cancer and palliative care? A Bayesian meta-analysis. Br J Cancer 2008; 98(12): 1934–1943.

20.

Vodermaier

Linden

Siu

. Screening for emotional distress in cancer patients: a systematic review of assessment instruments. J Natl Cancer Inst 2009; 101(21): 1464–1488.

21.

Bruera

Kuehn

Miller

. The Edmonton Symptom Assessment System (ESAS): a simple method for the assessment of palliative care patients. J Palliat Care 1991; 7(2): 6–9.

22.

Wasteson

Brenne

Higginson

. Depression assessment and classification in palliative cancer patients: a systematic literature review. Palliat Med 2009; 23(8): 739–753.

23.

Chochinov

Wilson

Enns

. ‘Are you depressed?’ Screening for depression in the terminally ill. Am J Psychiatry 1997; 154(5): 674–676.

24.

Lloyd-Williams

Dennis

Taylor

. A prospective study to compare three depression screening tools in patients who are terminally ill. Gen Hosp Psychiatry 2004; 26: 384–389.

25.

Kawase

Karasawa

Shimotsu

. Evaluation of a one-question interview for depression in a radiation oncology department in Japan. Gen Hosp Psychiatry 2006; 28(4): 321–322.

26.

Warmenhoven

van Rijswijk

Engels

. The Beck Depression Inventory (BDI-II) and a single screening question as screening tools for depressive disorder in Dutch advanced cancer patients. Support Care Cancer 2012; 20(2): 319–324.

27.

Taylor

Lovell

Ward

. Diagnosis of depression in patients receiving specialist community palliative care: does using a single screening question identify depression otherwise diagnosed by clinical interview? J Palliat Med 2013; 16(9): 1140–1142.

28.

Nekolaichuk

Watanabe

Beaumont

. The Edmonton Symptom Assessment System: a 15-year retrospective review of validation studies (1991–2006). Palliat Med 2008; 22(2): 111–122.

29.

Richardson

Jones

. A review of the reliability and validity of the Edmonton Symptom Assessment System. Curr Oncol 2009; 16(1): 55.

30.

Cummings

Biondo

Campbell

. Can the global uptake of palliative care innovations be improved? Insights from a bibliometric analysis of the Edmonton Symptom Assessment System. Palliat Med 2011; 25(1): 71–82.

31.

Sigurdardottir

Kaasa

Rosland

. The European Association for Palliative Care basic dataset to describe a palliative care cancer population: results from an international Delphi process. Palliat Med 2014; 28(6): 463–473.

32.

Zigmond

Snaith

. The Hospital Anxiety and Depression Scale. Acta Psychiatr Scand 1983; 67: 361–370.

33.

Vignaroli

Pace

Willey

. The Edmonton Symptom Assessment System as a screening tool for depression and anxiety. J Palliat Med 2006; 9(2): 296–303.

34.

Teunissen

de Graeff

Voest

. Are anxiety and depressed mood related to physical symptom burden? A study in hospitalized advanced cancer patients. Palliat Med 2007; 21(4): 341–346.

35.

Delgado-Guay

Parsons

. Symptom distress in advanced cancer patients with anxiety and depression in the palliative care setting. Support Care Cancer 2009; 17(5): 573–579.

36.

Noguera

Centeno

Carvajal

. Spanish ‘fine tuning’ of language to describe depression and anxiety. J Palliat Med 2009; 12(8): 707–712.

37.

Ripamonti

Bandieri

Pessi

. The Edmonton Symptom Assessment System (ESAS) as a screening tool for depression and anxiety in non-advanced patients with solid or haematological malignancies on cure or follow-up. Support Care Cancer 2014; 22(3): 783–793.

38.

De Haes

van Knippenberg

Neijt

. Measuring psychological and physical distress in cancer patients: structure and application of the Rotterdam Symptom Checklist. Br J Cancer 1990; 62(6): 1034–1038.

39.

Philip

Smith

Craft

. Concurrent validity of the modified Edmonton Symptom Assessment System with the Rotterdam Symptom Checklist and the Brief Pain Inventory. Support Care Cancer 1998; 6(6): 539–541.

40.

Carvajal

Centeno

Watson

. A comprehensive study of psychometric properties of the Edmonton Symptom Assessment System (ESAS) in Spanish advanced cancer patients. Eur J Cancer 2011; 47(12): 1863–1872.

41.

Spitzer

Kroenke

Williams

. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. JAMA 1999; 282(18): 1737–1744.

42.

Bagha

Macedo

Jacks

. The utility of the Edmonton Symptom Assessment System in screening for anxiety and depression. Eur J Cancer Care 2013; 22: 60–69.

43.

Jimenez

Madero

Alonso

. Symptom clusters in advanced cancer. J Pain Symptom Manage 2011; 42(1): 24–31.

44.

Kirkova

Aktas

Walsh

. Cancer symptom clusters: clinical and research methodology. J Palliat Med 2011; 14(10): 1149–1166.

45.

Brenne

Loge

Kaasa

. Depressed patients with incurable cancer: which depressive symptoms do they experience? Palliat Support Care 2013; 11(6): 491–501.

46.

Hjermstad

Lie

Caraceni

. Computer-based symptom assessment is feasible in patients with advanced cancer: results from an international multicenter study, the EPCRC-CSA. J Pain Symptom Manage 2012; 44(5): 639–654.

47.

Knudsen

Brunelli

Kaasa

. Which variables are associated with pain intensity and treatment response in advanced cancer patients? – implications for a future classification system for cancer pain. Eur J Pain 2011; 15(3): 320–327.

48.

Fearon

Strasser

Anker

. Definition and classification of cancer cachexia: an international consensus. Lancet Oncol 2011; 12(5): 489–495.

49.

Solheim

Blum

Fayers

. Weight loss, appetite loss and food intake in cancer patients with cancer cachexia: three peas in a pod? – analysis from a multicenter cross sectional study. Acta Oncol 2014; 53(4): 539–546.

50.

Capital Health. Edmonton Symptom Assessment Scale (ESAS), 2001, http://www.npcrc.org/files/news/edmonton_symptom_assessment_scale.pdf (accessed 2008).

51.

Pfizer. Patient Health Questionnaire (PHQ) Screeners, 2008, http://www.phqscreeners.com/

52.

Bland

Altman

. Survival probabilities (the Kaplan-Meier method). BMJ 1998; 317(7172): 1572.

53.

Löwe

Spitzer

Grafe

. Comparative validity of three screening questionnaires for DSM-IV depressive disorders and physicians’ diagnoses. J Affect Disord 2004; 78(2): 131–140.

54.

Swets

. Measuring the accuracy of diagnostic systems. Science 1988; 240(4857): 1285–1293.

55.

Fischer

Bachmann

Jaeschke

. A readers’ guide to the interpretation of diagnostic test properties: clinical example of sepsis. Intensive Care Med 2003; 29(7): 1043–1051.

56.

Cohen

. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 1968; 70(4): 213–220.

57.

Landis

Koch

. The measurement of observer agreement for categorical data. Biometrics 1977; 33(1): 159–174.

58.

Watanabe

Nekolaichuk

Beaumont

. The Edmonton symptom assessment system-what do patients think? Support Care Cancer 2009; 17(6): 675–683.

59.

Bergh

Kvalem

Aass

. What does the answer mean? A qualitative study of how palliative cancer patients interpret and respond to the Edmonton Symptom Assessment System. Palliat Med 2011; 25: 716–724.

60.

Lasalvia

Van Weeghel

Reneses

. Discrimination reported by people with major depressive disorder – authors’ reply. Lancet 2013; 381(9873): 1181–1182.

61.

Selby

Chakraborty

Myers

. High scores on the Edmonton Symptom Assessment Scale identify patients with self-defined high symptom burden. J Palliat Med 2011; 14(12): 1309–1316.

62.

Holland

Bultz

; National comprehensive Cancer Network. The NCCN guideline for distress management: a case for making distress the sixth vital sign. J Natl Compr Canc Netw 2007; 5(1): 3–7.

63.

Payne

Barry

Creedon

. Sensitivity and specificity of a two-question screening tool for depression in a specialist palliative care unit. Palliat Med 2007; 21(3): 193–198.

64.

Watanabe

Nekolaichuk

Beaumont

. The Edmonton Symptom Assessment System, a proposed tool for distress screening in cancer patients: development and refinement. Psychooncology 2012; 21(9): 977–985.

65.

Pirl

Greer

Traeger

. Depression and survival in metastatic non-small-cell lung cancer: effects of early palliative care. J Clin Oncol 2012; 30(12): 1310–1315.

66.

Baker-Glenn

Park

Granger

. Desire for psychological support in cancer patients with depression or distress: validation of a simple help question. Psychooncology 2011; 20(5): 525–531.

67.

Kroenke

Spitzer

Williams

. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001; 16(9): 606–613.

68.

Diez-Quevedo

Rangil

Sanchez-Planell

. Validation and utility of the patient health questionnaire in diagnosing mental disorders in 1003 general hospital Spanish inpatients. Psychosom Med 2001; 63(4): 679–686.

69.

Williams

Brizendine

Plue

. Performance of the PHQ-9 as a screening tool for depression after stroke. Stroke 2005; 36(3): 635–638.

70.

Carballeira

Dumont

Borgacci

. Criterion validity of the French version of Patient Health Questionnaire (PHQ) in a hospital department of internal medicine. Psychol Psychother 2007; 80(Pt 1): 69–77.

71.

Thekkumpurath

Walker

Butcher

. Screening for major depression in cancer outpatients: the diagnostic accuracy of the 9-item patient health questionnaire. Cancer 2011; 117(1): 218–227.

72.

Wittkampf

Naeije

Schene

. Diagnostic accuracy of the mood module of the Patient Health Questionnaire: a systematic review. Gen Hosp Psychiatry 2007; 29(5): 388–395.

73.

Luckett

Butow

King

. A review and recommendations for optimal outcome measures of anxiety, depression and general distress in studies evaluating psychosocial interventions for English-speaking adults with heterogeneous cancer diagnoses. Support Care Cancer 2010; 18(10): 1241–1262.

74.

Kathol

Mutgi

Williams

. Diagnosis of major depression in cancer patients according to four sets of criteria. Am J Psychiatry 1990; 147(8): 1021–1024.

75.

Lie

Hjermstad

Fayers

. Depression in advanced cancer – assessment challenges and associations with disease load. J Affect Disord 2015; 173: 176–184.