Validation of the K6/K10 Scales of Psychological Distress and Their Optimal Cutoff Scores for Older Koreans

Abstract

This study aims to assess the psychometric properties of the Korean versions of the Kessler 6 and 10 (K-K6/K10) to validate the two scales in relation to the Korean versions of the Geriatric Depression Scale-Short Form (K-GDS-SF) and the Center for Epidemiological Studies Depression Scale (K-CES-D) and to propose optimal cutoff scores appropriate for the elderly Korean population (n = 331). Psychometric performance was assessed with sensitivity, specificity, receiver operating characteristic curves, and area under the receiver operating characteristic curve. Cutoff scores were estimated from the Youden index, the closest-to-(0,1), and the balanced score. This study found that the K-K6 and K-K10 appeared to be robust screening tools, with the K-CES-D and K-GDS-SF as reference measures. The cutoff scores for the Korean senior population are proposed to be 12/13 for the K-K6 and 20/21 for the K-K10. This study indicates that the K6 and K10 can be valid and reliable screening tools of psychological distress for Korean seniors and may also be used among the Korean American elderly in the United States for both clinical and research purposes.

Keywords

depression Kessler K6/K10 Korean screening tool psychological distress

Introduction

Growing Importance and Risk of Mental Health Problems in the Elderly

Research and service development for geriatric disorders are garnering greater attention than ever before with the rapid aging of the population across the world. Mental health and emotional well-being are as important in older age as in any other time of life. Neuropsychiatric disorders among people over the age of 60 account for 6.6% of the disability adjusted life years for this age-group, and approximately 15% of adults in this age-group suffer from a mental disorder (Institute for Health Metrics and Evaluation, 2015). Among the common mental health problems affecting this population are depression and dementia, along with anxiety, substance abuse, and self-harm (World Health Organization [WHO], 2013). Poor mental health has an impact on physical health and vice versa, thus reducing the quality of life in older adults (Casey, 2012; Curtis & Baldwin, 2009). Mental health problems later in life often result in significant social and medical costs, as well as physical and emotional stress to families, often preventing adults from making meaningful contributions to society later in life (WHO, 2013).

Korea is a country whose population over the age of 65 is growing at one of the fastest rates. The number of people aged 65 and older has now reached 6.14 million, comprising 12.2% of the total population (Statistics Korea, 2013), and is projected to reach 24.3% in 2030 and 37.4% in 2050 (Statistics Korea, 2011). Given the rapid aging of the population, the public health burden of late-life mental health problems will likely grow as well, suggesting the importance of continued monitoring of the mental health status of adults in this age-group. Many adults lose the ability to live independently later in life due to limited mobility, chronic pain, frailty, or other mental and physical problems (WHO, 2013). Compared with other age-groups, they are more likely to experience loss of economic security, social status, and social support. The mental health of older Korean adults is further affected by the drastic sociocultural changes that have occurred over the past several decades. Unlike their parents’ generation, many aging Koreans feel pressured to be able to take care of themselves, instead of allowing their adult children to care for them in return for the devotion of most of their resources in life. Feeling uncertain of on whom to rely in the case that they lose independence seems difficult for older Koreans, causing them to feel increased anxiety.

Among people over the age of 65 in Korea, 43.0% reported stress in their overall life in 2008 and 48.6% in 2010, and this percentage continued to increase to 52.9% in 2012 (Statistics Korea, 2013). Health care utilization data (National Health Insurance Service, 2013) show that the number of outpatient visits by Korean adults aged 18 and over for depression (primary diagnoses of ICD F32 and F33) (WHO, 1992) increased by an annual average of 2.2% between 2007 and 2011. During the same period, those aged 80 years and older showed the largest increase of 7.8%. The number of outpatient visits per 100,000 for anxiety (primary diagnoses of ICD F40 and F41) (WHO, 1992) was the highest in adults aged 70 and over (3,051), which was three times higher than that of adults under the age of 60 (877) (National Health Insurance Service, 2014). Moreover, the suicide rate of 59.8 per 100,000 among adults of 65 to 74 years in Korea is three times greater than the average of 18.9 among countries in the Organization for Economic Cooperation and Development (2009) and is now the fifth leading cause of death for Korean adults in this age-group (Song, Jin, Song, An, & Cho, 2013). All this evidence points to the increased importance of detecting and intervening early for mental health problems in this age-group in Korea.

Challenges in Detecting and Screening Mental Health Problems in Older Adults

Mental health is essential to overall health and well-being. Prompt recognition and treatment of mental health problems in older adults are a critical component of active and healthy aging. Despite the increased risk of mental health problems in old age, current efforts for screening, early detection, diagnosis, and treatment of depression in the elderly still lag behind those for other age-groups (Lewis, Hems, Bosanquet, & Overend, 2013; WHO, 2012). One factor contributing to this is that symptoms such as subthreshold depression in older age are often overlooked by professionals who consider them to be natural processes associated with aging (Casey, 2012). Older adults perceive psychological symptoms as signs of personal weakness and shame (Jang, Chiriboga, & Okazaki, 2009) or as part of aging, which makes them less likely to seek professional help for psychological distress (Cho et al., 2011) and prevents them from expressing the need for help and receiving accurate assessment and timely intervention. In addition, gerontological/geriatric paraprofessionals providing direct care to older adults in various residential and community settings may not be equipped to detect and assess changes in their psychological status in a timely manner (Lee & Ko, 2009), further delaying appropriate interventions (Huang & Carpenter, 2011).

Current Status of Screening Tools for Mental Health Problems in Older Adults in Korea

With the passage of mental health legislation in 1995, the Korean government began to establish community mental health services. Since the mid-2000s, mental health promotion and the prevention of depression and suicide have been at the core of such efforts, in response to reports that Korea had the highest rate of suicide death among Organization for Economic Cooperation and Development countries (e.g., Lee & Suh, 2009). Community mental health services for older adults are thus focused on the prevention of depression and suicide, including education and public campaigns, identification and treatment referrals of high-risk groups, and education of potential gatekeepers such as providers from other human service organizations.

With emphasis on depression and suicide prevention, the most widely used screening tools of mental health symptoms for older adults in Korea include the Geriatric Depression Scale-Short Form (GDS-SF) (Bae & Cho, 2004; Jung et al., 1997; Kim et al., 2008; Yesavage et al., 1983), the Center for Epidemiological Studies Depression Scale (Cho & Kim, 1993, 1998; Radloff, 1977), and the Beck Depression Inventory (Beck, Ward, Mendelson, Mock, & Erbaugh, 1961; Han et al., 1986; Lee & Song, 1991). These screening tools have been validated with Korean populations, but the tools contain too many items to be used efficiently in the field (15, 20, and 21, respectively). Difficulty in understanding the items in the tools may not facilitate their use among older adults whose cognitive capacities may be in decline. This reduces the likelihood of obtaining reliable or valid responses to the items. While symptoms of depression often are perceived as measures of overall health status or quality of life in older adults, other symptoms frequently observed in this age-group, such as anxiety, need to be identified as well. In light of some of the challenges with the existing screening tools and the need for a measure detecting symptoms of depression as well as anxiety, the introduction of a measure to assess overall psychological distress may be necessary for the early detection of and intervention for mental health problems in older adults. It also would be beneficial to introduce a screening tool that could be administered easily, even by paraprofessionals in the fields of care-giving and social services for older adults.

The Kessler 6/10 Scale

The Kessler 10 (K10), of which the Kessler 6 (K6) is a subset, was originally developed as a brief and accurate distress scale to screen for serious mental illness (SMI) for possible use in the Substance Abuse and Mental Health Services Administration National Household Survey on Drug Abuse in the United States. The 12-month structured clinical interview for DSM-IV (SCID) and the Global Assessment of Functioning are used as the gold standards for the diagnosis of SMI. According to these standards, SMI is defined as a condition meeting the criteria for at least one of the DSM-IV/SCID diagnoses other than a substance use disorder, and having a Global Assessment of Functioning score less than 60 (Kessler et al., 2003). The results showed that the area under the receiver operating characteristic curve (AUC) of SMI was 0.865 for the K6 and 0.854 for the K10. It was found that the K6/K10 performed more efficiently in terms of time and cost than the WHO Composite International Diagnostic Interview (CIDI), which has been used in epidemiological surveys worldwide (Kessler et al., 2003, 2010).

The performances of the K6 and K10 in detecting CIDI/DSM-IV mood and anxiety disorders have been evaluated in Australians (Furukawa, Kessler, Slade, &Andrews, 2003), Japanese (Furukawa et al., 2008), and a Chinese population in Hong Kong (Lee et al., 2012). The results appear to provide evidence for strong psychometric properties of the K10 as a reliable and valuable tool for the screening of SMI, surveillance of behavioral risk factors, and community epidemiological surveys. Similar results were reported for the K6 in Australians and Japanese, but the results were less than satisfactory in the Chinese group. In addition, the assessment of the K6/K10 in cross-cultural settings of 14 countries indicated an overall high psychometric performance (Kessler et al., 2010). When the K6 was evaluated in reference to DSM-IV/CIDI in the World Mental Health survey, the scale showed good properties across a wide range of countries, with the range of AUCs being 0.76 to 0.89. Also, the K6 showed a minimal level of bias with regard to age, gender, and education (Kessler et al., 2010). However, while there is evidence for the use of the K6 and K10 as reliable and valid screening tools for DSM-IV disorders in more than a dozen countries, no K6 or K10 validation study has been conducted for the Korean population.

Therefore, in light of the need to better address increasing mental health challenges with a more efficient and feasible screening tool for psychological distress among the older population in Korea, this study aims to assess the psychometric properties of the Korean versions of the K6 and K10, to validate the two scales in relation to the two most widely used scales of mental illness for older Korean adults (i.e., the Korean-Geriatric Depression Scale-Short Form [K-GDS-SF] and the Korean-Center for Epidemiological Studies-Depression [K-CES-D]), and to propose optimal cutoff scores appropriate for the older Korean population.

Methods

Participants

A cross-sectional survey study was conducted with seven senior community centers throughout the Seoul Metropolitan area and the Kyunggi Province in Korea. Potential participants were initially identified with the help of gerontological social workers at the centers, based on the inclusion criteria of (a) being 60 years old or older, (b) being a current client of the center, (c) having the ability to communicate, and (d) having the ability to understand and complete a questionnaire and provide informed consent. The identified participants were then contacted by trained research assistants. The research assistants explained the purpose and procedures of the study to the potential participants and asked them if they agreed to participate. All agreeing participants then provided informed consent. The research assistants administered the questionnaires and conducted interviews with the participants. Approximately, 500 seniors were reached, and a total of 331 seniors agreed to participate and completed the questionnaire. The questionnaire had 67 items from various scales (i.e., the K-K6/K10, K-GDS-SF, and K-CES-D), along with seven basic questions regarding demographic characteristics. On average, a survey interview could be completed in about 30 to 40 minutes. Participants received in-kind compensation worth 5,000 won (around $5.00) as a token of appreciation for their participation. All research protocols for the recruitment, consent, and interview procedures were reviewed and approved by the institutional review board of the institution with which one of the authors is affiliated.

Measures

Diagnostic assessment

As reference measures for determining the presence (or caseness) of psychological symptoms, two scales that have been validated with the Korean population were used, that is, K-GDS-SF and K-CES-D. First, the K-GDS-SF is a tool used to screen for depression among older populations with a Yes/No response category (Sheikh & Yesavage, 1986). Based on the original 30-item GDS, a short form with 15 items was later developed with comparable performance to the original version (Sheikh &Yesavage, 1986). A score greater than 5 may be suggestive of depression, and a score of 10 or greater is almost always indicative of depression. The Korean versions of the GDS-30 and GDS-SF were validated by Bae and Cho (2004) as well as Cho et al. (1999), who suggested cutoff scores for depression of 10 and 8, respectively, for the Korean validated scales. In this study, a cutoff score of 8 was adopted based on the work of Cho et al. (1999) to identify potential cases of depression, so those with scores of 0 to 7 were considered “not depressed” and those with scores of 8 to 15 were considered “depressed.”

Second, the K-CES-D is a 20-item scale of depressive symptoms validated in the Korean population. Originally developed by Radloff (1977), the Center for Epidemiological Studies Depression Scale asks respondents to rate the frequency of symptoms they have experienced during the past week on a 4-point Likert scale ranging from “rarely or none of the time (less than 1 day)” to “most or all of the time (more than 5–7 days).” Radloff (1977) suggested a cutoff score of 16 to identify cases of depression. When the K-CESD-20 was validated by Cho and Kim (1993), two cutoff scores (21 and 25) were suggested for epidemiological surveys and clinical diagnoses, respectively. In this study, a cutoff score of 25 was used to identify clinical cases with a diagnosis of depression, so those with scores of 0 to 24 were considered “not depressed” and those with scores of 25 to 60 were considered “depressed.”

Development of the Korean Version of the K6 and K10

The K6 and K10 contain 6 and 10 questions, respectively, asking how the respondents have been feeling during the past 30 days. The K6 includes the items of “feeling nervous,” “feeling hopeless,” “feeling restless or fidgety,” “feeling depressed so that nothing could cheer you up,” “feeling that everything was an effort,” and “feeling worthless.” The K10 has four additional items: “feeling tired out for no good reason,” “feeling so nervous that nothing could calm you down,” “feeling so restless that you could not sit still,” and “feeling so sad that nothing could cheer you up.” Response categories are on a Likert scale with five categories (0 = all of the time, 1 = most of the time, 2 = some of the time, 3 = a little of the time, and 4 = none of the time). When the responses are reverse-scored, a total score ranges from 0 to 40 for the K10 and 0 to 24 for the K6, with a higher score representing a higher level of psychological distress. The optimal cutoff point is a score below 13 for the K6. No optimal cutoff point for the K10 has been suggested, although a score below 20 has been suggested based on an Australian population (Australian Department of Health, 2002).

To develop the Korean versions of the K6 and K10, the original English versions of the K6 and K10 were translated with back-translation methods. As a first step, the first author who was bilingual in English and Korean translated the K6 and K10 instruments into Korean. Then, the second author, who was also bilingual in English and Korean, back-translated the instruments from Korean to English. Subsequently, the original English and the back-translated Korean versions were examined for their consistency. This procedure was repeated three times to determine which expressions would be most appropriate and consistent with the original items. Some of the translated Korean expressions could be nuanced, with a possibility of having somewhat different meanings or intensities. For example, some variations were noted in two of the five response categories, “All of the time,” and “A little of the time.” “All of the time” was originally translated as “All throughout the 30 days,” and “A little of the time” was expressed as “Rarely” or “Almost none of the time” when back-translated into English. Among the items, “restless” was once translated as “anxious.” “Cheer you up” was translated as “making you feel better.” “Everything was an effort” was noted as “Everything was burdensome.” Finally, “worthless” was initially translated as “valueless.” As a final step, three doctoral students who previously or currently worked in the area of mental health verified the translated Korean versions of the K6 and K10.

Data Analysis

Analytic procedures for assessing the data on the psychometric performance of the Korean versions of the K6 and K10 scales (K-K6 and K-K10) were based on basic approaches to scale validation (DeLong, DeLong, & Clarke-Pearson, 1988). The internal consistency and convergent validity of the K-K6 and K-K10 were tested by Cronbach’s alpha and Pearson’s correlation coefficients, respectively. In addition, the caseness of “psychological distress” was determined by two reference scales, the K-CES-D and K-GDS-SF. The performance of the K-K6 and K-K10 as screening scales was assessed by calculating sensitivity, specificity, and receiver operating characteristic (ROC) curves (Hanley & McNeil, 1982). In the context of this study, sensitivity refers to the proportion of those who were correctly identified as “being distressed” (caseness), and specificity represents the proportion of those who were correctly identified as “not distressed” (noncases). The ROC curve is then constructed as a visual graph plotting sensitivity against (1-specificity), showing the diagnostic or discriminating capability of the measure. The AUCs and their 95% confidence intervals quantify the psychometric performances of the K6 and K10 in relation to the reference measures. A perfect performance is expected to have an AUC of 1, while a test with no diagnostic performance would have an AUC of 0.5 (a straight diagonal line).

Finally, three approaches based on ROC curves were used to determine cutoff values of the K-K6 and K-K10 for detecting or discriminating the presence of psychological distress because the use of only one approach may not always be consistent with the use of others (Perkins & Schisterman, 2006). The approaches used in this study included (a) the Youden index (Bewick, Cheek, & Ball, 2004; Youden, 1950), (b) the closest-to-(0,1) (Perkins & Schisterman, 2006), and (c) the criterion of a balanced score (Singer et al., 2009). A Youden index was obtained for potential cutoff values by calculating (sensitivity + specificity − 1). The index ranges from 0 (no diagnostic value) to 1 (perfect diagnostic value), indicating that the value with the highest Youden index would be the best cutoff value (Zweig & Campbell, 1993). The closest-to-(0,1) criterion is the shortest radius originating at the (0,1) point and terminating on the ROC curve, which minimizes misclassification (Perkins & Schisterman, 2006). It is based on the formula: (1-Sensitivity)²+ (1-Specificity)², with the smallest value being the best cutoff value. A balanced score refers to a cutoff score that suggests the best proportion of sensitivity and specificity, calculated as (sensitivity + specificity)/2 (Singer et al., 2009). SPSS for Windows V. 21 and Stata 12.1 were used to analyze the ROC curves and AUCs and to calculate cutoff scores for the three indices.

Results

Participant Characteristics

Table 1 presents the characteristics of the study participants. The participants for this study were 331 older individuals (60 years or older) living in the community. The vast majority of the participants (82.1%) were females, and the average age was 75.8 years (SD = 6.30). Approximately, 61% were widowed and another 30% were married. More than half of the participants (60.2%) reported either “no education” or “elementary schooling,” and participants had an average of 6.5 years of formal education. Approximately, 55% lived alone.

Table 1.

Participant Characteristics.

Characteristics	N	%
Gender
Male	58	17.9
Female	266	82.1
Age (M/SD, years)	(75.8/6.3)
Marital status
Single	7	2.2
Married	102	31.7
Widowed	196	60.9
Divorced	17	5.2
Education (M/SD, years)	(6.5/5.1)
No education	83	25.6
Elementary school	110	34.4
Middle school	43	13.4
High school	57	17.8
College or higher	28	8.8
Living arrangement
Living alone	178	54.8
Living with spouse	71	21.8
Living with spouse and children	31	9.5
Living with children	38	11.7
Other	7	2.2

Internal Consistency and Convergent Validity of the K-K6/K10

Table 2 reports the Pearson’s correlation coefficients and Cronbach’s alpha among the K-K6/K10, K-GDS-SF, and K-CES-D used in this study. Also presented are the means, standard deviations, and percentages of individuals classified as “depressed” based on the established cutoff values. Both the K-K6 and K-K10 total scores showed significant positive correlations with measures expected to represent similar constructs (the K-GDS-SF and K-CES-D). Cronbach’s alpha for the four measures ranged from .880 for the K-GDS-SF to .924 for the K-K10, indicating a high level of internal consistency among the items in each measure.

Table 2.

Pearson’s Correlation Coefficients and Cronbach’s Alpha: Four Depression Measures.

	K-K6	K-K10	K-GDS	K-CES-D	Cronbach’s alpha	M	SD	% Caseness
K-K6 (0–24)	–	.984***	.683***	.718***	.889	10.02	5.53	31.1^a
K-K10 (0–40)	–	–	.674***	.721***	.924	16.52	8.69	–
K-GDS-SF(0–15)	–	–	–	.714***	.880	5.99	4.43	36.5^b
K-CES-D (0–60)	–	–	–		.922	16.64	13.22	24.4^c

Based on the original cutoff point of the K6; those with scores of 13 or higher.

Based on the cutoff point of the K-GDS-SF; those with scores of 8–15.

Based on the cutoff point of the K-CES-D; those with scores of 25–60.

***

p < .001.

K-CES-D as a Reference Measure

Table 3 presents the sensitivity, specificity, and AUC, with the K-CESD as a reference measure. Also presented are three indices used to suggest cutoff values for the K-K6 and K-K10 scales. The AUCs of the K-K6 and K-K10 were .900 and .903, indicating strong performance of both scales in discriminating cases of “distress” as established by the K-CES-D. Figure 1 also shows the ROC curves for the K-K6 and K-K10 with the K-CES-D as the reference. No significant differences were found in the AUCs of the K-K6 and K-K10 (p = .6636), showing that the scales had equally robust psychometric performances.

Table 3.

AUC, Sensitivity, Specificity, and Three Indices for Cutoff Values Based on the K-CES-D.

	Areas under the ROC curve		Value	Sensitivity	Specificity	Youden index	The closest- to-(0,1)	Balanced
	Area (SE)	95% CI	Value	Sensitivity	Specificity	Youden index	The closest- to-(0,1)	Balanced
K-K6 scale	.900 (.022) p < .001	(.856 .943)	≥ 8	.97	.46	.43	.296	.71
			≥ 9	.94	.55	.48	.208	.74
			≥ 10	.92	.61	.53	.159	.77
			≥ 11	.92	.71	.63	.091	.82
			≥ 12	.84	.78	.62	.074	.81
			≥ 13	.78	.85	.63	.071	.82
			≥ 14	.75	.88	.63	.077	.82
			≥ 15	.68	.92	.60	.108	.80
			≥ 16	.54	.95	.49	.214	.75
K-K10 scale	.903 (.021) p < .001	(.861 .945)	≥ 16	.95	.64	.59	.136	.79
			≥ 17	.90	.68	.59	.112	.79
			≥ 18	.87	.72	.59	.095	.80
			≥ 19	.87	.76	.63	.075	.82
			≥ 20	.84	.81	.64	.062	.83
			≥ 21	.82	.85	.67	.055	.84
			≥ 22	.77	.87	.64	.070	.82
			≥ 23	.72	.89	.61	.090	.80
			≥ 24	.66	.93	.59	.123	.80
			≥ 25	.59	.95	.54	.171	.77
			≥ 26	.51	.95	.46	.244	.73
ROC comparison		chi-square (1) = 0.19, p = .6636

Note. CI= confidence interval; ROC = receiver operating characteristic.

Figure 1.

ROC curves of the K-K6 and K-K10 based on the K-CESD. ROC = receiver operating characteristic.

According to the results from the three indices used to determine cutoff values in relation to the K-CES-D, for the K-K6, a cutoff value of 13 or higher represented the highest value of the Youden index of .63, the smallest value of .071 for the closest-to-(1,0), and the highest balanced value of .82. The suggested cutoff value of 13 or higher showed a sensitivity of .78 and a specificity of .85. For the K-K10, a cutoff value of 21 or higher yielded the highest value of the Youden index of .67, the smallest value of .055 for the closest-to-(1,0), and the highest balanced value of .84. The value of 21 + had a sensitivity of .82 and a specificity of .85.

K-GDS-SF as a Reference Measure

Table 4 provides the sensitivity, specificity, and AUC of the K-K6 and K-K10, when the K-GDS-SF was used as the reference measure. Three indices for the suggested cutoff values are also provided. Results from the AUCs of the K-K6 and K-K10 indicated moderately strong diagnostic capacities of both scales to detect psychological distress as assessed by the K-GDS-SF, as indicated by their respective values of .867 and .863 (see Figure 2). No significant differences were found in the AUCs of the K-K6 and K-K10 (p = .4731).

Table 4.

AUC, Sensitivity, Specificity, and Three Indices for Cutoff Values Based on the K-GDS-SF.

	Area under the ROC curve		Value	Sensitivity	Specificity	Youden index	The closest- to-(0,1)	Balanced
	Area (SE)	95% CI	Value	Sensitivity	Specificity	Youden index	The closest- to-(0,1)	Balanced
K-K6 scale	.867 (.021) p < .001	(.825 .908)	≥7	.95	.42	.37	.338	.68
			≥8	.93	.50	.43	.252	.72
			≥9	.89	.60	.49	.172	.75
			≥10	.87	.67	.54	.126	.77
			≥11	.81	.78	.58	.085	.80
			≥12	.72	.82	.54	.111	.77
			≥13	.65	.88	.54	.137	.77
			≥14	.61	.92	.53	.158	.77
			≥15	.52	.93	.46	.232	.73
			≥16	.41	.96	.38	.346	.69
K-K10 scale	.863 (.022) p < .001	(.820 .905)	≥12	.96	.43	.40	.322	.70
			≥13	.83	.48	.40	.303	.65
			≥14	.91	.56	.47	.202	.73
			≥15	.88	.62	.49	.163	.75
			≥16	.85	.69	.54	.121	.77
			≥17	.81	.74	.55	.104	.78
			≥18	.78	.78	.56	.097	.78
			≥19	.76	.81	.58	.094	.79
			≥20	.71	.83	.54	.113	.77
			≥21	.67	.87	.54	.126	.77
			≥22	.61	.89	.50	.164	.75
			≥23	.59	.92	.50	.179	.75
			≥24	.49	.95	.44	.261	.72
			≥25	.43	.96	.39	.322	.70
			≥26	.96	.43	.40	.322	.70
ROC comparison		chi-square (1) = 0.51, p = .4731

Note. CI= confidence interval; ROC = receiver operating characteristic.

Figure 2.

ROC curves of the K-K6 and K-K10 based on the K-GDS-SF. ROC = receiver operating characteristic.

Regarding the potential cutoff value for the K-K6, a cutoff value of 11 or higher produced the highest value of the Youden index of .58, the smallest value of .085 for the closest-to-(1,0), and the highest balanced value of .80. The suggested cutoff value of 11 or higher showed a sensitivity of .81 and a specificity of .78. For the K-K10, a cutoff value of 19 or greater yielded the highest value of the Youden index of .58, the smallest value of .094 for the closest-to-(1,0), and the highest balanced value of .79. The value of 19 and higher had a sensitivity of .76 and a specificity of .81.

Discussion

This study examined the psychometric properties of the Korean versions of the K6 and K10 and assessed their validity as screening tools for psychological distress among Korean seniors, based on survey data from 331 respondents. The K6 and K10 were initially developed in the United States to screen for DSM-IV disorders in the community setting. This study found that the Korean versions of the K6/K10 appear to be robust screening tools for psychological distress, with the K-CES-D and K-GDS-SF as reference measures. In particular, with the K-CES-D as the reference, both the K-K6 and K-K10 had AUCs greater than .90, with no significant difference in AUC between the two scales. When the K-GDS-SF was used as the reference, the psychometric performance of both the K6 and K10 was at a moderately satisfactory level of .85, lower than those of the K-CES-D. However, again, no significant differences were found in the AUCs of the K6 and K10. Another finding was that the potential cutoff scores for the K-K6 were 12/13 with the K-CES-D as the reference, or 10/11 with the K-GDS-SF. For the K-K10, the cutoff scores were either 20/21 with the K-CES-D or 18/19 with the K-GDS-SF. Nevertheless because the AUC level was higher for the K-CES-D, this study proposes that the cutoff scores be 12/13 for the K-K6 and 20/21 for the K-K10 for the Korean senior population. The following is a more specific discussion of the significance, utility, and potential application of these results.

First, this study indicates that the K6 and K10 can be valid and reliable screening tools of psychological distress for Korean seniors. The strong performances of the K-K6 and K-K10 are consistent with previous studies on SMI (Kessler et al., 2003) and CIDI/DSM-IV mood or anxiety disorders (Furukawa et al., 2003, 2008) and are very similar to those found in the validation study with an Australian population (Furukawa et al., 2003) that reported an AUC of .90 for the K10 and .89 for the K6. In another validity study with a Japanese population, Furukawa et al. (2008) reported the excellent screening performance of the Japanese versions of the K6 and K10, equivalent to those of the original English versions. The AUCs were as high as 0.94 for the K6 and 0.94 for the K10. Of the K6 and K10, on the one hand, the K10 was preferred in an SMI study of a sample population from primary care settings with a higher proportion of clinically significant emotional problems (Kessler et al., 2003). On the other hand, with the Australian sample, the K6 was preferred in screening for any DSM-IV mood or anxiety disorder because of its brevity and consistency across subsamples (Furukawa et al., 2003). The two measures (K-CES-D and K-GDS-SF), which were used as references in the present study, still contain relatively large numbers of items for Korean older adults (20 and 15 items, respectively) (Cho et al., 1999; Cho & Kim, 1993), so the K6 may be suggested in our study as more useful than these two screening tools, primarily due to its equal psychometric performance and its brevity.

Second, this study proposes cutoff scores of 12/13 for the K-K6 and 20/21 for the K-K10. With respect to the cutoff score for the K-K6, many previous studies based on standard validation approaches reported subjects with scores of 13 to 24 as probably having SMI and those with scores of 0 to 12 as probably not having SMI (e.g., Kessler et al., 2003). The Hong Kong study (Lee et al., 2012) used this method and determined the cutoff to be 12/13 for the Chinese sample. Alternative approaches have been used to estimate cutoff scores, such as stratum-specific likelihood ratios based on sensitivity and specificity (Furukawa et al., 2003, 2008) and maximum-likelihood methods of estimation (Kessler et al., 2010). These alternative approaches were adopted to establish the population prevalence rate of SMI in the community epidemiological context, and for the purpose of policy planning. However, in this study, the cutoff scores for the K-K6/K10 were explored for the purpose of having a brief screening tool in the field to assess the need for follow-up through more rigorous and clinical assessment and diagnosis of SMI symptoms. The cutoff score for the K-K6 in this study is identical to those found in previous studies by Kessler et al. (2003) and Lee et al. (2012).

Third, our study adds to the evidence of the efficiency and utility of the K6/K10 in cross-national settings, in that the Korean version of the K6/K10 had somewhat higher AUCs (.90 with the K-CES-D as the reference measure and .87 with the K-GDS-SF) than those reported in the study of 14 nations (Kessler et al., 2010). The Korean version of the K6/K10 might be suitable for use in Korean Americans and elderly Korean immigrants in the United States. Studies of older adults with Korean cultural backgrounds have indicated that these adults report higher levels of psychological distress than older non-Hispanic White or other minority groups (Min, Moon, & Lubben, 2005; Sorkin, Nguyen, & Ngo-Metzger, 2011). Yet, their perception and attitude toward mental health issues, combined with the language barrier, tend to prevent them from identifying the problem and seeking professional help (Kim et al., 2010; Sorkin et al., 2011). Nevertheless, caution should be exercised in directly applying our suggested cutoff scores with Korean American and elderly Korean immigrants because of the potential incompatibility of the cutoff scores between Korean elders in Korea and Korean immigrants, due to migration and acculturation experiences (Plitas, Tucker, Kritikos, Walters, & Bardenhagen, 2009).

While the present study provides some promising evidence for the utility of the Korean version of the K6/K10, the following limitations should be noted. First, the study sample was not based on probability sampling. The sample was recruited from a pool of older adults living in the city of Seoul and the province of Kyunggi who were functionally able to visit and utilize aging services and programs at the senior center. Therefore, the findings may not be readily generalizable to older adults who are home bound or too frail to function independently nor to those living in other regions in Korea. Also, due to the relatively small sample size of the convenience sample in this study, further research should be conducted to reaffirm the cross-cultural efficiency and utility of the Korean version of the K6/K10. Second, this study employed two widely used depression measures validated in Korea as reference measures (the K-CES-D and K-GDS-SF) to establish the caseness of psychological distress. In most previous K6/K10 validation studies, CIDI/DSM-IV was used as the gold standard. The findings of the present study should be validated in future K6/K10 validation studies with the SCID or CIDI as the gold standard. Third, the K-K6/K10 scales will identify people with anxiety disorders, but the concordance with the gold standard scales (e.g., CIDI) is likely to have lower bounds.

In conclusion, our study focused on older adults and their psychological distress and showed the potential utility of the K-K6/K10 as brief screening measures to be used in various residential and community settings that provide aging services and programs in Korea. Due to the brevity of the scales, their use may increase the rates of response and accuracy in elderly respondents in aging mental survey research or in health/social service agencies serving older adults with mental health issues. The K-K6/10 may also enable service providers with diverse training backgrounds to administer the scales easily, make timely assessments, and develop effective interventions under some circumstances where time and staff resources are limited.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Jong Won Min is an Associate Professor of Social Work, at the San Diego State University and serves as a co-Director of the SDSU University Center on Aging. He earned his doctoral degree in social welfare at the University of California, Los Angeles. His Master of Social Work degree came from University of Calgary, Canada. Prior to that, he graduated from Yonsei University, Seoul, Korea, with BA degree in Social Work. He has expertise, experiences and leadership in social work, with specific expertise in gerontology. He has published on socioeconomic-based health inequalities, racial disparity in health, minority aging, long-term care, mental health stigma, and cultural diversity.

Sun Hae Lee is a Professor at Chung-Ang University Department of Social Welfare in Seoul, Korea. She completed her Ph.D. in social welfare at the University of California, Berkeley and her Master of Social Work at Portland State University in Oregon. Her research interests lie in the areas of mental health and family therapy. She has authored multiple papers on mental health literacy, help-seeking behaviors, and service delivery issues in Korea. She is also an active Educator and Practitioner of narrative therapy and is an active consultant for government and social service agencies. She serves on the board of various academic associations and is the current Editor-in-chief of the Journal of the Korean Academy of Family Therapy.

References

Australian Department of Health. (2002). The Kessler Psychological Distress Scale (K10). Brief reports: Population research and outcome studies (No. 2002-14). Retrieved from http://health.adelaide.edu.au/pros/docs/reports/br200214_k10.pdf.

Bae

J. N.

Cho

M. J.

(2004) Development of the Korean version of the Geriatric Depression Scale and its short form among elderly psychiatric patients. Journal of Psychosomatic Research 57: 297–305.

Beck

A. T.

Ward

C. H.

Mendelson

Mock

Erbaugh

(1961) An inventory for measuring depression. Archives of General Psychiatry 4: 561–571.

Bewick

Cheek

Ball

(2004) Statistics review 13: Receiver operating characteristic curves. Critical Care 8(6): 508–512. doi:10.1186/cc3000.

Casey

D. A.

(2012) Depression in the elderly: A review and update. Asia-Pacific Psychiatry 4: 160–167.

Cho

M. J.

Bae

J. N.

Suh

G. H.

Hahm

B. J.

Kim

J. K.

Lee

D. W.

Kang

M. H.

(1999) DSM-III-RJuyouuljeunge daehan hangugeopan Geriatric Depression Scale (GDS)ui jindanjeok tadangseong yeongu [Validation of Geriatric Depression Scale, Korean version (GDS) in the assessment of DSM-III-R major depression]. Journal of Korean Neuropsychiatric Association 38(1): 48–62.

Cho

M. J.

Kim

K. H.

(1993) Juyouuljeunghwanja yebipyeonggaeseo center for epidemiologic studies depression scale (CES-D)ui jindanjeok tadangseong yeongu [Diagnostic validity of the CES-D (Korean version) in the assessment of DSM-III-R major depression]. Journal of Korean Neuropsychiatric Association 32: 381–399.

Cho

M. J.

Kim

K. H.

(1998) Use of the Center for Epidemiologic Studies Depression (CES-D) scale in Korea. Journal of Nervous & Mental Diseases 186: 304–310.

Cho, M. J., Park, J. I., Bae, A., Bae, J. N., Sohn, J. W., Ahn, J. H., . . . Hong, J. P. (2011). Jeongsin Jilhwan Siltae Yeoghag Josa [The epidemiological survey of mental disorders in Korea]. Sejong City, South Korea: Ministry of Health and Welfare.

10.

Curtis

M. J. R.

Baldwin

R. C.

(2009) Late-life depression in the general hospital. CME Journal Geriatric Medicine 11: 91–103.

11.

DeLong

E. R.

DeLong

D. M.

Clarke-Pearson

D. L.

(1988) Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 44: 837–845.

12.

Furukawa

T. A.

Kawakami

Saitoh

Ono

Nakane

Nakamura

Kikkawa

(2008) The performance of the Japanese version of the K6 and K10 in the world mental health survey Japan. International Journal of Methods in Psychiatric Research 17(3): 152–158. doi:10.1002/mpr.257.

13.

Furukawa

T. A.

Kessler

R. C.

Slade

Andrews

(2003) The performance of the K6 and K10 screening scales for psychological distress in the Australian National Survey of Mental Health and Well-Being. Psychological Medicine 33: 357–362.

14.

Han

H. M.

Yum

T. H.

Shin

Y. W.

Kim

K. H.

Yoon

D. H.

Chung

K. J.

(1986) Beck depression inventory ui hangukpan pyojunhwa yeongu [A standardization study of Beck depression inventory in Korea]. Journal of Korean Neuropsychiatric Association 25: 487–502.

15.

Hanley

J. A.

McNeil

B. J.

(1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143(1): 29–36.

16.

Huang

Carpenter

(2011) Identifying elderly depression using the depression rating scale as part of comprehensive standardized care assessment in nursing homes. Aging & Mental Health 15(8): 1045–1051. doi:10.1080/13607863.2011.583626.

17.

Institute for Health Metrics and Evaluation. (2015). GBD database. Seattle, WA: IHME, University of Washington. Retrieved from http://www.healthdata.org/search-gbd-data?s.

18.

Jang

Chiriboga

D. A.

Okazaki

(2009) Attitudes toward mental health services: Age-group differences in Korean American adults. Aging & Mental Health 13(1): 127–134. doi:10.1080/13607860802591070.

19.

Jung

I. K.

Kwak

D. I.

Shin

D. K.

Lee

M. S.

Lee

H. S.

Kim

J. Y.

(1997) Noinuulcheokdo (Geriatric Depression Scale)ui silloedo, tadangdo yeongu [A reliability and validity study of Geriatric Depression Scale]. Journal of Korean Neuropsychiatric Association 36: 103–112.

20.

Kessler

Barker

Colpe

Epstein

Gfroerer

Hiripi

Zaslavsky

(2003) Screening for serious mental illness in the general population. Archives of General Psychiatry 60(2): 184–189.

21.

Kessler

R. C.

Green

J. G.

Gruber

M. J.

Sampson

N. A.

Bromet

Cuitan

Zaslavsky

A. M.

(2010) Screening for serious mental illness in the general population with the K6 screening scale: Results from the WHO world mental health (WMH) survey initiative. International Journal of Methods in Psychiatric Research 19(Suppl 1): 4–22. doi:10.1002/mpr.310.

22.

Kim

Chiriboga

D. A.

Jang

Lee

Huang

C. H.

Parmelee

(2010) Health status of older Asian Americans in California. Journal of the American Geriatrics Society 58(10): 2003–2008. doi:10.1111/j.1532-5415.2010.03034.x.

23.

Kim

J. Y.

Park

J. H.

Lee

J. J.

Huh

Lee

S. B.

Han

S. K.

Woo

J. I.

(2008) Standardization of the Korean version of the Geriatric Depression Scale: Reliability, validity, and factor structure. Psychiatry Investigation 5(4): 232–238. doi:10.4306/pi.2008.5.4.232-238.

24.

Lee

S. H.

J. E.

(2009) Noinui uuljeungsang sikbyeollyeoge yeonghyangeul michineun yoin [Factors affecting the ability of older adults to identify symptoms of depression elderly depression and mental health services]. Journal of the Korean Gerontological Society 29(2): 529–546.

25.

Lee

Y. H.

Song

J. Y.

(1991) BDI, SDS, and MMPI-Dcheokdoui silloedo min tadangdoe daehan yeongu [A study of the reliability and the validity of the BDI, SDS, and MMPI-D Scales]. The Korean Journal of Clinical Psychology 10(1): 98–113.

26.

Lee

S. H.

Suh

J. H.

(2009) Ulinala seonginui uuljeungsang sigbyeollyeoggwa haegyeolbangbeobe daehan sinnyeom [Korean public’s ability to recognize depressive symptoms and beliefs on helplessness of various methods]. Mental Health and Social Work 32(8): 41–77.

27.

Lee

Tsang

K. L.

Y. L.

Guo

Mak

Kwok

(2012) Performance of the 6-item Kessler scale for measuring serious mental illness in Hong Kong. Comprehensive Psychiatry 53(5): 584–592. doi:10.1016/j.comppsych.2011.10.001.

28.

Lewis

H. J.

Hems

D. J.

Bosanquet

K. N.

Overend

K. J.

(2013) Is enough being done to treat depression in the elderly? Aging Health 9: 243–245.

29.

Min

J. W.

Moon

Lubben

J. E.

(2005) Determinants of psychological distress over time among older Korean Americans and Non-Hispanic White elders: Evidence from a two-wave panel study. Aging and Mental Health 9(3): 1–23.

30.

National Health Insurance Service (2013, February 28). Bodojaryo: maeumui gamgi uuljeung hwanja, 70dae yeoseongi gajang manha [Press release: Depression the cold of mind patient, most prevalent among women in their 70s]. Retrieved from http://www.nhis.or.kr/bbs7/boards/B0039/13469.

31.

National Health Insurance Service (2014, July 21). Bodojaryo: bulanjangae70dae isang noini 60dae iha boda 3bae isang manha [Press release: People in 70s treated for anxiety disorders three times more than those in 60s]. Retrieved from http://www.nhis.or.kr/bbs7/boards/B0039/3127.

32.

Organization for Economic Cooperationand Development (2009) Sickness, disability and work: Keeping on track in the economic downturn—Background paper, Geneva, Switzerland: Author.

33.

Perkins

N. J.

Schisterman

E. F.

(2006) The inconsistency of ‘optimal’ cutpoints obtained using two criteria based on the receiver operating characteristics curve. American Journal of Epidemiology 163: 670–675.

34.

Plitas

Tucker

Kritikos

Walters

Bardenhagen

(2009) A comparative study of the cognitive performance of Greek Australian and Greek national elderly: Implications for neuropsychological practice. Australian Psychologist 44(1): 27–39. doi:10.1080/00050060802587694.

35.

Radloff

L. S.

(1977) The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement 1: 385–401.

36.

Sheikh

J. I.

Yesavage

J. A.

(1986) Geriatric Depression Scale (GDS). Recent evidence and development of a shorter version. In: Brink

T. L.

(ed.) Clinical gerontology: A guide to assessment and intervention, New York, NY: The Haworth Press, Inc, pp. 165–173.

37.

Singer

Kuhnt

Götze

Hauss

Hinz

Liebmann

Schwarz

(2009) Hospital anxiety and depression scale cutoff scores for cancer patients in acute care. British Journal of Cancer 100(6): 908–912. doi:10.1038/sj.bjc.6604952.

38.

Song

T. M.

Jin

Song

J. Y.

Cho

Y. H.

(2013) Urinara sedaebyeol jasal teukseong bunseok [A comparative analysis of suicide charateristics by age group in Korea]. Journal of the Korea Society of Health Informatics and Statistic 38(1): 122–142.

39.

Sorkin

D. H.

Nguyen

Ngo-Metzger

(2011) Assessing the mental health needs and barriers to care among a diverse sample of Asian American older adults. Journal of General Internal Medicine 26(6): 595–602. doi:10.1007/s11606-010-1612-6.

40.

Statistics Korea (2011) Janglae ingu chugye [National population prospects], Daejeon, Republic of Korea.

41.

Statistics Korea (2013) Goryeongjatonggye [Older adults statistics], Daejeon, Republic of Korea.

42.

World Health Organization (1992) The ICD-10 classification of mental and behavioural disorders: Clinical descriptions and diagnostic guidelines, Geneva, Switzerland: Author.

43.

World Health Organization. (2012). The global burden of mental disorders and the need for a comprehensive, coordinated response from health and social sectors at the country level (Sixty Fifth World Health Assembly WHA65.4 Agenda item 13.2). Retrieved from http://apps.who.int/gb/ebwha/pdf_files/WHA65/A65_R4-en.pdf?ua=1.

44.

World Health Organization. (2013). Mental health and older adults (Fact sheet N 381). Retrieved from http://www.who.int/mediacentre/factsheets/fs381/en/.

45.

Yesavage

J. A.

Brink

T. L.

Rose

T. L.

Lum

Huang

Adey

Leirer

V. O.

(1983) Development and validation of a geriatric depression screening scale: A preliminary report. Journal of Psychiatric Research 17: 37–49.

46.

Youden

W. J.

(1950) An index for rating diagnostic tests. Cancer 3: 32–35.

47.

Zweig

M. H.

Campbell

(1993) Receiver operator characteristic (ROC) plots; a fundamental evaluation tool in clinical medicine. Clinical Chemistry 39: 561–577.