Development and Validation of the Self-Rating Suicide Risk Screening Questionnaire

Abstract

Since suicide is one of the major health problems worldwide, the goal of the present study is to develop a psychometrically sound suicide risk screening tool named Suicide Screening Questionnaire- Self-Rating (SSQ-SR) to quantify the risk of suicidal behavior. The development process includes item generation, item revision and refinement, and validation. A total sample of 2,238 adults in South Korea was used for analysis. The results of factor analysis showed that the SSQ-SR, containing 25 items, supported a three-factor model: “Situational factors,” “Negative self-perceptions and mental health factors,” and “Suicidal thoughts and behaviors.” Also, SSQ-SR showed satisfying internal consistency, concurrent validity and discriminant validity. Lastly, the results of ROC analysis showed a cut-off point of 33.5 for the low-risk group and 36.5 for the high-risk group. Considering these results, the current results demonstrate the SSQ-SR is a valid scale and therefore can contribute to future suicide-related studies and prevention.

Keywords

suicide screening questionnaire suicide screening questionnaire-self-rating suicide risk South Korea scale development validation

Introduction

In recent decades, suicide has become a major public health priority and remains one of the most prevalent causes of death worldwide. The World Health Organization (WHO) has advocated the implementation of LIVE LIFE, a guide for suicide prevention that recommends effective evidence-based interventions (World Health Organization, 2021). Amid such a situation, South Korea has maintained the first- and second-highest suicide rates among the Organization for Economic Cooperation and Development (OECD) member countries for ∼20 years since 2003. The suicide mortality rate (the number of deaths per 100,000 people) was 26.9 persons, which is more than double the average across the 38 OCED member countries, meaning that an average of 36 people commit suicide daily.

According to statistics on the causes of death in South Korea carried out by the National Statistical Office, suicide ranked fourth as the overall cause of death and the first cause of death among those in their teens, 20’s, and 30’s. Suicide has long been recognized as a national and social health problem rather than an individual problem. Hence, to protect citizens’ lives while fostering a culture of respect for life by exploring the necessary issues regarding national responsibility for suicide and prevention policies, the government has been promoting the National Action Plan since 2011, and the Central Suicide Prevention Center was established in 2012 as a countermeasure as a part of Act for prevention of suicide and the creation of culture of respect for life 2012 (KOR) (Korean Law Information, 2012).

Furthermore, according to the coronavirus disease 2019 (COVID-19) National Mental Health Survey, which was conducted in 2021 in South Korea, the suicide rate has skyrocketed to 12.4%, which is 2.5 times higher than the prior suicide rate in 2019 (Korea Suicide Prevention Center, 2021). Consequently, the prolonged pandemic continues to negatively affect citizens’ mental health. Moreover, several international studies have found that continuous exposure to infection and disaster situations elevates suicide-related thoughts and behaviors (Bryant-Genevier et al., 2021; Czeisler et al., 2020). Therefore, a practical and effective solution to screen suicide-related risks and improve interventions seems necessary at both the national and international levels.

Suicide is defined as “a death that occurs when an individual engages directly or indirectly in a positive or negative act of the victim toward oneself with intention” or “the act of intentionally ending one’s life” (Shneidman, 1977; Wreen, 1988). However, this definition does not cover the complexity of the matter itself. Suicidal ideation, often called suicidal thoughts or ideas, is defined as “passive thoughts of ending one’s life but not taking any active efforts to do so, which may or may not accompany the exact planning or intent” (Posner et al., 2007; Rozanov & Rakhimkulova, 2017). A suicidal attempt is “non-fatal self-directed and potentially injurious behavior with any intent to end one’s life, that does not result in death” (Krug et al., 2002). Suicidal behavior includes suicidal ideation, suicide attempts, preparatory acts, and completed suicide (Castle & Kreipe, 2007). Since all of these concepts are closely related to the final act of the behavior leading to suicide, the risk assessment should encompass all spectra of suicide.

To prevent suicide-related deaths, it is important to prevent suicidal behaviors in advance among high-risk suicide groups. In this context, several scales have been developed and validated domestically and internationally to empirically assess suicide risk. One of the most common scales used to assess and screen suicide risk through a self-report method is the Beck Scale for Suicidal Ideation (BSSI), developed by Beck et al. (1974). It consists of 21 items composed of statements describing actual suicidal desire, duration, frequency, attitude toward ideation, capability, expectancy, actual preparation, specificity of thoughts and plans, and so on. The scale has the advantage of reducing the physical and time burden in the clinical field because of its simple and convenient configuration (Steer et al., 1993). However, even though suicidal ideation is an important factor in predicting suicide attempts and eventual suicide, only focusing on suicidal thoughts is significantly different from suicidal thoughts leading to action, which leads to the limitation of evaluating the overall risk through the causal relationship between suicidal ideation and suicide attempts (Klonsky et al., 2016).

Other widely used scales are the Columbia University Suicide Severity Rating Scale (C-SSRS) and the Suicide Intent Scale (SIS). The C-SSRS (Posner et al. 2011), is composed of questions about suicidal ideation, intensity, behavior, and lethality. It is a semi-structured interview tool that the US Food and Drug Administration used to evaluate suicide risk during research on the relationship between drug use and suicide. Even though it is regarded as the gold standard tool, which has been translated into over 50 languages to evaluate suicidal thoughts and behaviors in both clinical and research settings (Al-Halabí et al., 2016; Conway et al., 2017; Lindh et al., 2018; Posner et al., 2008; Zakhour et al., 2021), expert diagnostic interviews are always required for assessment. As there are no quantitative indicators, such as cut-off scores for certain groups, this can create errors such as overestimation and underestimation due to the subjectivity and personal interpretation of the interviewer. Moreover, the C-SSRS has limitations in identifying the complete spectrum of suicidal thoughts and behaviors. Conversely, the SIS (Beck et al., 1974) is a self-report scale administered by interviewers to assess the severity of suicidal intentions and examine the factual aspects of suicide attempts. Although the “precaution” subscale was found to be associated with an increase in the risk of suicide, it is insufficient to assess suicide risk due to its contents being limited to suicidal intentions that only focus on patients who have attempted suicide.

In addition, the suicide risk assessment tools developed for the general population in South Korea which considers domestic circumstances are the Korean Suicide Risk Screening Tool, validated by modifying the two items from the Screening for Depression and Thoughts of Suicide in the Norton Sound region of Alaska (Kim et al., 2013). While the advantage of this quick and simple screening was apparent, using only two items to collect information on the suicide risk level of respondents is insufficient. Another validated South Korean scale is the South Korean validation Suicidal Imagery Questionnaire (SIQ) (Ko & You, 2020). However, there are limitations owing to the lack of supporting studies that confirm the direct effect of suicidal imagery on the risk of overall suicidal behavior, leading to limitations in evaluating suicide risk as a whole.

Existing scales only capture the unidimensional nature of suicide or some of the dimensions of the construct that factors other than suicidal ideation and behavior do not reflect, overlooking the various interactive factors that promote the risk of suicide. The currently available suicide risk tools lack optimal cut-off points to screen high-risk groups and do not report predictive validity for explaining the future risk of suicide (Baek et al., 2021; Franklin et al., 2017). Therefore, to evaluate an individual’s suicide risk quickly and to effectively understand and appropriately intervene in individuals, a scale that can reliably measure individual suicide risk while supplementing these limitations is needed (Lotito & Cook, 2015).

According to Granello (2010), a comprehensive evaluation of both the “risk factors” and “warning signs” of individuals is essential to assess suicide risk. Suicide risk is complex and involves a multitude of factors, such as mental, personal experience, socioeconomic, and physical illness (Hawton & van Heeringen, 2009; O’Connor & Nock, 2014; Steele et al., 2018). For example, clinical (e.g., sleep disorders, personality disorders), family and environmental factors (e.g., stressful events, loss of parents, relational problems), mental state (e.g., alcohol or substance abuse, state of anxiety, and hopelessness), and the case history (e.g., previous thought/attempts of suicide) are common risk factors known to contribute to suicidal thoughts and behaviors (Dilillo et al., 2015; Tucker et al., 2015). Regarding the warning signs of suicide victims, the Korea Psychological Autopsy Center (KPAC) under the Ministry of Health and Welfare (MHW) has been analyzing the cause of death and providing psychological support to bereaved families through psychological autopsy interviews and police investigation records. Psychological autopsy is one of the most useful tools for completed suicides (Isometsä, 2001; Mérelle et al., 2020; Zhang et al., 2019; Zhou et al., 2019). The method entails gathering available information about the deceased from multiple informants and records to confirm psychological behavior patterns and changes in suicide victims to verify the specific cause of suicide. In most cases, warning signs are expressed verbally, behaviorally, and emotionally (Rudd et al., 2006), and proper recognition of signs can prevent suicide in advance (Bagge et al., 2017; Tucker et al., 2015).

While our scale overlaps with certain dimensions from other existing scales, it adds compensatory validity to the conceptualization of the key dimensions and establishes the operational definition of suicide risk more fully by analyzing actual suicidal death data collected via psychological autopsy and national statistical surveys. Since suicide is the final act of behavior caused by interactions between various proximity and distance factors, it is necessary to consider the broader aspects of an individual’s life, such as warning signals, life situation, mental illness, personality, attitude toward suicide, and environmental factors. Moreover, based on a multidisciplinary perspective, to increase the possibility of generalization to widely utilize the assessment scale in various groups, there is a need for a proven tool that can be applied not only to certain groups but also to citizens in general. Thus, the current study aimed to develop and validate a new questionnaire, the Suicide Screening Questionnaire-Self-Rating (SSQ-SR), among non-clinical samples in South Korea. In addition to reflecting the limitations of existing suicide assessment tools, the SSQ-SR provides a cut-off score to screen risk groups.

The objectives of this study are manifold and as follows:

1. To develop a psychometrically sound SSQ-SR to quantify the risk of suicidal behaviors.

2. To assess the validity of the SSQ-SR in a non-clinical sample using exploratory factor analysis (EFA) and confirmatory factor analysis (CFA).

3. To verify internal consistency and concurrent validity by exploring the relationship between the SSQ-SR and related measures, such as depression and suicidal ideation.

4. To determine the optimal cut-off value of the SSQ-SR to appropriately screen the risk groups.

Methodology for Scale Development

The current study was performed in three phases to develop a screening tool for high suicide risk groups to evaluate the severity of suicide risk for 3 years, from 2019 to 2021. The first phase included developing indigenous items to screen for high-risk suicidal behaviors. The second phase of the scale’s refinement process included assessing the scale through content validity and a pilot study. The third phase revolved around determining the psychometric properties of the newly created questionnaire, including establishing its validity, reliability, and cut-off value (see Figure 1).

Figure 1.

Scale development process. MHW: Ministry of Health and Welfare; KNPA: Korean National Police Agency; ROC: receiver operating characteristic.

Phase 1: Item Generation

Formation of an Expert Committee

In this study, an expert committee (the Task Force for Suicide Screening and Prevention (TFSSP)) was formed as a multidisciplinary team. The overall aim of the committee was to develop and validate a high-risk suicide assessment and screening tool. The committee consisted of professionals, clinicians, and community practitioners affiliated with the KPAC, regional branches of the Mental Health Welfare Center, and the Korea Suicide Prevention Center (KSPC) under the MHW, such as psychiatrists (n = 10), clinical nurses and nursing professors (n = 4), clinical social workers and professors of social welfare (n = 2), clinical and counseling psychologists (n = 18), professionals in integrative medicine and emergency medicine (n = 2), and statisticians (n = 1). Joint meetings were held to discuss the following item generation process.

A Detailed Literature Review of Suicide Measurement Tools

From a multidisciplinary perspective, based on the opinions of the TFSSP team, a detailed systematic literature review was performed using domestic and international academic information search engines, such as the Research Information Service System (RISS), Korean Studies Information Service System (KISS), PLOS ONE, PubMed, and MEDLINE. More than 50 articles were found using key terms such as suicide risk, assessment tools, self-report, and scale validation. By searching and analyzing the features, limitations, and psychometric characteristics of 18 domestic and 20 international assessment tools currently used in numerous studies, we found that the South Korean assessment tools were insufficient to explain and evaluate suicide risk comprehensively and could only be applied to specific groups. Moreover, the most widely used tools, such as the BSSI and C-SSRS, focused only on suicide ideation and did not report predictive validity. Overall, the tools only measure a subset of factors related to suicide, overlooking the various interactive factors that promote suicide risk. Therefore, the committee raised the need to collect reports from suicide victims’ families or acquaintances and statistical data to evaluate suicide risk considering the multidimensional aspects of the individual’s life, such as the social, economic, cultural, and psychological characteristics of suicide deaths, including warning signs and risk factors.

Review of Psychological Autopsy and Suicide-Related Data

The Central Psychological Autopsy Center (CPAC), established in 2014 due to the prolonged high suicide rate in South Korea, began conducting psychological autopsies in 2015. In the present study, the data from 2015 to 2017 following the Korea Psychological Autopsy Checklist 2.1 (K-PAC 2.1; KPAC, 2019), an evidence-based psychological autopsy tool developed by extracting common items from domestic and international tools, was used for the analysis. The questions of the K-PAC 2.1 include information about the bereaved families (general information, psycho-emotional state, reaction, and coping method) and the suicide victims (general information, suicidal behavior, death information, interpersonal relationships, and mental health information), as well as the interviewer’s assessment (lethality at the time of suicide and presumed mental illness of the victims and family members). By reviewing the statements of psychological autopsies of 355 bereaved family members across 289 suicide deaths, the patterns and changes in the psychological behaviors of suicide victims were verified.

According to the bereaved family members’ reports, of the total 289 suicide victims, 264 (91.3%) showed warning signs before their death. The signs were largely expressed verbally, behaviorally, and emotionally. Among the verbal warning signs, “frequently talking about suicide, murder, or death” was the most common; among the behavioral warning signs, “change in sleep state” was the most common; among the emotional warning signs such as guilt, shame, and loneliness, “changes in emotional state” such as irritability was the most common.

Additionally, complete enumeration surveys of 9,893 suicide deaths conducted by the Korean National Police Agency (KNPA) from 2012 to 2017 and an epidemiological survey of mental illnesses in South Korean adults conducted in 2016 by the National Center for Mental Health under the MHW were included in the analysis. Based on the data from the survey by the KNPA, the risk factors for suicidal death were mental health problems, economic difficulties, physical illness problems, family problems, and school/work problems—in descending order. Moreover, it reconfirmed that mental health problems, including alcohol and drug use disorders, are closely related to suicidal behaviors. People with alcohol use disorder were more likely to commit suicide by alcohol poisoning, and those who died from drug addiction had a history of receiving psychiatric treatment, with a high prevalence of comorbid alcohol use disorder. Lastly, according to the mental illness survey data, 3.2% of the total South Korean population had attempted suicide, and 1.1% had attempted suicide twice or more. In addition, 80 times more people have attempted suicide compared with the number of suicide deaths, and ∼350 times more people have experienced suicidal thoughts.

Reflecting the derived results through the process of a literature review and analyzing the results of the psychological autopsy and suicide-related data from the surveys, 77 items were designed as a draft questionnaire. The item content included five key factors: suicide warning signs, suicide attitude, mental health problems, personality, and socioeconomic factors.

Phase 2: Item Revision and Refinement

Item Content Validity Through Expert Review

Two expert evaluations were conducted. The 77 items derived went through the initial review process by the TFSSP research team members by considering the potential for screening high-risk suicidal behaviors and the readability of each item. Content validity was rated on a 6-point Likert scale ranging from 0 (very unrelated; very unclear) to 5 (very related; very clear), along with open-ended questions where experts could freely give their opinions on items. For items that scored 3 points or higher, discussions were held based on the related literature, psychological autopsy reports, and hands-on experience of experts in suicidal crises, leading to the removal of 44 items. Next, the chosen items were submitted for an expert panel review, excluding the TFSSP members. A total of 251 suicide-related experts and practitioners, including psychiatrists, clinical nurses, clinical social workers, clinical psychologists, counseling psychologists, and professors in all fields, were invited to evaluate the clarity, conciseness, and readability of the individual items and whether the content is essential and appropriate for the actual screening of high-risk groups. They were also asked to rate on a 6-point Likert scale ranging from 0 (very unrelated; very unclear) to 5 (very related; very clear), and additional space was provided for qualitative opinions. By examining the high-ranked items regarding the possibility of item content redundancy and applicability only to specific age groups based on quantitative scores and qualitative feedback, the final 29 items were extracted.

Pilot Study

With the final 29-item pool, a pilot study was conducted by administering an online questionnaire (Lee et al., 2022) and the face validity of the scale was reconfirmed through the following process. The clinical and counseling psychologists and mental health counselors in university counseling centers reached out to their clients with repeated high-risk suicidal ideation and past suicide attempts, and 71 of them agreed to participate in the pilot study. To further refine the items, interviews were conducted to determine whether the characteristics of the high-risk suicide group were properly reflected in composing the items. Based on the result that most of the items were easily understood due to appropriate expression and no item exclusion, the authors further refined the individual items according to the feedback. Finally, the newly developed 29-item scale was named the Suicide Screening Questionnaire Self-Rating (SSQ-SR).

Phase 3: Questionnaire Survey, Validation, and Determining Psychometric Properties

Participants

Participants in this study were recruited in two rounds. First, for the EFA, the national online sample consisted of 1200 South Korean adults over the age of 19, based on the South Korean population census standard, considering criteria such as sex, age, and residential area. Initially, 1986 individuals were sent the questionnaire, of which 1236 responded. Of these, 36 individuals who answered insincerely by randomly or disingenuously marking the questionnaire were excluded. Thus, 1200 participants were selected for further analysis. The mean age was 47.76 years (SD = 13.89), with 602 (50.2%) men and 598 (49.8%) women. Participants were categorized into three groups: 9 participants in the suicide attempt (SA) group, 124 in the suicide ideation (SI) group, and 1067 in the normal control group.

In the second offline survey, 1038 participants responded to the questionnaires, targeting Korean adults over the age of 19. Among the sample, 381 (36.7%) were men, and 657 (63.3%) were women. The mean age was 25.07 (SD = 7.88) years, with 850 people (81.9%) in their 20’s and 188 people (18.1%) aged over 30 years. The final participants were divided into the following groups: 20 in the suicide attempt group, 195 in the suicide ideation group, and 823 in the control group.

All participants met the following eligibility criteria: they were able to read and write Korean fluently and provided informed consent. Table 1 provides extensive socio-demographic information for each survey, such as marital status, educational background, occupational information, and religion.

Table 1.

Socio-Demographic Information of the Study Sample.

Online sample
Variables	Total sample (N = 1,200)	SA group (N = 9)		SI group (N = 124)		Normal group (N = 1067)
Variables	Total sample (N = 1,200)	N	%	N	%	N	%
Sex	Men (n = 602)	5	55.6%	40	32.3%	557	52.2%
Sex	Women (n = 598)	4	44.4%	84	67.7%	510	47.8%
Age	19–29 years (n = 176)	3	33.3%	32	25.8%	141	13.2%
	30–39 years (n =204)	3	33.3%	29	23.4%	172	16.1%
	40–49 years (n = 240)	2	22.2%	24	19.4%	214	20.1%
	50–59 years (n = 253)	0	0.0%	19	15.3%	234	21.9%
	>60 years (n = 327)	1	11.1%	20	16.1%	306	28.7%
Occupation	Student (n = 40)	0	0.0%	5	4.0%	35	3.3%
	Office worker (n = 629)	3	33.3%	55	44.4%	571	53.5%
	Practitioner (n = 80)	2	22.2%	8	6.5%	70	6.6%
	Self-employed (n = 94)	0	0.0%	8	6.5%	86	8.1%
	Housewife (n = 148)	0	0.0%	18	14.5%	130	12.2%
	Military Service (n = 2)	0	0.0%	0	0.0%	2	.2%
	Retired (n = 72)	1	11.1%	5	4.0%	66	6.2%
	Unemployed (n = 87)	3	33.3%	19	15.3%	65	6.1%
	Others (n = 48)	0	0.0%	6	4.8%	42	3.9%
Marital status	Single (n = 344)	5	55.6%	54	43.5%	285	26.7%
	Married (n = 768)	4	44.4%	60	48.4%	704	66.0%
	Cohabitation (n = 8)	0	0.0%	0	0.0%	8	.7%
	Separated (n = 2)	0	0.0%	1	.8%	1	.1%
	Divorced (n = 48)	0	0.0%	7	5.6%	41	3.8%
	Widowed (n = 30)	0	0.0%	2	1.6%	28	2.6%
Religion	Christian (n = 258)	1	11.1%	19	15.3%	238	22.3%
	Buddhism (n = 182)	3	33.3%	25	20.2%	154	14.4%
	Catholic (n = 137)	1	11.1%	13	10.5%	123	11.5%
	Others (n = 16)	0	0.0%	0	0.0%	16	1.5%
	Non-religion (n = 607)	4	44.4%	67	54.0%	536	50.2%
Offline sample
Variables	Total sample (N = 1,038)	SA group (N = 20)		SI group (N = 195)		Normal group (N = 823)
Variables	Total sample (N = 1,038)	N	%	N	%	N	%
Sex	Men (n = 381)	6	30.0%	47	24.1%	328	39.9%
Sex	Women (n = 657)	14	70.0%	148	75.9%	495	60.1%
Age	19–29 years (n = 850)	18	90.0%	163	83.6%	669	81.3%
	30–39 years (n = 118)	2	10.0%	23	11.8%	93	11.3%
	40–49 years (n = 46)	0	0.0%	6	3.1%	40	4.9%
	50–59 years (n = 22)	0	0.0%	2	1.0%	20	2.4%
	>60 years (n = 2)	0	0.0%	1	.5%	1	.1%
Occupation	Student (n = 650)	14	70.0%	119	61.0%	517	62.8%
	Full time employee (n = 197)	1	5.0%	30	15.4%	166	20.2%
	Irregular worker (n = 81)	0	0.0%	17	8.7%	64	7.8%
	Self-employed (n = 15)	0	0.0%	0	0.0%	15	1.8%
	Housewife (n = 24)	1	5.0%	5	2.6%	18	2.2%
	Military service (n = 7)	2	10.0%	2	1.0%	3	.4%
	Unemployed (n = 64)	2	10.0%	22	11.3%	40	4.9%
Marital status	Single (n = 894)	17	85.0%	171	87.7%	706	85.8%
	Married (n = 127)	3	15.0%	21	10.8%	103	12.5%
	Cohabitation (n = 10)	0	0.0%	1	.5%	9	1.1%
	Separated (n = 2)	0	0.0%	0	0.0%	2	.2%
	Divorced (n = 4)	0	0.0%	1	.5%	3	.4%
	Widowed (n = 1)	0	0.0%	1	.5%	0	0.0%
Religion	Christian (n = 219)	1	5.0%	39	20.0%	179	21.7%
	Buddhism (n = 59)	1	5.0%	10	5.1%	48	5.8%
	Catholic (n = 75)	2	10.0%	14	7.2%	59	7.2%
	Others (n = 7)	1	5.0%	1	.5%	5	.6%
	Non-religion (n = 678)	15	75.0%	131	67.2%	532	64.6%

Procedures

The online national survey for EFA was conducted via an Internet survey company, Invite, for 1 month between January and February 2021. The online survey company was ISO 9001 certified, indicating that it follows the most widely accepted quality management system standards. The firewall (WAF) and DigiCert security solutions were deployed to ensure the survey’s security. All survey results were gathered via an encrypted secure socket layer, which allowed for data authentication, encryption, and decryption. After the system’s operation expired, all data were securely removed.

For CFA, an offline survey was conducted for ∼19 months, from June 2020 to December 2021. First, to recruit study participants, the main researcher attended the annual academic conference of the South Korean National Association of University Counseling Centers in 2019 and 2020, where 262 universities and colleges are affiliated. An introductory session was held regarding the purpose and aim of the study to request participation in the research. As 60 universities expressed their intention to participate, an additional online meeting regarding specific guidance on the research purpose, process, procedures, and data collection method was held for the head and counselors of each university counseling center. Ultimately, 30 universities decided to participate in the study. The survey was conducted under the supervision of the persons in charge of each university who were acquainted with the guidance. The supervisors mainly consisted of those affiliated with psychological counseling centers, mental health centers, career development counseling centers, and job support centers for gender equity. They recruited the participants by explaining the current research and obtaining agreement from undergraduate and graduate students who were receiving mental health or career-related services. Participants were allowed to choose between an online Google form and a written questionnaire. In the likely chance that participants would experience psychological difficulties when responding, detailed information on mental health institutions on- and off-campus was provided on the back of the questionnaire.

Throughout both the EFA and CFA surveys, all participants were assured of the privacy and anonymity of their information. Informed consent to participate in this study was obtained from each participant before the investigation. The surveys took ∼20–30 min to complete, and the participants were compensated with mobile gift vouchers of approximately 3–6 US dollars. Participants were first required to fill in their socio-demographic information and questions about the criteria for inclusion, after which they responded to the administered questionnaires.

During the survey, the participants were asked to select whether they had attempted suicide or suicidal ideation. Depending on the responses, participants were grouped according to the following criteria: those who attempted suicide within 6 months were classified into the suicide attempt group; those who had never attempted suicide within 6 months but had thought of suicide were classified into the suicide ideation group; others were classified as the normal control group. Additionally, in the process of the offline survey, diagnostic interviews with clinical psychologists and counseling psychologists with professional qualifications were conducted, leading to the participation of actual SA and SI groups.

The current study was authorized by the Institutional Review Board of the University (2019-12-001-003), to which the researchers were affiliated, and all methods were carried out in compliance with relevant norms and regulations.

Measures

Suicide screening questionnaire-self-rating

The SSQ-SR is a self-report measure designed to assess the risk of suicide in individuals, such as socioeconomic factors, mental health factors, personality factors, suicide attitude, and suicide warning signs. It consists of 29 items rated on a 4-point Likert scale. From items 1 to 23, the participants were asked to rate the degree of experiencing each statement from 1 (not at all) to 4 (strongly agree), and for the remaining items from 24 to 29, they were asked to select the relevant period of each suicide-related thought or behavior from 1 (none at all) to 4 (within 1 month). The total score ranges from 29 to 116, with higher scores indicating a higher risk of suicide. In this study, Cronbach’s alpha coefficient was .92.

Depression: Patient health questionnaire-9

The PHQ-9 (Spitzer et al., 1999), which has been validated in the South Korean context (Park et al., 2010), was used to determine the severity of depression. The scale is a self-report scale that screens for depression and evaluates severity, consisting of nine items according to the diagnostic criteria for a major depressive disorder in the DSM-IV. Participants were asked to rate the level of symptoms regarding their major depressive disorder in the past 2 weeks on a 4-point Likert scale ranging from 0 (never) to 3 (almost every day), with higher scores indicating higher levels of depression. According to the Korean version of the PHQ-9 (Park et al., 2010), Cronbach’s alpha coefficient was .81. In this study, Cronbach’s alpha coefficient was .90.

Beck scale for suicidal ideation

The BSSI, originally developed by Beck and Steer (1993) and later validated in a South Korean context (Choi et al., 2020), was used to evaluate the presence and severity of suicidal thoughts. It is a self-report scale comprising 19 items rated on a 3-point Likert scale ranging from 0 to 2, based on experiences over the past week. Higher scores indicated higher levels of suicidal thoughts, ranging from 0 to 38. According to (Choi et al., 2020), Cronbach’s alpha coefficient for the scale was .90. In this study, Cronbach’s alpha coefficient was .91.

Data Analysis

First, to confirm the general characteristics of the study participants, a frequency analysis was performed on the socio-demographic variables. Second, to improve the reliability of the multi-item scale, an item-total correlation analysis was performed to remove the items indicating low correlations between an individual item and the total score. Third, a scree plot and parallel analysis were used to determine the suitability of the data for factor analysis. Based on these results, the scale was subjected to EFA. Maximum likelihood estimation and the direct oblimin rotation technique were used. Through this process, items with factor loadings less than 0.4, regarded as unacceptable, were deleted. Fourth, a CFA was used to verify the overall fit of the measurement models. Each model was examined using several goodness-of-fit measures, such as the chi-square test, comparative fit index (CFI), and Tucker-Lewis Index (TLI), with values over .90 interpreted as satisfying and the Root Mean Square Error Approximation (RMSEA) with values lower than .5 and .8 considered as a good model fit (Hong, 2000). Fifth, reliability was confirmed through the internal consistency coefficient (Cronbach’s α), Omega Coefficient, and Omega Hierarchical. Sixth, to confirm the concurrent validity of the scale, a correlation analysis was conducted between the developed scale and suicidal ideation, BSSI, which is used as a reliable measure of suicidal ideation, and depression, which was found to be highly related to suicidal thoughts. Next, to verify the discriminant validity between groups, the difference between the self-reported total scores of the suicide attempt, suicide ideation, and normal control groups was analyzed. Prior to the analysis, the homogeneity of variance between groups was confirmed using Levene’s test. When the homogeneity of variance was satisfied, the analysis of variance (ANOVA) was used, and in case of unsatisfactory homogeneity, the Kruskal–Wallis test was used. Finally, the receiver operating characteristic (ROC) analysis was performed to evaluate the usefulness of the scale by confirming whether the scale could properly discern the high-risk group for suicide by using an optimal cut-off point. The data were statistically analyzed using SPSS 21.0 and Mplus 8.0.

In this study, the first online survey participants’ responses were used for the EFA, followed by a CFA with the second offline survey participants’ responses and the reliability, validity, and ROC analyses for both the online and offline participants.

Results

To examine the items of the SSQ-SR, the averages and standard deviations are reported in Table 2.

Table 2.

Descriptive Statistics of SSQ (N = 2,238).

Item	EFA data (N = 1,200)		CFA data (N = 10,38)
Item	Mean	Standard Deviation	Mean	Standard Deviation
1. I have a disease or disability and it is difficult to live.	1.384	0.695	1.150	0.429
2. I am distressed because of financial problems.	1.675	0.854	1.327	0.635
3. I am distressed because of unfair treatment or harassment.	1.402	0.709	1.165	0.480
4. I am suffering from the death of a loved one. (e.g., family, friend, lover, pet, etc.)	1.380	0.710	1.099	0.444
5. I am suffering from conflicts with a family member or a lover	1.442	0.754	1.346	0.703
6. I have been suffering ever since experiencing violence	1.284	0.641	1.183	0.532
7. I rarely meet anyone and spend most of the day alone.	1.868	0.876	1.716	0.965
8. I have no one to rely on in times of trouble.	1.692	0.789	1.476	0.775
9. I am frustrated due to repeated failures of important things.	1.554	0.743	1.434	0.745
10. I am a useless person.	1.431	0.733	1.353	0.667
11. Unlike usual, I do dangerous or reckless behaviors.	1.295	0.601	1.121	0.415
12. My life is miserable.	1.412	0.736	1.297	0.625
13. I feel sad or hopeless.	1.479	0.756	1.423	0.723
14. I have severe mood swings that are hard to control.	1.497	0.737	1.427	0.731
15. I feel depressed or lethargic in everything.	1.647	0.772	1.573	0.824
16. I feel anxious and nervous.	1.691	0.717	1.640	0.841
17. I have trouble sleeping.	1.906	0.784	1.671	0.923
18. It is difficult to control the urge to die when I drink.	1.239	0.588	1.054	0.299
19. When I hear the news of suicide by a celebrity or someone else, I want to die as well.	1.221	0.566	1.108	0.417
20. I feel the urge to kill myself.	1.293	0.632	1.223	0.534
21. There is no way to solve the current situation other than by dying.	1.248	0.599	1.106	0.352
22. People around me would be better off if I am gone.	1.282	0.632	1.130	0.423
23. I am planning to kill myself.	1.173	0.517	1.039	0.218
24. Someone close to me, such as a family member or friend, has committed suicide.	1.234	0.501	1.104	0.371
25. I have attempted suicide.	1.128	0.402	1.125	0.423
26. I have planned a specific suicide method, location, or date/time.	1.148	0.464	1.157	0.504
27. I have written a message implying suicide.	1.130	0.443	1.120	0.446
28. I have actually prepared for suicide before.	1.125	0.428	1.109	0.401
29. I have searched the internet about suicide or participated in suicide-related communities.	1.133	0.462	1.172	0.546

SSQ: Suicide Screening Questionnaire; EFA: Exploratory Factor Analysis; CFA: Cinformatory Factor Analysis.

To determine the homogeneity of the scale, the correlation between each item and the entire scale minus the item itself was analyzed, and .3 or less was used as the removal criterion. The correlations ranged from .445 to .806 (p < .01), and all items were judged to be appropriate. Therefore, no item was deleted.

As a result of reviewing the sample fit of the Kaiser–Meyer–Olkin (KMO) test and Bartlett’s test of sphericity, the KMO fitness index was .969, and the result of Bartlett’s test of sphericity was χ² (df = 406, p = .000) = 21,715.544, indicating that the sample was suitable for factor analysis (Kaiser, 1974). The results of the scree plot are shown in Figure 2. Additionally, as a result of confirming the difference in eigenvalues according to factor changes, eigenvalues change rapidly from 2 to 3 factors, normalizing from 3 factors onwards. In the case of the cumulative variance ratio, a value of 50 or higher was judged appropriate, indicating that more than two factors would be appropriate, The results are presented in Table 3.

Figure 2.

Scree plot of the components of the SSQ-SR.

Table 3.

Eigenvalues and Cumulative Variance (N = 1,200).

Component	Eigenvalue	Difference	Variance: %	Cumulative variance: %
1	12.956	10.672	44.676	44.676
2	2.284	1.018	7.877	52.553
3	1.266	0.087	4.367	56.920
4	1.179	0.300	4.064	60.984
5	0.879	0.039	3.031	64.015

The results of parallel analysis of SSQ-SR factors showed that the variance of the eigenvalues of the mean of random % of the variance is larger than the real data % of the variance of the data in four-factorial solutions (mean of random % of variance = 1.200; real data % of variance = 1.179), leading to the judgment that the 2 or 3 factors are appropriate.

As a result of confirming the number of factors through the heuristic method, the fit measures of 2–4 factors were analyzed. As shown in Table 4, it was found that each time the factor increased, χ² was lowered, and the factor congruence of 2–4 factors was found to be appropriate. Therefore, reflecting the results of the heuristics and inferential approach, a three-factor model was selected as the final model.

Table 4.

Factor Model Congruence for EFA (N = 1,200).

Model	χ² (df)	CFI	TLI	RMSEA
Two-factor model	1467.058 (349)***	0.972	0.967	.052
Three-factor model	1111.368 (322)***	0.980	0.975	.045
Four-factor model	851.526 (296)***	0.986	0.981	.040

CFI: comparative fit index; TLI: Tucker-Lewis fit index; RMSEA: root mean square error of approximation. ***p < .001.

An exploratory rotation of the three-factor model was performed using the GEOMIN rotation strategy. The factor loadings of the EFA for a total of 29 items are shown in Table 5. As a result, the factor loadings of items 2, 11, and 17 were less than .40, leading to the exclusion of three items. The factor loadings of the remaining 26 items were found to be .40 or higher. Factor 1 comprised seven items and was named “situational factors,” including items such as “I am suffering from unfair treatment or harassment” and “When I hear the news of suicide by a celebrity or someone else, I want to die as well.” With a total of 12 items, “negative self-perceptions and mental health factors” includes “My life is miserable” and “There is no way to solve the current situation other than by dying.” Factor 3 was named “suicidal thoughts and behaviors,” including “planning suicide” and “having attempted suicide,” and comprised a total of seven items.

Table 5.

Summary of Exploratory Factor Analysis results with three factors (N = 1,200).

Item	Primary subtest			Secondary subtest
Item	Factor 1	Factor 2	Factor 3	Factor 1	Factor 2	Factor 3
1	0.560	−0.013	0.266	0.628	−0.021	0.167
2	0.356	0.337	−0.008	––	––	––
3	0.706	0.023	0.146	0.753	0.032	0.033
4	0.588	−0.012	0.274	0.671	−0.003	0.141
5	0.526	0.287	−0.009	0.613	0.272	−0.122
6	0.629	0.099	0.209	0.732	0.104	0.056
7	−0.255	0.748	−0.005	−0.209	0.720	0.003
8	−0.072	0.693	0.023	−0.046	0.677	0.024
9	0.116	0.701	−0.019	0.100	0.714	−0.033
10	−0.065	0.877	0.042	−0.047	0.876	0.031
11	0.380	0.145	0.343	––	––	––
12	−0.001	1.068	−0.137	−0.003	1.060	−0.132
13	0.008	0.967	−0.087	0.016	0.964	−0.095
14	0.252	0.501	0.081	0.318	0.501	−0.006
15	−0.024	0.746	0.033	0.042	0.717	0.002
16	0.051	0.618	0.054	0.052	0.636	0.036
17	0.069	0.392	0.027	––	––	––
18	0.408	0.170	0.419	0.525	0.178	0.273
19	0.300	0.322	0.392	0.422	0.322	0.258
20	0.168	0.434	0.414	0.289	0.428	0.304
21	0.189	0.442	0.390	0.289	0.445	0.284
22	0.148	0.440	0.391	0.243	0.452	0.285
23	0.240	0.282	0.521	0.345	0.291	0.405
24	0.011	−0.263	0.859	0.011	−0.210	0.825
25	−0.151	0.028	0.981	−0.140	0.083	0.952
26	−0.041	−0.128	1.047	−0.020	−0.061	0.991
27	−0.031	0.011	0.942	−0.001	0.069	0.884
28	−0.072	−0.064	1.036	−0.042	−0.008	0.984
29	−0.005	0.069	0.802	0.030	0.106	0.752

Table 6 shows the results of the CFA to confirm the structural adequacy of the three-factor model, including and excluding item 24 (Someone close to me, such as a family member or friend, has committed suicide), of which the factor loading in the CFA was less than .4. As a result, the χ² value of the three-factor model was 760.361 (df = 296, p < .001), the TLI was .982, the CFI was .983, and the RMSEA was .039, indicating that the three-factor model was appropriate. As for the model fit of the three-factor model with item 24 deleted, the χ² value was 720.558 (df = 272, p < .001) for χ², .982 for TLI, .984 for CFI, and .040 for RMSEA, with factor loadings of .4 or higher. Therefore, a three-factor model without item 24, with superior goodness-of-fit measures, was selected.

Table 6.

Model Fit Indices for CFA Models of SSQ-SR Scale (N = 1,038).

Model	χ² (df)	CFI	TLI	RMSEA
Three-factor model (including item 24)	760.361 (296)***	.983	.982	.039
Three-factor model (excluding item 24)	720.558 (272)***	.984	.982	.040

***p < .001

The SSR-SQ shows an internal reliability of (Cronbach’s α) .939 for the total scale. Cronbach’s alpha was .821 for situational factors, .920 for negative self-perception and mental health, and .869 for suicidal thoughts and behaviors. Additionally, the omega coefficient for the total items was .941, and the omega coefficients for difficult situations and environmental factors, negative self-awareness and mental health, and suicidal thoughts and behaviors were .821, .921, .871, respectively.

To assess the concurrent validity of the SSQ-SR, correlation analysis with related variables was conducted. The BSSI, which is widely used to measure suicidal thoughts, was positively correlated with the whole SSQ-SR (r = .671, p < .001). In addition, depression which, from previous studies, is known as one of the high comorbidities with suicidal thoughts and behaviors, was analyzed. A positive correlation between the entire SSA-SR and depression (r = .766, p < .001) was observed. In addition, all sub-factors of the SSQ-SR were positively correlated with both depression and suicidal ideation. The results are presented in Table 7.

Table 7.

Correlations Coefficients Between the Full Scale/Subscales and Other Variables (N = 2,238).

	Factor 1	Factor 2	Factor 2	Suicide ideation (BSSI)	Depression (PHQ-9)
Total scale	.847***	.954***	.754***	.671***	.766***
Factor 1		.697***	.581***	.455***	.574***
Factor 2			.603***	.667***	.797***
Factor 3				.585***	.479***

***p < .001.

To verify the group discrimination validity of the SSQ-SR, we checked whether the full self-report scale and the sub-factors differed according to the SA, SI, and normal control groups. Since Levene’s test did not satisfy the homogeneity of variance (equal variance), the Kruskal–Wallis test, a nonparametric statistic, was used, and post-hoc analysis was performed via pairwise comparison. As shown in Table 8, the results show no significant differences between the SA and SI groups, but that both SA and SI groups showed significantly higher scores than the normal control group in factors 1 and 2 and the total scores. In the case of factor 3, the SI group showed a higher score than the normal control group and a lower score than the SA group.

Table 8.

Discriminant Validity and Tests of Differences Between Three Groups (N = 2,238).

	SA group (N = 29) (A)		SI group (N = 319) (B)		Normal group (N = 1,890) (C)		Statistics test Kruskal–Wallis test	Post hoc comparison pairwise comparison
	Mean	Standard deviation	Mean	Standard deviation	Mean	Standard deviation	Statistics test Kruskal–Wallis test	Post hoc comparison pairwise comparison
Total score	53.966	19.584	43.546	11.567	30.944	7.865	χ² = 467.525, p = .000	A, B > C
Factor 1	12.586	5.961	10.489	3.521	8.424	2.566	χ² = 227.645, p = .000	A, B > C
Factor 2	27.448	10.228	24.846	7.588	16.095	4.827	χ² = 461.464, p = .000	A, B > C
Factor 3	13.931	5.203	8.210	2.752	6.425	1.490	χ² = 426.031, p = .000	A > B > C

To confirm the discriminatory power between the suicide risk group with suicidal ideation, and the high-risk group with suicidal attempts, an ROC analysis was performed by dividing participants who attempted suicide within the last 6 months (n = 29) into the high-risk group for suicide, and the study participants in the suicidal ideation group (n = 348) were set as the low-risk group standard. The ROC analysis results are shown in Figure 3 and Table 9. Checking the area under the curve (AUC), which indicates the accuracy of the screening tool, showed a value of .842 and .863 for the high-risk group and low-risk group, respectively. Both were higher than the .80 standard suggested by Holmes (1998), which was judged to be appropriate.

Figure 3.

ROC curves of SSQ-SR in identifying low-risk (left) and high-risk groups (right).

Table 9.

ROC Analysis and Cut-Off Points for High-Risk and Risk Group.

	AUC	95% CI	Cut-off point	Sensitivity	Specificity	Youden Index	PPP	NPP	Accuracy
High risk	.842	.760	36.5	.828	.780	.608	.828	.997	.781
High risk	.842	−.925	36.5	.828	.780	.608	.828	.997	.781
Low risk	.863	.843	33.5	.807	.793	.600	.807	.957	.795
Low risk	.863	−.882	33.5	.807	.793	.600	.807	.957	.795

AUC: Area under the curve; CI: Confidence interval; PPP: Positive Predictive Power; NPP: Negative Predictive Power.

Based on Youden’s J value for estimating the cut-off point, 36.5 points for the high-risk group and 33.5 points for the low-risk group were found to be the optimal cut-off points. Based on the high-risk group score of 36.5, the sensitivity was .828, specificity was .780, positive predictive power (PPP) was .828, and negative predictive power (NPP) was .997. The accuracy was .781, and 36.5 points were judged to have high discriminatory power. Based on the risk group standard of 33.5, the sensitivity was .807, specificity was .793, PPP was .807, and NPP was .957. The accuracy was .795, and 33.5 points were judged to have high discriminatory power.

Discussion

The current study aimed to develop and assess the psychometric properties of a newly created SSQ-SR scale to investigate the severity of suicide risk among South Korean non-clinical samples. Overall, the results of the construct validity using the EFA and the CFA and internal consistency supported a tridimensional construct of a 25-item scale: situational factors, negative self-perceptions and mental health factors, and suicidal thoughts and behaviors.

Many tools have been constructed that focus on the unifactorial side of suicidal risks, such as suicidal ideation or suicidal behavior. However, this study was based on the aim of measuring suicide risk multidimensionally. To accomplish this task, scale development was conducted through a comprehensive and systemic procedure. First, a multidisciplinary team was formed to examine the concept of suicide from various perspectives by reviewing psychological autopsies and statistical data across various strata in academic fields. According to psychological autopsy reports, victims displayed warning signs verbally, behaviorally, and emotionally prior to their death. The statistical data of the national mental illness survey identified the characteristics of suicidal attitudes and behaviors in South Korea, and the survey of suicide deaths confirmed that mental health problems, including alcohol and drug use disorders, affected suicide risk. Based on the committee’s emphasis on the importance of evaluating suicide risk in light of various aspects of an individual’s life, such as social, economic, cultural, and psychological characteristics, 77 draft items consisting of 14 questions on suicide warning signs, 15 questions on mental health problems, 8 questions on personality, 12 questions on suicidal attitude, and 29 questions on socioeconomic factors were generated. Furthermore, content validity was established twice through a review of practitioners and experts in the field of suicide (psychiatrists, clinical nurses, clinical social workers, clinical psychologists, and counseling psychologists) by asking them whether the questions were appropriate for screening risk groups for suicide, and the readability of each item. Ultimately, 29 questions were retained after receiving these expert opinions.

In this study, data were collected through online and offline questionnaires, and 2238 adults’ responses were analyzed. The factor structure, reliability, validity, and cut-off points of the SSQ-SR were checked. The EFA was run on 29 items, yielding a clearly defined three-factor structure with items’ with factor loadings of <.4 deleted. Next, the factor structure obtained from the EFA was confirmed by performing a CFA. The goodness of fit of the three-factor scale without one item (item 24) with a factor loading less than .4 was confirmed. The final 3-factor structure of the 25-item SSQ-SR, with 6 items on situational factors, 12 items on negative self-perceptions and mental health factors, and 6 items on suicidal thoughts and behaviors, showed very strong factor loadings with a superior model fit. What sets this scale apart from existing scales that consist of several sub-factors measuring only a single aspect is that each subscale of the SSQ-SR reflects various areas related to suicide.

The reliability and validity of the SSQ-SR were examined using a non-clinical sample. High internal consistency indices indicate a coherent instrument. Internal consistency is usually used for measuring single factors, and tau-equivalence must be satisfied, which is rarely achieved in practice. Therefore, as an alternative, the omega coefficient of the scale, which can more appropriately measure the reliability of a scale composed of multiple factors (Helms et al., 2006), was confirmed to have suitable values for each sub-factor as well as the overall scale. Such results indicate that each item consistently captures what the overall scale and each sub-factor intend to measure.

Additionally, the SSQ-SR demonstrated good concurrent validity when checking the correlation between suicidal ideation, a conceptually similar measure, and depression, one of the most common comorbidities related to suicide risk. Both suicidal ideation and depression were significantly correlated with the SSQ-SR, indicating that it is a valid tool for assessing suicide risk. This is consistent with many previous studies that support the relationship between preceding or comorbid phenomena such as depression symptoms and various forms of suicidal actions (de Paula et al., 2020; Farabaugh et al., 2012; Melhem et al., 2019; Vandivort & Locke, 1979).

In this study, the discriminant validity of the SSQ-SR was ascertained through a group difference verification. The overall SSQ-SR and situational factors, negative self-perceptions, and mental health factors showed significant differences between the risk groups (SA and SI groups) and the normal control group, without significant differences between the SA and SI groups. Individuals with suicidal ideation and attempts are more likely to be affected by suicide-related situational factors and have negative self-perceptions and mental health problems compared with the general population. In the case of suicidal thoughts and behavioral factors, differences were confirmed between the SA, SI, and normal groups. The probability was found to be high in the order of SA group, SI group, and normal control group, which is in line with many previous studies that supported the differences between individuals with suicide attempters and suicide ideators, with attempters usually demonstrating more severe symptoms or risk factors (Brezo et al., 2006; Pérez et al., 2017; Wiebenga et al., 2021).

This study also aimed to identify the prevalence of suicide risk in high- and low-risk groups. A ROC analysis was performed on all the participants based on whether they had a history of suicide attempts and suicidal ideation within the last 6 months to confirm the discriminatory power of the suicide risk assessment tool. As a result, the optimal cut-off scores were found to be 33.5 for the low-risk group and 36.5 for the high-risk group, and the AUC value of .842 and .863 for the high- and low-risk groups, respectively, were judged to be appropriate (Holmes, 1998). Accordingly, all the values of sensitivity, specificity, NPP, and PPP were found to be appropriate, indicating that these values can be seen as having high utility as a tool for distinguishing between the high- and low-risk groups and normal individuals in non-clinical samples in considering multidimensional perspectives.

Although the results of the present study are promising, several limitations have emerged, with suggested directions for future research. First, the sample’s age range is limited to adults over 18, meaning that the scale may not be generalizable across ages, including adolescents. In the past 2 years, people have had to adapt to sudden life-changing impacts on daily life owing to the COVID-19 pandemic; in particular, those in their teens and twenties have had more difficulty adapting to the current situation. Therefore, future research is necessary to develop unique scales that reflect the characteristics of specific target age groups, such as adolescents and the elderly. Second, data collection was conducted using a self-report questionnaire, and the SSQ-SR was developed as a self-report method. Since there are limitations in detecting the distortion of symptoms, including exaggerating or minimizing symptoms, data collection via multiple assessment methods such as informant reporting and interviews would more accurately examine the validity of the data. Furthermore, developing an observant scale for experts would better reflect the severity of symptoms. Third, owing to the situation brought on by the COVID-19 pandemic, the offline survey in this study was mainly conducted on people in their 20 and 30’s. In the future, in the absence of a pandemic, the SSQ-SR scale should be applied to all age groups using a national offline survey. Fourth, the nature of this cross-sectional study limits the scale’s consistency. If possible, a longitudinal validation study assessing the predictive validity and test-retest reliability may provide more insight into further validation of the scale. Moreover, it is necessary to analyze psychological autopsies later and accumulate additional statistical data to further elaborate on the items to screen for suicide risk. Finally, a non-clinical sample was recruited with comparatively low rates of suicidal ideation and behaviors. Therefore, future studies replicating the factor structure in a clinical population or a comparative study of clinical and non-clinical groups would lead to a broader understanding of the scale.

Despite these limitations, the findings of this study demonstrate that the SSQ-SR is a valid and reliable tridimensional scale with robust psychometric properties for assessing and screening suicide risk. One of the key strengths of the present study is the large nationwide sample size, retrieved through an online survey, which is representative of the South Korean population. In addition, while the C-SSRS or the BSSI, widely used to assess suicide risk, developed questions focusing on suicide ideation, our self-reporting multidimensional scale was developed based on the psychological autopsy of the bereaved families of those who committed suicide by the CPAC and the KNPA under the MHW, highlighting the characteristics and observed patterns of actual suicide deaths. Additionally, statistical data on suicide deaths and mental illness surveys were also analyzed comprehensively, leading to the evaluation of suicide risk considering social, economic, cultural, and psychological aspects. Therefore, the three-factor structure derived through factor analysis led to a broader examination of suicide risk in terms of situational and negative self-perceptions, mental health, and suicidal thoughts and behaviors. Lastly, the optimal cut-off point enabled the screening of low- and high-risk groups, which contributes to the early detection of individuals exposed to suicide risk for appropriate targeted programs and interventions across different groups. To the best of our knowledge, this is the first study to use a scale to assess the multidimensional aspects of suicide and screen various groups by applying cut-off scores according to the severity of suicide risk.

OECD member countries are paying increased attention to mental health problems caused by the prolonged impact of COVID-19 (OECD, 2021). In view of the Spanish flu outbreak of 1918, it is expected that COVID-19 will cause long-term mental health problems, including self-harm and suicide-related thoughts and behaviors (Beach et al., 2022; Hasking et al., 2021). As the number of suicide deaths increased during the COVID-19 pandemic, experts and practitioners have been developing guidelines to quickly detect suicide risk groups for practical prevention. Considering all these circumstances, developing a scale to measure suicide-related thoughts and behaviors is meaningful. In this light, we see the potential for not only inviting domestic and international research on SSQ-SR, reflecting the unique cultural, social, and psychological factors of different countries, but also promoting effective usage in suicide prevention approaches across various multidisciplinary fields, thereby contributing to the reduction of suicide rates on a global level.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2021S1A3A2A02089682), Korean Mental Health R&D Project, funded by the Ministry of Health & Welfare, Republic of Korea (HL19C0001; PI, HJ), and the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (HR21C0885).

Author Contributions

DL and HJ contributed to the research conception, design, and process. They are responsible for the survey design and development, data collection, statistical analysis consultation, and research funding. The first draft of the manuscript was written by HL and DL, and the draft has been thoroughly revised several times by DL, VY, and JJ. All authors commented on previous versions of the manuscript, read, and approved the final manuscript.

Data availability statement

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Ethics approval

The present study was approved by the Institutional Review Board (IRB) from SungKyunKwan University.

ORCID iD

Dong Hun Lee

Author Biographies

Hwa Jung Lee, PhD Candidate Researcher, Traumatic Stress Center, Department of Education, Sungkyunkwan University.

Dong Hun Lee, Professor, Traumatic Stress Center, Department of Education, Sungkyunkwan University.

Hong Jin Jeon, Professor, Samsung Medical Center, Sungkyunkwan University, Seoul, South Korea.

Deok Hee Lee, PhD Candidate Researcher, Traumatic Stress Center, Department of Education, Sungkyunkwan University.

Vidal Yook, MA researcher, Department of Psychiatry, Depression Center, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea.

Ji Yeol Jeon, PhD Researcher, Traumatic Stress Center, Department of Education, Sungkyunkwan University.

References

Al-Halabí

Sáiz

P. A.

Burón

Garrido

Benabarre

Jiménez

Cervilla

Navarrete

M. I.

Diaz-Mesa

E. M.

Garcia-Alvarez

Muniz

Posner

Oquendo

M. A.

Garcia-Portilla

M. P.

Bobes

(2016). Validation of a Spanish version of the Columbia-suicide severity rating scale (C-SSRS). Revista de Psiquiatría y Salud Mental (English Edition), 9(3), 134–142. https://doi.org/10.1016/j.rpsmen.2016.06.004

Baek

I. C.

Kim

E. J.

Lee

G. R.

Lee

D. H.

Jeon

H. J.

(2021). A review of suicide risk assessment tools and their measured psychometric properties in Korea. Frontiers in Psychiatry, 12, 679779. https://doi.org/10.3389/fpsyt.2021.679779

Bagge

C. L.

Littlefield

A. K.

Glenn

C. R.

(2017). Trajectories of affective response as warning signs for suicide attempts: An examination of the 48 hours prior to a recent suicide attempt. Clinical Psychological Science, 5(2), 259–271. https://doi.org/10.1177/2167702616681628

Beach

Clay

Saavedra

(2022). The 1918 influenza pandemic and its lessons for COVID-19. Journal of Economic Literature, 60(1), 41–84. https://doi.org/10.1257/jel.20201641

Beck

A. T.

Schuyler

Herman

(1974). Development of suicidal intent scales. Charles Press Publishers.

Brezo

Paris

Turecki

(2006). Personality traits as correlates of suicidal ideation, suicide attempts, and suicide completions: A systematic review. Acta Psychiatrica Scandinavica, 113(3), 180–206. https://doi.org/10.1111/j.1600-0447.2005.00702.x

Bryant-Genevier

Rao

C. Y.

Lopes-Cardozo

Kone

Rose

Thomas

Byrkit

Lynfield

Shah

Freeman

Becker

Williams

Gould

D. W.

Tiesman

Lloyd

Hill

(2021). Symptoms of depression, anxiety, post-traumatic stress disorder, and suicidal ideation among state, tribal, local, and territorial public health workers during the COVID-19 Pandemic—United States, March–April 2021. Morbidity and Mortality Weekly Report, 70(26), 947–952. https://doi.org/10.15585/mmwr.mm7026e1

Castle

Kreipe

(2007). Suicidal behavior. In Pediatric clinical advisor. Mosby.

Choi

Y.-H.

Lee

E.-H.

Hwang

S.-T.

Hong

S.-H.

Kim

J.-H.

(2020). Reliability and Validity of the Beck Scale for Suicide Ideation (BSS) in Korean Adult Participants. Korean Journal of Clinical Psychology, 39(2), 111–123. https://doi.org/10.15842/kjcp.2020.39.2.003

10.

Conway

P. M.

Erlangsen

Teasdale

T. W.

Jakobsen

I. S.

Larsen

K. J.

(2017). Predictive validity of the Columbia-suicide severity rating scale for short-term suicidal behavior: A Danish study of adolescents at a high risk of suicide. Archives of Suicide Research, 21(3), 455–469. https://doi.org/10.1080/13811118.2016.1222318

11.

Czeisler

M. É.

Lane

R. I.

Petrosky

Wiley

J. F.

Christensen

Njai

Weaver

M. D.

Robbins

Facer-Childs

E. R.

Barger

L. K.

Czeisler

C. A.

Howard

M. E.

Rajaratnam

S. M.

(2020). Mental health, substance use, and suicidal ideation during the COVID-19 pandemic—United States, June 24–30, 2020. Morbidity and Mortality Weekly Report, 69(32), 1049–1057. https://doi.org/10.15585%2Fmmwr.mm6932a1

12.

de Paula

Breguez

G. S.

Machado

E. L.

Meireles

A. L.

(2020). Prevalence of anxiety, depression, and suicidal ideation symptoms among university students: A systematic review. Brazilian Journal of Health Review, 3(4), 8739–8756. https://doi.org/10.34119/bjhrv3n4-119

13.

Dilillo

Mauri

Mantegazza

Fabiano

Mameli

Zuccotti

G. V.

(2015). Suicide in pediatrics: Epidemiology, risk factors, warning signs and the role of the pediatrician in detecting them. Italian Journal of Pediatrics, 41(1), 1–8. https://doi.org/10.1186/s13052-015-0153-3

14.

Farabaugh

Bitran

Nyer

Holt

D. J.

Pedrelli

Shyu

Hollon

S. D.

Zisook

Baer

Busse

Petersen

T. J.

Pender

Tucker

D. D.

Fava

(2012). Depression and suicidal ideation in college students. Psychopathology, 45(4), 228–234. https://doi.org/10.1159/000331598

15.

Franklin

J. C.

Ribeiro

J. D.

Fox

K. R.

Bentley

K. H.

Kleiman

E. M.

Huang

Musacchio

K. M.

Jaroszewski

A. C.

Chang

B. P.

Nock

M. K.

(2017). Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin, 143(2), 187–232. https://doi.org/10.1037/bul0000084

16.

Granello

D. H.

(2010). The process of suicide risk assessment: Twelve core principles. Journal of Counseling & Development, 88(3), 363–370. https://doi.org/10.1002/j.1556-6678.2010.tb00034.x

17.

Hasking

Lewis

S. P.

Bloom

Brausch

Kaess

Robinson

(2021). Impact of the COVID-19 pandemic on students at elevated risk of self-injury: The importance of virtual and online resources. School Psychology International, 42(1), 57–78. https://doi.org/10.1177/0143034320974414

18.

Hawton

van Heeringen

(2009). Background and epidemiology. Lancet, 373(1), 1372–1381. https://doi.org/10.1016/s0140-6736(09)60372-x

19.

Helms

J. E.

Henze

K. T.

Sass

T. L.

Mifsud

V. A.

(2006). Treating Cronbach’s alpha reliability coefficients as data in counseling research. The Counseling Psychologist, 34(5), 630–660. https://doi.org/10.1177/0011000006288308

20.

Holmes

W. C.

(1998). A short, psychiatric, case-finding measure for HIV seropositive outpatients: Performance characteristics of the 5-item mental health subscale of the SF-20 in a male, seropositive sample. Medical Care, 36(2), 237–243. https://doi.org/10.1097/00005650-199802000-00012

21.

Hong . (2000). Criteria for selecting the fitness index of the structural equation model and its basis. Korean Journal of Clinical Psychology, 19(1), 161–177.

22.

Isometsä

E. T.

(2001). Psychological autopsy studies–a review. European Psychiatry, 16(7), 379–385. https://doi.org/10.1016/s0924-9338(01)00594-6

23.

Kaiser

H. F.

(1974). An index of factorial simplicity. psychometrika, 39(1), 31–36. https://doi.org/10.1007/bf02291575

24.

Kim

J.-E.

Kang

E.-J.

Jeong

J.-W.

(2013). Korean Suicide Risk Screening Tool and its Validity. JOURNAL OF THE KOREA CONTENTS, ASSOCIATION, 13(3), 240–250. https://doi.org/10.5392/jkca.2013.13.03.240

25.

Klonsky

E. D.

May

A. M.

Saffer

B. Y.

(2016). Suicide, suicide attempts, and suicidal ideation. Annu Rev Clin Psychol, 12(1), 307–30. https://doi.org/10.1146/annurev-clinpsy-021815-093204

26.

You

(2020). Development and validation of the suicidal imagery questionnaire. Kor J Clin Psychol, 39, 1–14. https://doi.org/10.15842/kjcp.2020.39.1.001

27.

Korea Psychological Autopsy Center . (2019). Korea-psychological autopsy checklist 2.1 guideline. Korea Psychological Autopsy Center.

28.

Korea Suicide Prevention Center (2021). White Paper on Suicide Prevention. Seoul: Ministry of Health and Welfare.

29.

Korean Law Information

Center.

(2012). Act for the prevention of suicide and the creation of culture of respect for life. (KOR).

30.

Krug

E. G.

Mercy

J. A.

Dahlberg

L. L.

Zwi

A. B.

(2002). The world report on violence and health. Lancet, 360(9339), 1083–1088. https://doi.org/10.1016/s0140-6736(02)11133-0

31.

Lee Jeon Jung Kim Lee Yook (2022) (In press). A Preliminary Study for Development and Validation of Suicide Screening Questionnaire Self-Rating (SSQ-SR) (Suicide Risk Assessment tool): A Study for General adult. The Korean Journal of Counseling and Psychotherapy, 34(3).

32.

Lindh

Å. U.

Waern

Beckman

Renberg

E. S.

Dahlin

Runeson

(2018). Short term risk of non-fatal and fatal suicidal behaviours: The predictive validity of the Columbia-suicide severity rating scale in a Swedish adult psychiatric population with a recent episode of self-harm. BMC Psychiatry, 18(1), 1–9. https://doi.org/10.1186/s12888-018-1883-8

33.

Lotito

Cook

(2015). A review of suicide risk assessment instruments and approaches. Mental Health Clinician, 5(5), 216–223. https://doi.org/10.9740/mhc.2015.09.216

34.

Melhem

N. M.

Porta

Oquendo

M. A.

Zelazny

Keilp

J. G.

Iyengar

Burke

Birmaher

Stanley

Mann

J. J.

Brent

D. A.

(2019). Severity and variability of depression symptoms predicting suicide attempt in high-risk individuals. JAMA Psychiatry, 76(6), 603–613. https://doi.org/10.1001/jamapsychiatry.2018.4513

35.

Mérelle

Van Bergen

Looijmans

Balt

Rasing

van Domburgh

Nauta

Sijperda

Mulder

Gilissen

Franx

Creemers

Popma

(2020). A multi-method psychological autopsy study on youth suicides in The Netherlands in 2017: Feasibility, main outcomes, and recommendations. Plos One, 15(8), Article e0238031. https://doi.org/10.1371/journal.pone.0238031

36.

O’Connor

R. C.

Nock

M. K.

(2014). The psychology of suicidal behaviour. The Lancet Psychiatry, 1(1), 73–85. https://doi.org/10.1016/s2215-0366(14)70222-6

37.

OECD . (2021). COVID-19 and well-being: Life in the pandemic, OECD Publishing.

38.

Pérez

Marco

J. H.

García-Alandete

(2017). Psychopathological differences between suicide ideators and suicide attempters in patients with mental disorders. Clinical Psychology & Psychotherapy, 24(4), 1002–1013. https://doi.org/10.1002/cpp.2063

39.

Posner

Brent

Lucas

Gould

Stanley

Brown

Mann

(2008). Columbia-suicide severity rating scale (C-SSRS). Columbia University Medical Center.

40.

Posner

Brown

G. K.

Stanley

Brent

D. A.

Yershova

K. V.

Oquendo

M. A.

Mann

J. J.

(2011). The Columbia–Suicide Severity Rating Scale: Initial validity and internal consistency findings from three multisite studies with adolescents and adults. American Journal of Psychiatry, 168, 1266–1277. https://doi.org/10.1176/appi.ajp.2011.10111704

41.

Posner

Oquendo

M. A.

Gould

Stanley

Davies

(2007). Columbia classification algorithm of suicide assessment (C-CASA): Classification of suicidal events in the FDA’s pediatric suicidal risk analysis of antidepressants. American Journal of Psychiatry, 164(7), 1035–1043. https://doi.org/10.1176/ajp.2007.164.7.1035

42.

Rozanov

V. A.

Rakhimkulova

A. S.

(2017). Suicidal ideation in adolescents—a transcultural analysis. Handbook of Suicidal Behaviour, 2017, 267–285. https://doi.org/10.1007/978-981-10-4816-6_15

43.

Rudd

M. D.

Berman

A. L.

Joiner

T. E.

Nock

M. K.

Silverman

M. M.

Mandrusiak

Van Orden

Witte

(2006). Warning signs for suicide: Theory, research, and clinical applications. Suicide and Life-Threatening Behavior, 36(3), 255–262. https://doi.org/10.1521/suli.2006.36.3.255

44.

Shneidman

(1977). Definition of suicide. Jason Aronson.

45.

Steele

I. H.

Thrower

Noroian

Saleh

F. M.

(2018). Understanding suicide across the lifespan: A United States perspective of suicide risk factors, assessment & management. Journal of Forensic Sciences, 63(1), 162–171. https://doi.org/10.1111/1556-4029.13519

46.

Steer

R. A.

Rissmiller

D. J.

Ranieri

W. F.

Beck

A. T.

(1993). Dimensions of suicidal ideation in psychiatric inpatients. Behaviour Research and Therapy, 31(2), 229–236. https://doi.org/10.1016/0005-7967(93)90090-h

47.

Tucker

R. P.

Crowley

K. J.

Davidson

C. L.

Gutierrez

P. M.

(2015). Risk factors, warning signs, and drivers of suicide: What are they, how do they differ, and why does it matter? Suicide and Life-Threatening Behavior, 45(6), 679–689. https://doi.org/10.1111/sltb.12161

48.

Vandivort

D. S.

Locke

B. Z.

(1979). Suicide ideation: Its relation to depression, suicide and suicide attempt. Suicide and Life-Threatening Behavior, 9(4), 205–218. https://doi.org/10.1111/j.1943-278X.1979.tb00439.x

49.

Wiebenga

J. X.

Eikelenboom

Heering

H. D.

van Oppen

Penninx

B. W.

(2021). Suicide ideation versus suicide attempt: Examining overlapping and differential determinants in a large cohort of patients with depression and/or anxiety. The Australian and New Zealand Journal of Psychiatry, 55(2), 167–179. https://doi.org/10.1177/0004867420951256

50.

World Health Organization . (2021). Live life: An implementation guide for suicide prevention in countries. WHO.

51.

Wreen

(1988). The definition of suicide. Social Theory and Practice, 14(1), 1–23. https://doi.org/10.5840/soctheorpract19881416

52.

Zakhour

Haddad

Sacre

Fares

Akel

Obeid

Salameh

Hallit

(2021). Suicidal ideation among Lebanese adults: Scale validation and correlates. BMC Psychiatry, 21(1), 1–11. https://doi.org/10.1186/s12888-021-03111-7

53.

Zhang

Liu

Fang

(2019). Combined effects of depression and anxiety on suicide: A case-control psychological autopsy study in rural China. Psychiatry Research, 271, 370–373. https://doi.org/10.1016/j.psychres.2018.11.010

54.

Zhou

Wang

Jia

(2019). Being left-behind, mental disorder, and elderly suicide in rural China: A case–control psychological autopsy study. Psychological Medicine, 49(3), 458–464. https://doi.org/10.1017/S003329171800106X