Abstract
The literature on the utility of self-report instruments in determining caseness for posttraumatic stress disorder in South Africa is sparse. We administered the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders–Research Version and the Posttraumatic Stress Scale–Self-report version to a sample of 500 South African community members seeking HIV testing. Of our original sample of 500, 306 (61.2%) reported an index event for posttraumatic stress disorder and 25 (5.0%) met the criteria for this diagnosis. The Posttraumatic Stress Scale–Self-report displayed internal consistency of .95 as measured by Cronbach’s alpha. Using the Structured Clinical Interview for the Diagnostic and Statistical Manual of Mental Disorders as a gold standard, we conducted receiver operating curve analysis among the 306 participants who reported an index traumatic event to determine the extent to which the Posttraumatic Stress Scale–Self-report as a screening instrument successfully discriminated between participants who did and did not meet the diagnostic criteria for posttraumatic stress disorder. The Posttraumatic Stress Scale–Self-report yielded sensitivity of .76 and specificity of .78, with an area under the curve of .837. Positive and negative predictive values were .24 and .97, respectively. Our findings suggest that the Posttraumatic Stress Scale–Self-report may be effectively used to screen for posttraumatic stress disorder among community samples, including persons seeking HIV testing.
Considerable evidence suggests that posttraumatic stress disorder (PTSD) is associated with HIV. PTSD is associated with many adverse health outcomes, including poor treatment adherence and vulnerability to depression and suicide. In low-resource environments such as South Africa, there are few appropriately trained mental health care professionals working in HIV testing centres and clinics to diagnose and treat individuals with PTSD. There is thus a need to identify appropriate and effective screening instruments to detect individuals with PTSD in these settings. The need to establish the effectiveness of screening instruments for PTSD is also important in the context of the changes that were made to the diagnostic criteria for PTSD in the most recent version of the Diagnostic and Statistical Manual of Mental Disorders (DSM). These changes in diagnostic criteria call into question the effectiveness of previously validated PTSD screening instruments, which were anchored to the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV). It is within this context that we tested the utility of Posttraumatic Stress Scale–Self-report version in screening for PTSD among individuals seeking HIV testing in South Africa.
Psychologically disordered individuals often go undetected by the health care system (Coyne, Schwenk, & Fechner-Bates, 1995). Proponents of routine screening have called attention to its utility in identifying cases of individuals living with a mental disorder as well as those who are at risk of developing such a disorder. Screening may increase rates of detection and treatment of mental disorders and thus enhance clinical outcomes for patients who have contact with the health system. The advantages of routine screening include their ease and economy of use by persons with low levels of psychiatric training. Screening is thus potentially able to identify persons in several settings such as public health clinics, cancer patients, survivors of trauma, and persons who test positive for HIV, who need mental health or psychosocial services.
Screening for PTSD
Several instruments are available for screening for PTSD, including the PTSD checklist (Blanchard, Jones-Alexander, Buckley, & Forneris, 1996), the PTSD Symptom Scale (PSS) (Brewin et al., 2002), and the UCLA PTSD Reaction Index (Steinberg, Brymer, Decker, & Pynoos, 2004). Most of these were developed and validated prior to the DSM 5 revisions to the diagnostic criteria for PTSD. Effective screening instruments have particular application in low-resource environments such as South Africa, where prevalence rates of mental disorders are high but the number of trained health professionals who are able to recognise and diagnose mental health problems is low. Accurate screening instruments for common mental disorders are also important for public health reasons as they may assist with surveillance and the collection of accurate epidemiological data about the prevalence of these disorders. Yet, the imperfect concordance between an elevated score on a screening instrument for a mental disorder and a clinically accurate diagnosis may be a threat to the utility of routine screening (Coyne, Thompson, Palmer, Kagee, & Maunsell, 2000). Advocates of routine screening often assume that mental health professionals are available and ready to follow up with persons who screen positive on the screening instrument. However, pursuing false positives (i.e., those persons who screen positive for a mental disorder but in fact do not meet its diagnostic criteria) is often time-consuming and requires the human and infrastructural resources to do so effectively.
The twin concerns of sensitivity and specificity (Yerushalmy, 1947) are relevant to the debate about the utility and desirability of screening for common mental disorders in resource-scarce environments. Sensitivity refers to an instrument’s ability to correctly identify persons who meet the diagnostic criteria for a disorder (Lalkhen & McCluskey, 2008). If a screening instrument has poor sensitivity, it will fail to detect many individuals who meet the criteria for the disorder, thus producing a large number of false-negative results. Specificity refers to an instrument’s ability to correctly detect persons who do not meet the diagnostic criteria for a disorder (Lalkhen & McCluskey, 2008). Poor specificity of a screening instrument will result in large numbers of persons who do not meet the criteria for a disorder screening positive, thus producing results with a large number of false positives. PTSD, despite its high prevalence in many communities worldwide, is under-researched in the context of routine mental health screening.
Revised diagnostic criteria for PTSD
In the DSM5, PTSD was removed from the group of Anxiety Disorders and was given its own distinct category, ‘Trauma and Stressor-related Disorders’ (American Psychiatric Association [APA], 2013). This change saw a narrowing of the definition of trauma and an explicit acknowledgement that not all forms of stress entail trauma. The new criteria require ‘actual or threatened death, serious injury, or sexual violence’, but exclude stressful events that do not include an immediate threat to life or physical injury. While the changes to diagnostic criteria are important and significant, it is not clear what effect, if any, they will have on the validity and reliability of existing PTSD screening tools.
PTSD and HIV
Available data suggest that the prevalence of PTSD among individuals living with HIV may be high (Brief et al., 2004; Martinez, Israelski, Walker, & Koopman, 2002; Safren, Gershuny, & Hendriksen, 2003). Among South Africans living with HIV, the prevalence of PTSD was estimated at 14.8% and 26% at baseline and follow-up, respectively, as assessed by the Mini International Neuropsychiatric Interview (MINI; Olley, Seedat, & Stein, 2006). In general, PTSD prevalence reported in these studies is higher than that found in a national survey of South Africans (Stein et al., 2008), that is, 2.3%.
While research exists on PTSD among persons living with HIV, to our knowledge, there are no studies among persons seeking HIV testing. Such data may provide a perspective into the experiences of trauma of this population prior to receipt of an HIV test result. In this article, we report on the prevalence of traumatic events, PTSD caseness and sub-threshold symptoms, and the performance of the Posttraumatic Stress Scale in detecting PTSD among South Africans seeking an HIV test.
Method
Participants
Participants were recruited by convenience sampling at three HIV testing sites in the Western Cape, South Africa. Testing site personnel establish temporary structures in public areas such as parking lots, taxi ranks, and other transport centres and invite community members to receive a free HIV test as they approach the testing site.
Instruments
PTSD
We assessed PTSD by means of the Structured Clinical Interview for the DSM–Research Version (SCID-RV). In consultation with one of the SCID developers, Dr Michael First, we adjusted the SCID questions so that they corresponded to the DSM5 (APA, 2013) diagnostic criteria. We obtained the draft version of the newest version of the SCID from Dr First and adapted the language of the instrument accordingly for use in our study. The language of the SCID questions was changed so that it conformed to the criteria for PTSD as delineated in the latest version of the DSM. We conducted a pilot study to ensure that participants found the SCID items comprehensible and feasible. Data collectors received training in how to administer the interviews. Participants’ responses were recorded on a Lenovo tablet. The data were then transferred into SPSS.
Posttraumatic stress symptoms
We assessed symptoms of posttraumatic stress using the Posttraumatic Symptom Scale, Self-report version (PSS-SR). The PSS-SR has 17 items that assess the presence and severity of PTSD symptoms (Foa, Riggs, Dancu, & Rothbaum, 1993). It has three sub-scales which are Re-experiencing, Avoidance, and Hyper arousal. Severity of the symptoms is rated on a 4-point Likert-type scale ranging from 0 (not at all or only one time) to 3 (almost always or five or more times per week). The scale has shown to have good internal consistency (Cronbach’s alpha = .97) and high test–retest reliability (.95) in various populations (Coffey, Dansky, Falsetti, Saladin, & Brady, 1998; Stieglitz, Frommberger, Foa, & Berger, 2001). Among rape and non-sexual violence survivors, the PSS was found to have an internal consistency of .91; test–retest reliability of .74; and concurrent validity with the Impact of Event Scale and the State-Trait Anxiety Inventory that ranged from .52 to .81 (Foa et al., 1993).
Procedure
When individuals presented themselves at the reception desk of the testing site, they were given an information sheet about the study and were invited them to meet with a researcher in a private room, caravan, or temporary structure, depending on where the HIV testing was taking place. The researcher then informed people about the study. Those who agreed completed an informed consent form. The vast majority of participants were able to converse in English, even though it was their second language. Those who understood no English (n = 40) were excluded from the study.
Interviews took up to an hour to complete. In addition to the PTSD module, we also administered modules for depressive and anxiety disorders and alcohol use disorder. At various points in the study, six interviewers administered the SCID interviews. They underwent rigorous training that included viewing the SCID training videos which we purchased from the SCID developers, role-playing the administration of the relevant interview modules with other trainees, conducting a mock interview with the principal investigator and, when they were deemed ready, conducting a live interview with a participant with the principal investigator or senior team leader present.
Ethical considerations
We received approval from the Stellenbosch University Health Ethics Committee. Individuals who had clinically significant distress or a mental disorder were referred for further services.
Data analysis
We used SPSS version 24 to analyse the data. We calculated frequencies, means, and standard deviations of the scales used. Internal consistency of the PSS was calculated by means of Cronbach’s alpha. We used receiver operating characteristic (ROC) curve analysis to determine the ability of the PSS-SR to discriminate between individuals who did and did not meet the criteria for a PTSD diagnosis on the SCID. The area under the curve (AUC) provides an indication of the diagnostic ability of the PSS, which ranges from random (.50) to perfect performance (1.00). Values between .5 and .7 indicate low discriminatory ability; values between .7 and .9 indicate moderate discriminatory ability; and values above .9 indicate high discriminatory ability of a measure (Swets, 1988).
Results
Description of the sample
The sample consisted of 500 participants, of which 48.4% were males and 51.6% were females. The mean age was 36 years and most participants (72.6%) classified themselves as mixed race (Coloured), followed by 26.2% African and 0.8% White. More than two-thirds (69.0%) indicated that Afrikaans was their first language, and 6.0% and 19.6% indicated that English and Xhosa were their first languages, respectively. Nearly half the sample (46.6%) indicated that they were unemployed and had a family income of less than R10,000 (40.6%) per annum.
Prevalence of PTSD
Of the total sample of 500 participants, 306 (61.2%; confidence interval [CI] = [56.8%, 65.5%]) reported experiencing Criterion A of the DSM5 diagnostic criteria for PTSD, which specifies exposure to actual or threatened death, serious injury, or sexual violence. However, of the 500 participants, only 25 (5.0%; CI = [3.30%, 7.30%]) met the diagnostic criteria for PTSD on the SCID-RV. Thus, 25 of the 306 (8.17%) who reported a traumatic event as specified by Criterion A also met the criteria for PTSD. In other words, 281 (91.83%) of individuals who experienced an index traumatic event did not develop PTSD.
Symptoms of traumatisation
Table 1 presents a descriptive analysis of PSS-SR items among those persons who reported an index traumatic event, including the percentages of the sample who endorsed each item. In the last column on Table 1, we summed the percentages of participants who endorsed each item as ‘somewhat’ and ‘very much’. The items ‘Feeling emotionally upset when you are reminded of the event’ (25.8%), ‘Trying not to think or talk about the traumatic event’ (28.4%), ‘Trying to avoid activities or people that remind you of the traumatic event’ (35.3%), ‘Feeling emotionally distant or cut-off from the people around you’ (27.4%), ‘Feeling as if your future hopes or plans will not come true’ (29.1%), ‘Having trouble falling or staying asleep’ (27.8%), and ‘Being overly alert’ (31.7%) received the levels of endorsement of over 25%.
Percentage of the sample endorsing PSS items (N = 306).
As can be seen in Table 1, among those participants who reported an index traumatic event, responses on the PSS-SR were skewed towards less symptomatology. The mean score for the PSS was 13.92 (standard deviation [SD] = 13.4) with a range from 0 to 51 (out of a possible 51). The median score was 9.0. Internal consistency as measured by Cronbach’s alpha for the PSS was elevated (.95). All items had high corrected item-total correlations.
ROC analysis for PSS-SR
We conducted an ROC analysis for those participants who endorsed the A criterion, that is, those who reported a traumatic event. The ROC curve in Figure 1 displays the performance of the PSS in determining an optimal balance of sensitivity and specificity against caseness for PTSD as determined by the SCID-RV. As can be seen in Table 2, the AUC of .837 (CI = [.786, .889]) indicates that the PSS-SR is a useful screening tool in discriminating between persons who met the criteria for PTSD and those who did not. The AUC of .837 suggests that the PSS-SR performed significantly better than chance at discriminating between cases and non-cases of PTSD. According to Zweig and Campbell (1993), an AUC of .837 means that a randomly selected person who meets the criteria for PTSD is 83.7% more likely to have a positive score than a randomly selected person without PTSD.

Area under the curve.
Area under the curve (PSS-SR).
The test result variable(s): Total score of PSS has at least one tie between the positive actual state group and the negative actual state group.
Table 3 presents the coordinates of the curve for the PSS. As can be seen, the optimal cut-point on the PSS as determined by the smallest difference between sensitivity and specificity was 22.5. For this cut-point, sensitivity of the PSS was .76 and specificity was .78. Thus, using the cut-point of 22.5, the scale was successful in determining caseness for PTSD 76% of the time and non-caseness 78% of the time.
Coordinates of the curve: test result variable(s) for total score of PSS.
The test result variable(s): total score of PSS has at least one tie between the positive actual state group and the negative actual state group.
‘a’: the smallest cut-off value is the minimum observed test value minus 1, and the largest cut-off value is the maximum observed test value plus 1. All the other cut-off values are the averages of two consecutive ordered observed test values.
Table 4 shows that the positive predictive value (PPV) of the PSS was .24 and the negative predictive value (NPV) was .97. These data indicate a 24% probability that individuals who scored above 22.5 would meet the diagnostic criteria for PTSD, and a 97% probability that individuals who scored below 22.5 would not meet the criteria for the disorder. It is thus apparent that the PSS is highly successful at predicting non-caseness for PTSD but less successful at predicting caseness for this diagnosis.
Results of the receiver operating characteristic (ROC) curve analysis among 500 persons seeking HIV testing.
PTSD: posttraumatic stress disorder; PSS: posttraumatic stress scale; CI: confidence interval; PPV: positive predictive value; NPV: negative predictive value; AUC: area under the curve.
Discussion
Of our original sample of 500, 306 (61.2%) reported experiencing a traumatic event that was considered an index event for PTSD. Thus, traumatic events are common in the communities where the study was conducted. In socially and economically marginalised communities, traumatic events such as physical violence, accidents, and sexual assault are common. Traumatic experiences such as rape, attempted rape, sexual assault, and spousal violence create conditions of risk for HIV infection. Thus, regular HIV testing is of considerable importance.
Of the 17 items, 7 of the PSS were endorsed by more than 25% of participants. These items were as follows: feeling emotionally upset when reminded of the traumatic event; trying not to think about the traumatic event; trying to avoid activities or people that remind you of the traumatic event; feeling distant or cut-off from the people around you; feeling as if your future hopes or plans will not come true; having trouble falling or staying asleep; and being overly alert. The high level of endorsement of these items indicates that symptoms of PTSD are common among individuals seeking HIV testing. As the sample was recruited from community members, it appears that symptoms of PTSD are common in the communities where the study was conducted.
Our data suggest that the PSS-SR is a useful screening instrument in the context of routine HIV testing to determine which individuals require further assessment and possible treatment for PTSD. The robust AUC statistic of .837 indicates that persons who have PTSD are 83.7% more likely to have an elevated total score on the PSS than those who do not have PTSD. According to Swets (1988), an AUC of 83.7% indicates that a screening instrument, in this case the PSS-SR, may be considered a useful measure of screening for a target condition, that is, PTSD.
The PSS-SR performed adequately among our sample and similar to other samples in terms of its ability to discriminate between PTSD caseness and non-caseness. For example, the German version of the PSS-SR, the PTSD Diagnostic Scale (PDS), yielded an AUC of .81 (CI = [.75, .87]). Among train crash and crime victims, sensitivity and specificity of the PSS-SR compared with the Clinician-Administered PTSD Scale (CAPS-1) ranged from .5 to .86 and .93 to .97, respectively (Brewin et al., 2002). When compared to the Composite International Diagnostic Interview (CIDI) among South Africans living with HIV, the PSS-SR yielded a somewhat lower AUC of 74.9% (CI = [64%, 85%]; Martin, Fincham, & Kagee, 2009). Martin et al.’s study (2009) found that the optimal PSS-SR cut-point of ≥15 yielded sensitivity of .68 and specificity of .65, which was considerably lower than the results of this study.
Our findings suggest that the PSS may be appropriately used to screen for PTSD in South African communities, in this case persons seeking an HIV test. Considering the NPV of 97.3, persons who scored below the cut-point of 22.5 on the PSS-SR may be correctly regarded as not having PTSD. However, the PPV of 23.5% suggests that the PSS-SR is considerably less successful at predicting caseness for this diagnosis. It therefore appears that a two-stage approach to case identification may be most appropriate, in which the PSS-SR as a screening instrument might be applied at the first stage. Those persons who scored below the cut-point of 22.5 may indeed be regarded as true non-cases, while those who scored above 22.5 may then proceed to the second stage where they would receive a diagnostic interview to assess whether or not they meet the criteria for PTSD. However, such a two-step process assumes that appropriately trained mental health care workers are available to receive these referrals.
Case identification of persons living with a common mental disorder is important for a range of reasons. Untreated PTSD has a devastating effect on quality of life and may lead to poor functioning, dysfunctional social relationships, work problems that include absenteeism and loss of employment, and poor family functioning (Dekel & Monson, 2010; Evans, Cowlishaw, Forbes, Parslow, & Lewis, 2010; Gewirtz, Polusny, DeGarmo, Khaylis, & Erbes, 2010). Among HIV test seekers, PTSD may also make adaptation to a positive test result difficult, which may be a barrier to effective problem-solving, including gaining access to antiretroviral treatment (Vranceanu et al., 2008), and decision-making about appropriate disclosure to others (Whetten, Reif, Whetten, & Murphy-McMillan, 2008). Appropriate disclosure, including disclosure to sexual partners, is an important concern (Simbayi et al., 2007). For example, further infections may be prevented if serodiscordant couples adopt appropriate strategies to reduce risk such as the use of condoms and pre-exposure prophylactic medication.
Case detection for PTSD also introduces the need to apply evidence-based treatments to reduce symptoms associated with PTSD (Kagee, 2006). The most efficacious treatment for PTSD has been shown to be exposure therapy that includes imaginal and in vivo exposure (Cahill, Rothbaum, Resick, & Follette, 2009). However, the fact that efficacious treatments exist does not mean that they are easily available to those who need of them. HIV testing occurs in the context of a resource-constrained public health system in South Africa and referrals for psychological and psychiatric treatment are often not available for those who need them (Lund, Kleintjes, Kakuma, Flisher, & MHaPP Research Programme Consortium, 2010). Thus, screening practices for mental health problems should be considered alongside the availability of treatment so that those who screen positive for a mental disorder can receive follow-up services such as with diagnosis and treatment.
Despite the controversy surrounding routine screening for mental health disorders (see Coyne et al., 2000, for a review), efforts to identify individuals who may benefit from psychological treatment are important and necessary. Such persons include those living with or at risk of life-threatening conditions such as HIV.
Conclusion
Our findings suggest that the PSS-SR may be effectively used to screen for PTSD among community samples, including persons seeking HIV testing, and that this instrument has validity even within the context of the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM5) revisions to diagnostic criteria. Yet, given the sub-optimal specificity of the PSS-SR, a more robust screening measure for PTSD that corresponds more closely to the DSM5 criteria requires development and validation. Further health systems research is required to ensure the effective triaging and treatment of persons who screen positive for PTSD. Also, those persons who have elevated scores on the PSS-SR but are not PTSD cases may require psychological attention to ameliorate what may be termed sub-threshold symptoms of PTSD. Such interventions may also help offset risk behaviours associated with HIV infection.
To our knowledge, this is the first study to determine the utility of the PSS-SR in discriminating between cases and non-cases of PTSD using the SCID as a gold standard. The population of interest, that is, South African test-seekers, is also an understudied population, and the study provides important data to assist in decision-making about integrating mental health screening and services within HIV testing procedures. In terms of limitations, it is possible that individuals who enrolled in the study were a self-selected group who may have had more emotional distress or psychological concerns than those who did not enrol. Also, the PSS-SR showed limited success in predicting caseness for PTSD, and thus, future studies should assess the utility of screening instruments that are more aligned to the DSM5.
