Shyness 5: The Clinical Effectiveness of Internet-Based Clinician-Assisted Treatment of Social Phobia

Abstract

Objective: The present study explores the clinical effectiveness of an Internet-based cognitive behavioural treatment program for social phobia (the Shyness program) administered by a psychiatric registrar as part of standard clinical treatment at an outpatient mental health service (the Anxiety Disorders Clinic, St Vincent's Hospital, Sydney).

Method: Seventeen individuals with social phobia were assigned to this 8 week clinician-assisted program. All patients had access to online lessons, homework assignments, an online discussion forum with other patients, and regular emails from the clinician. Both completer and intention-to-treat models were used in data analyses.

Results: Fourteen (82%) patients completed all six lessons within the 8 weeks, and 11/17 (65%) completed post-treatment questionnaires. Mean within-group effect sizes (Cohen's d) for the two social phobia measures were 1.06 and 0.77, based on completer and intention-to-treat analyses, respectively. Data indicate that the procedure was acceptable to patients.

Conclusions: These results indicate that the procedure was clinically effective when operated as part of standard clinical treatment. These results provide further support for the potential clinical utility of the Shyness program.

Keywords

computerized cognitive behavioural therapy Internet social phobia treatment

The present study is part of a research program exploring the potential of computerized cognitive behavioural therapy (CCBT) to enhance the treatment of anxiety and depressive disorders. These high prevalence disorders affect significant proportions of the population and result in considerable disability. 1 Although effective psychological and pharmacological treatments exist, many people with these disorders do not seek treatment. Barriers to treatment include the limited availability of skilled mental health professionals, the inability of many patients to attend treatment during working hours, the expense of treatment, and the stigma of attending a mental health professional. 2 , 3

This study reports on the clinical effectiveness of the Shyness program, an Internet-based CCBT treatment program for people with social phobia. The clinical efficacy of the Shyness program, administered either as a self-guided program (CCBT), or with the assistance of a clinician (clinician-assisted CCBT, or CaCCBT) has been reported in four recent randomized controlled trials (RCTs). 4 – 7 These trials (total n = 438) have replicated the clinical efficacy of the Shyness treatment program with mean within-group effect sizes consistent with those obtained from face-to-face treatment programs for social phobia. 8 The present study contributes to this body of work by examining the clinical effectiveness of the Shyness program. Clinical effectiveness is concerned with clinical outcomes of a treatment when it is applied in standard clinical practice, beyond the rigour of research programs. In the present study, 17 adults with a DSM-IV diagnosis of social phobia who applied to participate in the Shyness program completed the CaCCBT version with a psychiatric registrar as part of outpatient treatment at the Anxiety Disorders Clinic, St Vincent's Hospital, Sydney. This paper reports the results for patients and the perceptions of the clinician, who had not participated in the RCTs.

METHOD

Recruitment, patients and outcome measures

Applicants applied online through a website designed for the research program (www.virtualclinic.org.all).

We provided material about the programme for newspaper articles and radio interviews, and interested people were directed to a website created specifically for the study (www.shyness.tv, now www.virtualclinic.org.au). This website provided information about social phobia, a description of the study, and a link to apply online to join the programme. The first stage of the recruitment process involved participants completing questionnaires online to determine that they: (i) were a resident of Australia; (ii) were at least 18 years of age, (iii) had access to a computer, the Internet, and use of a printer; (iv) were not currently participating in CBT; (v) were not using illicit drugs or consuming more than three standard drinks/day; (vi) were not currently experiencing a psychotic mental illness or severe symptoms of depression, defined as >22, or responding >1 to Question 9 (suicidal ideation) on the Patient Health Questionnaire − 9 Item; PHQ-9 9 ; and (vii) had been taking the same dose for at least 1 month and did not intend to change that dose during the course of the programme (if taking medication for social phobia). Applicants who failed one or more of these criteria were immediately informed via an on-screen message and were sent an email thanking them for their application, regretting that they did not meet the inclusion criteria and encouraging them to discuss their symptoms with their primary care physician. Details about participant flow are shown inFigure 1. Applicants who met the inclusion criteria completed the following questionnaires prior to treatment: the Social Interaction Anxiety Scale 10 (SIAS), the Social Phobia Scale 10 (SPS), the Patient Health Questionnaire − 9 Item 9 (PHQ-9), the Kessler 10 11 (K-10), and the Sheehan Disability Scales 12 (SDS). Changes in the SIAS and SPS were considered the primary social phobia outcome measures, and changes in the PHQ-9, K-10, and SDS were the secondary outcome measures.

Figure 1

Shyness 5 study flow chart. PHQ-9, MINI, Mini International Neuropsychiatric Interview; CBT, cognitive behavioural therapy; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition.

During the 2 weeks of recruitment, 25 individuals applied for the Shyness program through the www.climateclinic.tv website. Twenty-one people were able to be contacted and were administered the social phobia section of the Mini International Neuropsychiatric Interview Version 5.0.0 (MINI) 13 during a telephone interview to determine whether they met DSM-IV 14 diagnostic criteria for social phobia. Nineteen applicants met all inclusion criteria, and 17 began the program. Comorbidity with depression and other anxiety disorders was common. The mean age of patients was 42.47 years (SD = 9.54), and 29% were men. Seventy percent were employed full-time, 59% had participated in tertiary education, 41% were single or never married, 47% were married or in a de facto relationship, and 12% were separated or divorced. At post-treatment, patients completed the above questionnaires and a seven-item treatment satisfaction questionnaire based on the Credibility Expectancy Questionnaire. 15 , 16

Treatment procedure

The treatment conditions and procedure were described to each applicant during the telephone interview, and immediately prior to beginning the program each patient received, via email, an introductory guide to using the software. There were six online lessons, a summary/homework assignment for each lesson, comments by prior participants about each lesson, and access to the supplementary materials. The content of the Shyness program is based on best-practice evidence-based CBT techniques for social phobia. 17 Part of the content of each lesson was presented in the form of an illustrated story about a young man with social phobia who, with the help of a clinical psychologist, successfully gains mastery over his symptoms. Lessons 1 and 2 provided education about the symptoms and treatment of social phobia, lesson 3 provided instructions on how to develop an exposure hierarchy and about practicing graded exposure, lessons 4 and 5 reinforced principles of graded exposure and demonstrated principles of cognitive restructuring, while lesson 6 included information about relapse prevention. Each lesson begins with a summary of the material described in the previous lesson and the summary/homework assignment for that lesson describes relevant and practical techniques and exercises. Patients were expected to attempt these tasks and to regularly review the material. They were encouraged to complete the first four lessons within the first 2 weeks, in order to provide more opportunity to practice the graded exposure, cognitive skills and other coping techniques. All patients were asked to complete the six lessons within 8 weeks of starting.

Patients were expected to complete each homework assignment and post messages on a secure online discussion forum, using an alias. The clinician moderated the forum and aimed to respond to postings and to emails within 24 hours. After completing each lesson, patients were sent an email from the clinician. The themes of the clinician's emails varied from encouragement to practice the relevant treatment skills, encouragement to continue participation and to complete lessons and homework assignments, enquiries about progress, discussions about putting together an appropriate graded exposure hierarchy, difficulties participants encountered while engaging in exposure tasks, discussing participants’ understanding related to cognitive restructuring and cognitive errors, as well as matters related to future follow up. Telephone contact with patients was offered and utilized on six occasions.

Clinician

One psychiatric registrar (LA) provided all clinical contact with patients. LA received supervision from a clinical psychologist experienced in treating social phobia via Internet and face-to-face modalities. A research assistant provided administrative support to collate data, follow progress and assist in the telephone interviews. The duration of each contact with participants was recorded.

Statistical analyses

Changes in outcome measures were evaluated using paired samples t-tests. Analyses were conducted using both completer and intention-to-treat designs. Data concerned with treatment satisfaction were not formally analysed. Within-group effect sizes were calculated as Cohen's d, based on the pooled standard deviation.

RESULTS

Attrition

Fourteen of the 17 (82%) patients completed all six lessons within the 8 week program. No patients formally withdrew from the program during treatment; however, one patient completed lesson 1 only, and two patients stopped after completing lesson 3. One of these patients remained in contact with the clinician throughout the program. Post-treatment data were obtained from 11/17 (65%) patients. For the intention-to-treat analyses, the pre-treatment scores from the six patients who did not complete the post-treatment questionnaires were carried forward.

Results of outcome measures

Means and standard deviations for pre-treatment and post-treatment, and effect sizes (completer and intention-to-treat) are shown inTable 1. The results of paired-sample t-tests of these data are as follows.

Table 1:

Results of primary and secondary outcome measures: means, standard deviations, and effect sizes (Cohen's d) (completer and intention to treat; ITT)

	Pre-treatment (n = 17)		Post-treatment Completers (n = 17)		Post-treatment ITT (n = 17)		Effect sizes
Outcome measure	M	SD	M	SD	M	SD	Completers	ITT
Social Interaction Anxiety Scale	60.24	9.27	42.36	13.99	47.47	14.43	1.51	1.05
Social Phobia Scale	35.47	17.86	25.00	16.85	27.18	16.35	0.60	0.48
Patient Health Questionnaire − 9 Item	10.82	5.98	5.27	3.85	7.47	5.12	1.10	0.60
Kessler 10	24.88	8.02	18.45	4.28	20.76	5.70	1.00	0.59
Sheehan Disability Scale	12.71	7.02	6.91	5.24	8.47	5.58	0.94	0.67

Completers

Paired-sample t-tests for completers revealed statistically significant improvements across treatment for the SIAS, SPS, PHQ-9, K-10, and SDS (t ₁₀ = 2.38–6.92, p = 0.00–0.04). Large effect sizes were found for the SIAS (1.51), PHQ-9 (1.10), K-10 (1.00), and SDS (0.94) and a moderate effect size was found on the SPS (0.60).

Intention-to-treat

Paired-sample t-tests for completers also revealed statistically significant improvements across treatment for the SIAS, SPS, PHQ-9, K-10, and SDS (t ₁₆ = 2.21–4.29, p = 0.00–0.04). A large effect size was found for the SIAS (1.05), moderate effect sizes on the PHQ-9 (0.60), K-10 (0.59), and SDS (0.67) and a small effect size was found on the SPS (0.48).

Treatment satisfaction

Patients indicated adequate satisfaction with the overall treatment, with 11/11 (100%) of responding patients reporting being either very satisfied or mostly satisfied. All responding patients rated the quality of the treatment modules as excellent or good; 10/11 (91%) rated the quality of Internet correspondence with the clinician as excellent or good, and 1/11 (9%) rated it as satisfactory.

When asked to provide a rating from 1 to 10, where 10 indicates a high level of agreement, on average patients rated the treatment as very logical (8/10); they reported feeling very confident that the treatment would be successful at teaching them techniques for managing their symptoms (9/10); and they reported a high level of con-fidence in recommending this treatment to a friend with social phobia (9/10).

Time/contact events per patient

The average clinician time per patient was 155 min including monitoring of the discussion forum and feedback. An additional average 25 min per patient was required for administrative purposes, including the diagnostic telephone interview. The clinician sent a total of 159 emails to patients (mean = 9, range = 6–16), and made a total of 23 posts in the forum.

Clinician observations

Reaching out to people who stated they would have not otherwise sought treatment was reassuring. The structure of the program being delivered to patients was a considerable support to the clinician and allowed time to concentrate on each patient's individual difficulties. The time dedicated to each individual was largely tailored by the patients themselves and their level of participation, which was related to their readiness for treatment. Three people did not complete the lessons; two lost interest, and one was hospitalized for another condition. This retention rate is satisfactory. Three who did complete did not provide post-treatment progress scores; one refused, as described below, and two were travelling in mid December when the questionnaires were emailed.

The clinician's role was more didactic than in a face-to-face interaction. The email interaction was quite personal. Counter-transference issues occurred and could be quite intense. For instance, one patient had difficulty accepting help and responded irritably to any suggestions made by the clinician. While this patient completed all the lessons, no evaluation of the program was offered. The asynchronous treatment delivery allowed both patient and clinician a reflective space in which to consider what was about to be written in an email. The quality of the interchange between clinician and patients and patients and clinician, therefore, in most cases, improved.

DISCUSSION

The present study examined the clinical effectiveness of an Internet-based CCBT program for 17 people with social phobia. The clinical efficacy of this program has been established in several RCTs. 4 – 7 However, it is important to examine the performance of this program outside the bounds of research. The clinician administering this program had not previously participated in the research programs, but received supervision from a clinical psychologist experienced in face-to-face and Internet-based administration of cognitive behavioural programs for social phobia.

The patients achieved completion rates of 82%, and those who completed the post-program questionnaires achieved within-group ESs on the SIAS and SPS of 1.51 and 0.60, respectively. The comparative outcomes based on the ITT model were 1.05 and 0.48, respectively. The completer effect sizes are similar to outcomes reported from good quality face-to-face treatment for social phobia, 8 and are also similar to those obtained in previous Shyness programs, providing preliminary evidence for the reliability of the treatment procedure in everyday clinical practice.

Levels of patient satisfaction were also high, with 100% of patients reporting being either very or mostly satis-fied with their overall treatment, and 91% rating the quality of Internet correspondence with the clinician as excellent or good. The clinician also found the experience satisfactory and commented on the enormous potential of Internet treatment.

Limitations

These findings require replication with a broader range of clinicians and clinical environments. The gender distribution was similar to that of people attending the St Vincent's Anxiety Disorder Clinic but the patients were some 9 years older. The present group was similar to the other patients who have applied for Internet therapy for social phobia.

CONCLUSIONS

These results provide preliminary evidence for the clinical effectiveness of the Shyness program. The findings must be replicated, but provide further evidence that Internet-based treatment programs for common mental disorders may considerably enhance existing mental health services. Providing they are safe, effective, and acceptable to consumers, clinicians, managers of mental health services, and funders, they may be usefully implemented within the first stage in a stepped care model 18 of service provision.

Footnotes

Acknowledgements

The authors gratefully acknowledge the patients for their involvement and helpful comments. We also wish to gratefully acknowledge the contributions to the development of the ClimateClinic program by Mr Jason Hando, and Mr Daniel Winter.

References

1. Andrews

Henderson

Hall

. Prevalence, comorbidity, disability and service utilisation. British Journal of Psychiatry 2001; 178:145–153.

2. Andersson

Cuijpers

. Pros and cons of online cognitive behavioural therapy. British Journal of Psychiatry 2008; 193:270–271.

3. Titov

. Status of computerized cognitive behavioural therapy for adults. Australian and New Zealand Journal of Psychiatry 2007; 41:95–114.

4. Titov

Andrews

Schwencke

Drobny

Einstein

. “Shyness 1”: a randomized controlled trial of an internet-based treatment for social phobia. Australian and New Zealand Journal of Psychiatry 2008; 42:585–594.

5. Titov

Andrews

Schwencke

. Shyness 2: A replication of Shyness 1. Are the results reliable, and who benefits?Australian and New Zealand Journal of Psychiatry 2008; 42:595–605.

6. Titov

Andrews

Choi

Schwencke

Mahoney

. Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia. Australian and New Zealand Journal of Psychiatry 2008; 42:1030–1040.

7. Titov

Andrews

Choi

Schwencke

Johnston

. An RCT of web-based treatment of social phobia without clinician guidance. (In press). Australian and New Zealand Journal of Psychiatry.

8. McEvoy

. Effectiveness of a cognitive behavioural group therapy for social phobia in a community clinic: A benchmarking study. Behavior Research and Therapy 2007; 45:3030–3040.

9. Kroenke

Spitzer

Williams

JBW

. The PHQ-9. Validity of a brief depression severity measure.Journal of General Internal Medicine 2001; 16:606–613.

10.

10. Mattick

Clarke

. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behavior Research and Therapy 1998; 36:455–470.

11.

11. Kessler

Andrews

Colpe

. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychological Medicine 2002; 32: 959–976.

12.

12. Sheehan

. The Anxiety Disease. New York: Scribner, 1983.

13.

13. Sheehan

Lecrubier

Sheehan

. The Mini-International Neuropsychiatric Interview (M.I.N.I.): The Development and Validation of a Structured Diagnostic Psychiatric Interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry 1998; 59 (S20): 22–33.

14.

14. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, 4th edn, text revision. Washington, DC: American Psychiatric Association, 2000.

15.

15. Borkovec

Nau

. Credibility of analogue therapy rationales. Journal of Behavior Therapy and Experimental Psychiatry 1972; 3:257–260.

16.

16. Devilly

Borkovec

. Psychometric properties of the credibility/expectancy questionnaire. Journal of Behavior Therapy and Experimental Psychiatry 2000; 31:73–86.

17.

17. Andrews

Creamer

Crino

Hunt

Lampe

. The Treatment of Anxiety Disorders: Clinician Guides and Patient Manuals. New York: Cambridge University Press, 2003.

18.

18. Andrews

Titov

. Changing the face of mental health care through needs-based planning. Australian Health Review 2007; 31: S122–S128.