Abstract
Keywords
INTRODUCTION
The last decade has seen a movement towards the practice of evidence-based medicine, a trend that has shaped the delivery of psychiatric care. Market forces that emphasize acute cost containment, governmental policy, and the development of practice guidelines 1 have all contributed to the emphasis on evidence-based practice. Although the influence of insurance and public policy has been less intense in Australia than in the USA, there has nevertheless been a similar movement in the practice of psychiatry in the southern hemisphere. While psychopharmacological management of patients has been more easily adapted to the milieu of evidence-based medicine, psycho-therapeutic treatments have lagged in their use in the community.
On the one hand, the current health-care climate has provided incentives for clinicians to utilize treatments of demonstrable efficacy. An additional positive influence in this movement has been a concern that patients receive the best possible care, which from a scientific perspective is that with empirical support. The time-limited nature of most of the empirically supported therapies and their acute cost-effectiveness have also been primary forces moving psychotherapeutic practices in this direction. 2 On the other hand, the adoption of psychotherapeutic treatments that have been tested in efficacy studies for use in community clinical settings has been minimal. Explanations for this phenomenon can be summarized as falling into one of two categories (see Nathan et al. for a more complete review 3 ). These include concerns about the validity of the data supporting empirically tested psychotherapies, and concerns about their applicability to community settings.
That this impasse has occurred is a reflection of the current academic debate regarding the relative efficacy and effectiveness of psychotherapeutic interventions, and the degree to which these two kinds of evidence should influence clinical practice. 1 Barlow, in his discussion regarding the empirical evaluation of clinical guidelines for psychosocial treatments, has provided an excellent definition of both the efficacy and effectiveness of treatment. 4 Efficacy refers to ‘the results of a systematic evaluation of the intervention in a controlled clinical research context. Considerations relevant to the internal validity of these conclusions are usually highlighted’. In contrast, effectiveness refers to ‘the applicability and feasibility of the intervention in the local setting where the treatment is delivered’, and effectiveness studies are designed to ‘determine the generalizability of an intervention with established efficacy’.
Effectiveness studies are structured to emphasize external validity and generalizability. The term ‘clinical utility’ has also been used interchangeably with effectiveness by some authors. 1 Barlow and Hoagwood et al., among many others, have advocated the use of both efficacy and effectiveness studies as a means of validating treatment. 4 , 5 The American Psychological Association has also adopted this position, and has developed templates for the evaluation of clinical practice guidelines. 1 The template requires that clinical practice guidelines be evaluated on the basis of both the efficacy and the effectiveness of a given intervention. The template describes these two types of studies as falling on two axes, both of which inform the utility and benefit of a given intervention. It also specifies that the guidelines should be disorder based, a notion consistent with the current thrust in psychotherapy research to delineate more precise and effective treatments for specific disorders.
While there is general agreement that the practice of psychotherapy should be influenced to some degree by both efficacy and effectiveness studies, there have been a number of authors who have been critical of psychotherapy efficacy results in general, arguing that the extant data do not yet support the widespread adoption of these psychotherapeutic treatments as ‘preferable’ to current community practices. Although not critical per se of the efficacy studies that have been conducted, it is held that there is simply not enough evidence about the effectiveness of nearly all psychotherapies to support guidelines that suggest, much less mandate, that empirically tested psychotherapies be used in the community. 6– 9 These authors uniformly call for a greatly increased emphasis on research examining the effectiveness of various forms of psychotherapy in the community.
Another critique that has been levelled against psychotherapies that have been evaluated for efficacy is that the interventions that are being tested are not feasible or adaptable to a community setting at all. For instance, efficacy studies are typically conducted in academic centres in which study therapists have reduced caseloads, time for additional supervision, and advanced training in the modality being tested. 10 Further, efficacy research is conducted with subjects who are well selected for their therapeutic suitability, who have a well-defined Diagnostic and Statistical Manual of Mental Disorders (4th edn; DSM-IV) disorder, and who do not have significant comorbid features. In contrast, patients treated in the community may have multiple comorbid disorders and psychological features that may render them less suitable for therapy. Although subjects screened for possible inclusion in efficacy studies can be ‘turned away’ and referred to other treatment providers, patients seeking treatment at community mental health centres all receive treatment. In fact, one might argue that community mental health centres are the ‘treatment of last resort’ for those subjects who do not qualify for efficacy studies.
Two of the most thoroughly empirically tested psychotherapies for depression are good examples of the current state of empirical research. Both cognitive behaviour therapy (CBT) 11 and interpersonal psychotherapy (IPT) 12 have been intensively investigated in efficacy studies. 13– 16 Nearly all critics of psychotherapy research would agree that they are efficacious treatments as tested in well-controlled studies conducted in academic settings. There are, however, no studies that examine either the feasibility or effectiveness of these treatments in a community setting. Accordingly, health professionals from Mental Health ACT in the Australian Capital Territory set out both to improve the quality of treatments provided to patients affected by perinatal depression, and to investigate the feasibility and effectiveness of IPT for this population. A review of current practices revealed that patients were receiving pharmacological treatments and/or supportive counselling. The type and quality of the counselling varied among clinicians, and it was not consistently based upon models with empirically demonstrated efficacy.
The present study was therefore designed to address several concerns. First, we wished to study the effectiveness of an empirically supported psychotherapeutic intervention either on its own or as a combination treatment with medication. Second, we wished to study the feasibility of the intervention in a non-academic setting. This involved several elements. The first was the degree to which a more structured approach would be accepted by clinicians practising in Mental Health ACT. The second was the extent to which training would be required to deliver an effective treatment, and whether this training was feasible and acceptable. The treatment approach we elected to introduce into our service was IPT, a time-limited, empirically validated efficacious therapy for depression. 12 The decision to use IPT was driven by three considerations: (i) the treatment has well-established efficacy for depression in general; 14 , 17 , 18 (ii) the treatment is suitable for delivery by clinicians with variable degrees of training and experience; 19 and (iii) the treatment is suitable for the service's special emphasis on the psychotherapeutic treatment of perinatal disorders. 20 , 21
Although CBT may well be an effective treatment for post-partum depression and depression during pregnancy, evidence of efficacy exists for the use of IPT for post-partum depression 20 and it has been tested in open trials for depression during pregnancy. 22 , 23
METHOD
Therapist training
Forty community mental health professionals attended 20 h of didactic lectures and videotape demonstrations of IPT conducted by an experienced trainer. Although the training had a focus on perinatal depression, it was also tailored to include the treatment of other mood disorders that are commonly treated within Mental Health ACT. Monthly follow-up supervision led by the trainer was provided to 13 clinicians for a period of 10 months. The clinicians were supervised in small groups of 4–6 people at their workplace. Supervision involved audio and video case presentations of sessions, during which the staff were continually encouraged to adhere to the IPT manual. These clinicians were from a range of professions, including nursing (n = 2), psychology (n = 3), social work (n = 4), psychiatry (n = 2) and occupational therapy (n = 1). The typical level of postgraduate counselling experience was between 6 and 9 years. Both the intensive training and supervision were evaluated in terms of its quality and impact on clinical practice.
Treatment implementation
Participating clinicians were asked to select patients with non-psychotic acute depression, who were motivated to participate in treatment and had an absence of severe personality disorders or significant substance abuse. Clinicians were asked to have their patient complete pre- and post-therapy measures of their symptoms using one of the following scales:
the Edinburgh Postnatal Depression Scale (EPDS), a reliable and validated 10-question self-report measure
24
(when using a threshold of 12.5, 80% of subjects are correctly identified as depressed); the Zung Depression Scale (ZDS; a cut-off score of ≥50 indicates clinically significant depression
25
) or the Beck Depression Inventory (BDI),
26
one of the most widely used self-report measures of depression. Cut-off scores of ≥21 have been recommended for clinical research.
27
Various measures were used in the present study due to their utility with different populations (e.g. EPDS for postnatal depression; ZDS and BDI for major depressive disorder) and because of the therapists’ experience with different measures. Patients were assessed at the end of treatment for the need for maintenance IPT 28 , 29 or further intervention.
Twenty-one patients who met criteria for major depression commenced treatment with IPT. However, only 17 patients completed a course of therapy. Of these 17, 13 had post-partum onset of their depression. The patients were routine referrals to the community mental health teams in a 10-month period. The majority of patients (n = 13) were assessed by a psychiatrist using DSM-IV criteria. 30 The remainder were assessed by a health professional trained to conduct thorough psychiatric assessments. The patients were non-indigenous Australians (n = 13), indigenous Australian (n = 1), Maori (n = 1) and European (n = 2). Table 1 provides a summary of other patient characteristics.
Characteristics of treatment completers (n = 17)
RESULTS
Therapist training
A total of 35 of the 40 participants in the IPT programme completed a questionnaire evaluating the training. On a five-point questionnaire, the mean rating of the quality of the training was 4.0 (1 = poor to 5 = excellent). Thirty-two clinicians stated they intended to use IPT in their clinical setting with patients. The most valuable aspects of the training were consistently reported to be observing videotapes of real patients in therapy sessions, role-plays, and the presentation of clinical vignettes by the trainer. The 13 clinicians that went on to participate in a course of supervision were also surveyed about their experiences in the training; 86% of respondents reported that the supervision programme had ‘greatly assisted’ them in their practice. The therapists emphasized the usefulness of videotaping and reviewing patient sessions within the supervision group. The involvement of an experienced IPT therapist was viewed as essential to the program (see Reay et al. for further results of the supervision process 31 ).
Although a high proportion of the workshop participants reported that they intended to deliver the treatment in their clinical setting, less than one-half of them actually attended regular supervision. Over time, several clinicians dropped out of the supervision group prior to completing an IPT case. The clinicians who dropped out were surveyed in an attempt to understand the reasons for discontinuing supervision. Apart from those who had a personal or career event that prevented them from participating (e.g. maternity leave, transfers, sick leave), reported reasons for discontinuation of supervision included: (i) inability to adhere to an IPT approach; (ii) patient selection problems; and (iii) preference for other therapies.
Treatment effectiveness
Twenty-one patients commenced treatment, which involved between 12 and 16 sessions of IPT on its own or in conjunction with antidepressant medication; two postnatal patients elected to end treatment at session four and five, respectively, citing improvement in their symptoms; one patient dropped out after three sessions and another was withdrawn. Therefore, 17 patients completed between 12 and 16 sessions of IPT. The therapists did not collect follow-up data on those patients who commenced but did not complete a course of treatment.
Comparison of pre- and post-treatment scores of treatment completers revealed a significant decrease in mean depression scores. The EPDS scores declined from 17.9 to 7.2 (two-tailed t-test: p < 0.001), BDI scores declined from 21 to 3 (only one subject), while ZDS scores improved from 68.5 to 37 (two-tailed t-test: p < 0.001). Of the patients diagnosed with post-partum depression at the onset of treatment, all but one had scores < 13 on the EPDS, indicating they were unlikely to be depressed. The one patient who was still symptomatic at the end of treatment had received a combination of antidepressants and psychotherapy.
Analysis using two-tailed t-tests revealed no significant differences (p = 0.55) between the mean age of patients who received a combination of anti-depressant therapy and IPT (mean = 32.9 years) compared with those patients who received IPT alone (mean = 29.6 years). In summary, the changes for all three measures were significantly improved at termination and all but one patient who completed a course of 12–16 sessions of IPT were below the cut-off scores used to screen for depression for each scale. Table 2 summarizes these results.
IPT treatment effectiveness: results
Follow-up results
Three postnatal women were offered monthly maintenance sessions 28 , 29 with IPT due to their history of frequent recurrent episodes of depression (three or more episodes); two of the three received maintenance IPT while the other declined the offer, although she agreed to recontact the therapist in the event of early warning signs of a relapse. The remaining patients (n = 14) were seen by their respective therapists on one occasion at approximately 6 weeks after treatment for a review of their symptoms. Thirteen patients were discharged back to their general practitioner and required no further follow up. One patient experienced a relapse of her symptoms within 3 months after treatment after ceasing her anti-depressant medication. Antidepressant medication was restarted and a brief six-session course of IPT was provided, with the patient no longer meeting criteria for major depression at the end of this intervention.
DISCUSSION
Therapist participation and involvement
The present study relied on a high degree of involvement and cooperation by mental health clinicians from the local service. These therapists often had high caseloads, work pressures and time limitations. We found significant variance in the level of enthusiasm, involvement in supervision and cooperation with data collection. These potential problems should be addressed and anticipated early in supervision to avoid problems with staff non-participation and missing data.
Despite these difficulties, one-third of the trained mental health professionals were able to complete a course of supervision with at least one client. These therapists tended to have a more positive attitude towards clinical supervision, and were willing to review their performance as therapists, adhere to the IPT model and collect outcome data. In addition, although most staff reported benefiting from training in a structured evidence-based psychotherapy, many preferred to incorporate the techniques and strategies used in IPT into their current approach without having to adhere to a manualized therapy.
There is at present no evidence that adherence to a specific manualized form of psychotherapy is more effective than an eclectic approach in general clinical settings, although there is evidence that adherence to the IPT manual is more efficacious in research settings. 28 In addition, it is not clear whether specific training in an empirically based manualized psychotherapy has any impact on therapists who receive training but subsequently incorporate only a few elements into their therapeutic work, as opposed to conducting the therapy as specified in the manual. Potentially, such manual-based training could impact upon subsequent therapy techniques and/or outcomes.
Treatment effectiveness
Effectiveness studies have a number of inherent limitations, which include the lack of a control group, small sample sizes, and limited follow-up data. In addition, in the present study a number of patients (76%) received both medication and IPT, which places obvious limitations on the conclusions that can be drawn about the effectiveness of the psychotherapy on its own. The practice of using both psychotherapy and medications is commonplace clinically, although there is only limited evidence supporting the efficacy of combined treatment. Other limitations of the present study include the lack of homogenous subjects and variation in outcome measures used by the treating clinicians. While efficacy studies are conducted on homogenous groups of patients for reasons of replicability and reliability, effectiveness studies usually include patients who require treatment regardless of specific diagnoses, comorbidity or length of illness. 3 The present study applied an efficacious treatment with a cohort of patients who are usually seen in community settings.
The absence of follow-up data on participants is another limitation of the present study because the evidence has shown that a proportion of patients relapse (approx. 30%) within 1 year of treatment. 14 Apart from those patients who were identified as being at high risk of relapse and who were offered maintenance IPT, the remaining patients were not followed up at regular intervals. This is because community patients are usually referred back to primary care on discharge and they are not usually routinely followed up. Our belief was that the examination of effectiveness and feasibility would be maximized if therapists followed current community-based practices.
CONCLUSIONS
The hypothesis that this intervention could be feasibly delivered in a non-academic setting was supported by both therapists' reports and outcome measures. The teaching and supervision programme, tailored to the requirements of the clinical setting, was viewed as feasible and acceptable to both staff and management, resulting in the development of an ongoing training programme. Clinicians who completed the course found that they were able to confidently apply IPT in a clinical setting. The study also encountered some obstacles to the uptake of the treatment by clinicians, as evidenced by patient selection problems, difficulties adhering to the IPT approach, and preference for other models by some therapists. Due to the small numbers of clients who received either IPT alone or IPT in combination with medication, further study is required to determine the feasibility of IPT in other non-academic settings using larger sample sizes. Despite these limitations, the present pilot study has provided important information on the preparation and training of mental health professionals in assisting them to adopt an evidence-based treatment.
Footnotes
Acknowledgements
We wish to thank Dr Michael Robertson who contributed his expertise as our trainer and supervisor. Thanks also to Dr Kathy Rohan, Dr Emma Adams, Yvonne Fisher, Michael Woodward, Sue Lawless, Brigid Byrne, Sharon Kerr, Louise Camm and all the clinicians who participated in the study. Last, our appreciation to Mental Health ACT who funded and supported the project.