Developing New Services for Eating Disorders: An Evaluation Study

Abstract

Objective: This paper aims to evaluate the structures and processes of eating disorders services in two regional cities in Australia.

Method: Stakeholder evaluation undertaken between 2002 and 2005 uses interviews, questionnaires and service delivery data to examine: structure and patient profile of the two services, barriers and success factors and local factors influencing development of the services.

Results: The Bendigo service provided secondary consultation and specialist management with upskilling of primary care workers as a key goal. Patients were referred to the service via mental health triage. The Geelong service initially offered assessment only, with direct access for the general public. Treatment was offered from early 2004. The Bendigo service assessed 41 patients, 63% were diagnosed with anorexia nervosa or bulimia nervosa. Most patients had a moderate or severe eating disorder. The Geelong service assessed 186 patients, 55% were diagnosed with anorexia nervosa or bulimia nervosa and 80% of this subset had not previously been treated for an eating disorder. General practitioners identified barriers to development of the services as: problems with capacity building and unrealistic expectations. The success factors were providing a locally based service with credible clinicians and effective communication.

Conclusions: Activity analysis demonstrates that the goals for both services were met. There is a need to measure both short- and long-term patient outcomes in order to fully assess effectiveness and applicability to other settings. Local factors, such as availability of specialist providers, would need to be taken into account.

Keywords

eating disorders evaluation mental health rural

BACKGROUND

Rates of eating disorders in the Australian population have been estimated at 0.5% for anorexia nervosa (AN) and 0.5–1% for bulimia nervosa (BN). Eating disorders are associated with considerable morbidity 1 with a crude mortality rate for AN across 119 outcome studies reported at 5.0%. 2 In Australia and the UK, treatment guidelines recommend a multidisciplinary approach. This includes comprehensive assessment, the consideration of multiple treatment modalities and multiple interventions. 3 , 4 However, it is estimated that only 30% of patients who present with AN and 6% with BN are seen by mental health specialists. 5

Problems of access to specialist multidisciplinary care are exacerbated in regional and rural communities where there is generally less direct access to mental health specialist services. 6 ^– 8 Specialist eating disorders services (EDS) are limited, usually metropolitan-based and generally linked to inpatient facilities. 9 Further, regional and rural services often have no primary health supporting structures in the form of support groups, consumer advocacy groups or allied health support. 10

In 2002, the Victorian Department of Human Services provided pilot funding to establish and evaluate eating disorders programmes in Bendigo through the Area Mental Health Service, and in Geelong through the Association of General Practice. The service models were tailored to the local environment, needs and constraints, and informed by national clinical practice guidelines. 3 Needs analysis undertaken in Geelong and Bendigo prior to funding submissions emphasized the lack of established pathways of care and inadequate provision for people with eating disorders.

THE STUDY

Aims

The aims of the study were: to evaluate the effectiveness of specific structures, processes and activities of eating disorders programmes in Bendigo and Geelong and to make recommendations as to their general applicability across the primary health and mental health sectors.

Context of the two services

The two EDS were deliberately different, reflecting the findings of needs analysis undertaken in Bendigo and Geelong to inform the funding submissions. Differences between the two services are presented at Table 1. The Bendigo service was established in the context of a 4-level stepped collaborative care model. 11 This approach allowed the service to be part of a broader programme which included the Primary Mental Health and Early Intervention Team. 12 The service model emphasized the provision of secondary consultation and limited shared care arrangements (level 3) and ongoing specialist treatment for severely unwell patients (level 4). Levels 1 and 2 (screening, diagnosis, patient education, treatment in primary care, monitoring outcomes) were addressed through upskilling of general practitioners (GPs) and other primary care workers. Staffing included a clinical psychologist, consultant psychiatrist and dietitian. The Geelong service was established in a primary care setting (Association of General Practice) and was initially developed as the first point of contact, including direct access for the general public. Staffing included a clinical psychologist, dietitian and family therapist. Emphasis was placed on integration with other services and interagency liaison. From the beginning of 2004, the service expanded to include client management.

Table 1:

Differences between the Geelong and Bendigo services

	Geelong Service	Bendigo Service
Auspice of the services	Association of General Practice	Mental Health Service
Geographical coverage	Geelong and Surf Coast/Apollo Bay Coverage expanded to include Colac and Portland	Loddon Southern Mallee region (Manangatang to Gisborne)
Primary goal	To provide a central assessment point with direct access for clients	To provide upskilling and advice to enable GPs to continue to manage their patients
Service activities	Primary consultation self referral GP referral	Cascade training of primary healthcare workers and support for their delivery of care Secondary consultation and limited shared care arrangements Direct care for severe/complex cases
Management	Not initially part of the service but developed later Advice to patients' re-management, with exceptions in severe cases During 2004 client management became a more central focus of the service	Formulation of management plan Medication review and advice Review of suitability for psychological therapy Time-limited psychological therapy for severe or complex cases
Access for patients	Direct	Via Area Mental Health Service triage Referrals accepted from: GPs AMHS case managers community health workers
Client impact	Direct	Indirect/direct
Service/system impact	Indirect	Direct

GP, general practitioner.

METHODs

The research employed stakeholder evaluation methods 13 and took a formative approach with regular feedback to the clinicians delivering the services to enable stakeholder perspectives to be addressed as the services developed. Stakeholders included consumer representatives, clinicians, reference group members and GPs. Data were collected between 2002 and 2005. Qualitative content analysis was used to identify themes in interview data.

Data collection

Data were collected in five stages:

individual semistructured interviews with clinicians delivering the service (n = 5) (December 2002 and various stages throughout the study);

telephone interviews with GPs (n = 8) (November 2003–January 2004);

semistructured interviews with consumers and reference group members (n = 10) (mid-2004);

postal survey of stakeholders (n = 31) (November 2004);

final semistructured interviews with clinicians (n = 4) (March 2005).

Each service had a multi-professional Reference Group consisting of the programme team, key stakeholders and consumer representatives. These groups were used as part of the evaluation to help clarify issues and concerns. Two cross-service meetings were held to enable the two teams to share experiences.

Survey tool

The survey questionnaire (stage 4) was devised using outcome statements related to the aims of each service. Respondents were asked to rate their agreement with each outcome statement using a 5-point Likert scale from ‘strongly agree’ to ‘strongly disagree’, with the additional option of a ‘don't know’ response. Space for free-text comments was also provided beneath each statement. Different questionnaires were used for each service, to allow the specific service aims to be evaluated. Respondents were also invited to rate the importance of 10 success factors for the initiation, development and sustainability of the service, using a 5-point scale (from ‘very important’ to ‘not at all important’). These factors were identified in the stage 2 GP interviews as essential to the development of the service. Development of the survey tool was guided by a combination of earlier stakeholder data and issues raised at Reference Group meetings. Following pilot work, minor amendments were made to the wording of some statements. Stakeholders were identified via the two services and comprised clinicians with whom the services had contact (making or receiving referrals) plus members of the reference groups. The largest single group of respondents was GPs (16/31). Other respondents were: dietitians, psychologists, psychiatrists and specialist medical paediatric services.

FINDINGS

Geelong service activity

One hundred and eighty-six patients were assessed by the Geelong Disordered Eating Service (DES) between December 2002 and March 2005; 136 (74%) were self or family referrals. One hundred and one (55%) were diagnosed with AN or BN. Of these 101 patients, 19 (20%) had been previously treated for an eating disorder.

Analysis of the raw data revealed the following subgroups:

Of those with AN or BN (n = 101), patients reported a history of symptoms of between 6 weeks and 20 years: a total of eight patients had 20–30 year symptomology.

Patients with binge eating and/or compulsive over-eating (n = 25) were predominantly self referrals (n = 20).

Of the patients self-referring (n = 90), 47 (52%) were diagnosed with AN or BN. Of these 47, 34 (72%) had not been previously treated for an eating disorder.

These data highlight the potential unmet need being identified through the approach taken at Geelong.

The focus of the service shifted from assessment and referral to the management of patients with eating disorders, with a total of 376 treatments provided for patients during 2004. Those patients identified by the service as benefiting from management included:

acutely unwell patients requiring close monitoring, possible hospitalization and co-ordination of a number of therapists involved in their care;

unwell and ‘difficult to engage’ patients requiring close monitoring;

patients in whom there exists significant eating pathology of indeterminate aetiology. Intervention with this group is seen by DES as of great importance to prevent progression to significant eating disorders.

Ninety-five multidisciplinary case conferences were held and six education workshops were undertaken with school nurses, GPs and the public. The GP workshops initially had some success but barriers to GPs accessing these types of workshops included issues such as too few patients with eating disorders to warrant skill acquisition, or to maintain skill level.

The DES responded to 500 calls regarding individual patient management, and a further 500 calls were taken related to other aspects of disordered eating.

Bendigo service activity

A total of 55 referrals were received by the Bendigo Service between November 2002 and March 2005; of these, 41 were assessed. Diagnoses were: AN 41% (n = 17), BN 22% (n = 9), eating disorder not otherwise specified 22% (n = 9) and no eating disorder 15% (n = 6). Most patients assessed had moderate or severe eating disorders. The Eating Disorders Inventory Subscales 14 demonstrated that the severity of psychological disturbance for those treated by the EDS is comparable to an inpatient sample of AN patients (Table 2).

Table 2:

Symptom measures of patients offered treatment at Bendigo EDS, with measures for AN inpatients and non-eating disordered females for comparison

Symptom measure	Mean (SD) of patients offered treatment by EDS (n = 19)∗	Mean (SD) of inpatient AN sample 15 (n = 113)	Mean (SD) of non-eating disordered female sample 15 (n = 577)
Eating Disorders Inventory Subscales
Drive for thinness	15.0 (6.0)	15.0 (±0.5)	5.0 (±0.2)
Body dissatisfaction	14.8 (8.7)	14.0 (±0.1)	6.0 (±0.1)
Perfectionism	9.2 (4.9)	10.0 (±0.5)	7.0 (±0.9)

∗

Twenty-two of those assessed and 19 of those offered treatment at the EDS completed questionnaire measures.

± refers to standard error.

AN, anorexia nervosa; EDS, eating disorders service; SD, standard deviation.

Some patients assessed and treated had significant comorbid features (or a diagnosis) of another mental illness. For example, 24% (n = 10) had a diagnosis of mood disorder, and a further 34% (14) had features of mood disorder, while 5% (2) had a diagnosis of anxiety disorder and a further 22% (n = 9) had features of anxiety disorder. Ten per cent (n = 4) had a diagnosis of a personality disorder, while a further 22% (n = 9) had significant personality problems.

Direct treatment for those with complex and severe disorders was a key component of the service. Typically, patients were seen weekly for therapy; the number of sessions ranged between 4 and 43 per patient. A total of 245 secondary consultations were provided predominantly not only to clinicians (n = 200, 82%), but also to parents (n = 31, 13%), other relatives/ friends (n = 11, 4%) and potential patients (n = 3, 1%). Education sessions were provided to a small number of GPs and to a number of other clinicians, community groups and carers. The planned education and training activities of the EDS were amended to fit in with other training activities provided for local GPs following launch of the Commonwealth Better Outcomes in Mental Health Care (BOiMHC) GP funding initiative. 16 The 80 GPs in the region who have participated in level 1 BOiMHC training are now being offered eating disorders specific training which builds on this knowledge and skills. However, GPs who responded to the stage 4 survey cited the requirement to first attend level 1 BOiMHC training as a barrier to accessing eating disorders education.

The carer education programme was evaluated as a separate sub-study. Findings emphasize the importance of all aspects of the programme to the carers, but the content is best suited to those who are new to the issues of caring for a person with an eating disorder. 17

PROCESS EVALUATION

The processes involved in establishing the two services were evaluated with regard to factors limiting and factors facilitating the service (barriers and success factors, respectively). The role of consumers was also explored.

Barriers to service development

Key barriers to development of the service identified through the stage 2 GP interviews across both services (n = 8) were:

Capacity building. This posed a difficulty for two reasons: first, the movement of GPs and second, small numbers of patients accessing the service via an individual GP. Both of these were out of the control of the eating disorders services; however, they will limit the ability of the services to build capacity. This suggests the need for ongoing input from experts, both for GP education and for advice with patient management.

Local ownership. Both services made substantial investment to achieve local ownership of the service. However, this remained difficult in two areas: interface with the local hospital and a lack of specialists to refer patients to.

Differing expectations of the service. General practitioner interviews indicated that GPs expected the service to be a complete package. This is at odds with the partnership approach adopted by both services. The extent to which this was related to time factors/workload pressures was not clear.

These barriers were also identified in individual interviews with clinicians delivering the services.

Success factors

Ten factors key to the success of the eating disorders service were identified in initial interviews with GPs across both services (total n = 8); the three most important were:

delivery of the service by credible clinicians;

locally based service focused on primary care;

timely two-way communication.

SURVEY DATA

All 10 success factors were subsequently explored with the wider group of stakeholders across both services (n = 31). Respondents were also invited to rate the importance of each factor for the initiation, development and sustainability of the service. Despite the different orientation of the two services, responses from all stakeholders were similar across the two groups. As is often the case with positive statements, there was a large degree of agreement with all statements, at all three stages of the service. However, review of individual questionnaires identified that respondents shifted between ‘very important’, ‘important’ and ‘neither important nor unimportant’, with no evidence of a response set bias. Five factors were rated ‘important’ or ‘very important’ by all respondents for sustainability of the service: timely feedback, credibility of clinicians leading the service, involvement of local clinicians, locally based service and accessibility of contact with the service.

All evaluation data were anonymized and fed back to the individual service as part of the evaluation process.

CONSUMER PARTICIPATION

Both services had difficulty recruiting consumer representatives to their Reference Group. The role of consumer participants was explored as a separate substudy by a member of the research team (SK). Eleven participants were interviewed: two consumer advocates and nine clinicians. Lack of definition for the term ‘consumer’ (sufferer, carer, primary care worker) and confusion about the role were felt to limit the impact on development of the services. Four specific barriers to consumer participation were identified:

difficulties in providing a representative voice;

lack of funding;

absence of processes to integrate the consumer role;

the lack of power of the consumer relative to other stakeholders.

There was some scepticism about the ability of consumer-sufferers to contribute in a meaningful manner. Consumer involvement was seen as ‘a condition of the funding, rather than a role instigated by the clinicians’ and consumer advocates highlighted factors influencing participation such as timing and cancellation of meetings. There was also a tendency for clinicians to blame failure or problems with consumer participation on the consumer.

DISCUSSION

The two services described here were different in a variety of ways. First, they were embedded in very different contexts and so had different aims. In Geelong, where there is a specialist provider infrastructure, the service focused on assessment and appropriate referral to others available and able to provide appropriate treatment. The service also aimed to extend the reach to include more early intervention. By contrast, in Bendigo, where there is a paucity of specialist providers outside the public mental health service, the aim was to upskill primary care providers to enable them to assess and treat those with milder or uncomplicated presentations and to provide direct care for those with more severe and complex problems. Thus, the second key difference, the patient population seen by the two services is predictable. Those seen in Geelong tended to have a broader range of disorders, while those seen in Bendigo generally had a severe eating disorder.

Of interest, both services changed over the period of the evaluation, responding to changing demands and circumstances. Any eating disorders service will need to include the ability for some outreach, both for clinical service provision and to support others providing care. The move to provision of treatment as well as assessment in Geelong was seen as a necessary development and should be considered in the development of future services. However, the need for increased psychiatry sessions to manage this shift in focus was noted by the clinicians delivering the Geelong service.

CONCLUSIONS

The goals for both services were met to a large extent. The services faced a range of challenges, many of which were specific to the service model, target client group and local context. The stepped care model was selected by Bendigo because of the lack of specialist providers in the Loddon Mallee region; an area with more specialist providers would not need to rely to the same extent on upskilling primary care workers. The Geelong (direct access) model, where patients receive an accurate diagnosis and either referral to appropriate services or treatment/management ‘inhouse’, is cost-effective and streamlines the management of eating disorders in the community. However, there is a need to measure both short- and long-term patient outcomes in order to fully assess effectiveness and applicability of the models to other settings.

Study findings emphasize that, in spite of government policy which positions consumer participation as an essential part of mental health service, 18 ^– 20 the establishment of new mental health services are occurring with poorly designed consumer participation strategies. A number of recommendations are made for the development of future eating disorders services; these are summarized in Table 3.

Table 3:

Recommendations for the development of future eating disorders services (EDS)

1.	EDS need to be locally based.
2.	Communication pathways need to be established from the outset of the service, including feedback to those delivering the service.
3.	EDS should be delivered by a team of clinicians including clinical psychologist, psychiatrist, dietitian and family therapist.
4.	New services should be developed in consultation with both primary care and mental health services.
5.	The service model for eating disorders should be appropriate for the local context, geographical coverage and perceived needs.
6.	Clear guidelines should be developed for consumer participation at the initiation and development stages of a service.
7.	It is not realistic to expect an EDS to only provide assessment and care planning. This causes frustration both for those referring into the service and the service providers.

References

1. Patton

G C

Selzer

Coffey

Carlin

J B

Wolfe

. Onset of adolescent eating disorders: population-based cohort study over 3 years. British Medical Journal 1999; 318: 765–768.

2. Steinhausen

H-C

. The outcome of anorexia nervosa in the 20th Century. American Journal of Psychiatry 2002; 159: 1284–1293.

3. Beumont

Hay

Beumont

. Summary: Australian and New Zealand clinical practice guidelines for the management of anorexia nervosa. Australasian Psychiatry 2003; 11: 129–133.

4. National Institute for Clinical Excellence . Clinical Guideline 9: eating disorders. Core interventions in the treatment and management of anorexia nervosa, bulimia nervosa and related eating disorders. NICE. 2004, http://www.nice.org.uk/CG009NICEguideline. Accessed 26 August 2005.

5. Hoek

van Hoeken

Katzman

M A

. Epidemiology and cultural aspects of eating disorders: a review. Eating Disorders, Maj

Halmi

Lopez-Ibor

Sartorius

. John Wiley, Chichester 2003; 75–104.

6. Judd

F K

Fraser

Grigg

. Rural psychiatry: special issues and models of service delivery. Disease Management and Health Outcomes 2002; 10: 771–781.

7. Dossetor

D R

Nunn

K P

Fairley

Eggleton

. A child and adolescent psychiatric outreach service for rural New South Wales: a telemedicine pilot study. Journal of Paediatrics and Child Health 1999; 35: 525–529.

8. Clayer

Bookless

T A

McFarlane

. Psychiatric disorder and disability in a rural community. Social Psychiatry and Psychiatric Epidemiology 1998; 33: 269–273.

9. Gaskill

Saunders

. The Encultured Body: Policy Implications for Healthy Body Image and Disordered Eating Behaviours. Queensland University of Technology, Kelvin Grove, Old 2000.

10.

10. Cornett

. Report of Eating Disorders 2000 Project. GP Association of Geelong, Barwon Health and The Geelong Clinic, Geelong, Vic. 2000.

11.

11. von

Korff M

Glasgow

R E

Sharpe

. Organising care for chronic illness. British Medical Journal 2002; 325: 92–94.

12.

12. Judd

Davis

Hogdkins

Scopelitti

Agin

Hulbert

. Rural Integrated Primary Care Psychiatry Programme: a system's approach to education, training and service integration. Australasian Psychiatry 2004; 12: 42–47.

13.

13. Guba

E G

Lincoln

Y S

. Fourth Generation Evaluation. Sage, Newbury ParkCA 1989.

14.

14. Garner

D M

. Manual of Eating Disorders Inventory-2. Psychological Assessment Resources, Odessa, FL 1991, Psychological Assessment Reviews.

15.

15. Menzel

M D

. Group cognitive behavioural therapy for eating related emotions and behaviours in anorexia nervosa. University of Melbourne, Melbourne 1992, Unpublished doctoral dissertation.

16.

16. Australian Divisions of General Practice . Familiarisation training: GP and Practice Manual. Better Outcomes in Mental Health Care Initiative, 2nd edn. Commonwealth Department of Health and Ageing, Canberra 2003, Australian Divisions of General Practice: http://www.adgp.com.au/client_images/7061.pdf. Accessed August 2005.

17.

17. Fraser

. An evaluation of an education program for carers of people with eating disorders. University of Ballarat, Vic. 2005, Unpublished thesis submitted for Postgraduate Diploma of Psychology.

18.

18. Australian Health Ministers . National Mental Health Plan. Australian Government Publishing Service, Canberra 1992.

19.

19. Australian Health Ministers . Second National Health Plan. Mental Health Branch, Commonwealth Department of Health and Family Services, Canberra 1998.

20.

20. Australian Health Ministers . National Mental Health Plan 2003–2008. Australian Government, Canberra 2003.