Abstract
Routine outcomes measurement in mental health services can be traced to the National Mental Health Policy, one of the objectives of which was ‘To institute regular reviews of outcomes of services provided to persons with serious mental health problems and mental disorders as a central component of mental health service delivery’. 1 Since that time, much effort has been put into implementation into public and private mental health services.
While the initial implementation has been confined to the ‘mainstream’ forms of service delivery, it has remained unclear whether certain types of services (e.g. forensic and consultation-liaison (C-L) mental health) should be in or out of ‘scope’ as far as routine data collection is concerned. 5 The Second National Mental Health Plan acknowledged that, in certain quarters, severity has been equated with diagnosis rather than level of need or disability, and that ‘there should be no financial disincentives to C-L psychiatry services participating fully in the mental health care system’. 6 ,7 A meeting of the Consultation-Liaison Psychiatry Section of the Royal Australian and New Zealand College of Psychiatrists was held in November 2002 and produced a consensus statement that C-L psychiatry was an appropriate setting for the collection of outcomes and casemix data.
The potential benefits of standardized data in C-L psychiatry have been appreciated for many years. McKegney and Beckhardt recommended that the shift from descriptive studies to more outcome-orientated research, particularly regarding patient outcomes, should be the direction of the future. 8 Popkin et al. described a system designed, among other things, to establish standards with which to evaluate the effectiveness of psychiatric consultation, and signalled the need for further outcome studies and the development of data-based consultation practices. 9 Shortly after-wards, Mitchell and Thompson concluded that C-L psychiatry needs a solid research base to legitimatize its contribution to cost-efficient patient care. 10 Others, for example von Korff et al., have raised the question of what outcomes should be measured. 11 More recently, Bower and Gask have argued that newer approaches to C-L emphasize the importance of the development of effective systems of delivering care, with attention given to the practices of clinicians and the outcomes of patients. 12
A review of C-L services in Victorian hospitals iden-tified severity and case complexity as a key consideration for the staffing and funding of C-L services. 13 On the basis of data available at the time, it was concluded that it was ‘…not possible to draw any definitive conclusions about the relative severity of patients receiving a C-L service’ (p. 35). The report concluded that ‘The paucity of reliable information on C-L services will undermine arguments that indicate that the efficacy and cost efficiency of C-L services justify the expenditure and priority. A data-base developed over the coming years will greatly assist this effort. The development of a data base over an extended period is the most effective method of determining future directions for service provision’ (page 68). Most recently, Smith has reinforced the need for an outcomes approach, in the context of a need to reorganize services. 7
The present study looked at the information provided by a widely used outcome measure that was used routinely by clinicians in a C-L service over a period of nearly 3 years. Assessments of patients on the same instrument in the acute inpatient psychiatric unit of the same hospital were used for comparison purposes.
METHOD
In mid-2000, St Vincent's Mental Health Service, Melbourne, along with three other Victorian area mental health services, began routine outcome measurement. Following the training of trainers, 2 clinical staff were trained, and outcome measurement began in all parts of the adult service, including the C-L psychiatry service. Outcome measures were entered into a purpose-designed database; 14 this utility has the capacity to record certain other patient details, including date of birth, sex, and principal and secondary psychiatric diagnoses.
The present report focuses on a single measure, the Health of the Nation Outcome Scales (HoNOS), which was the only measure used routinely in both the acute psychiatric inpatient unit and the C-L service. 15 Assessments in the C-L service were conducted either in the emergency care centre (ECC) or the general hospital (GH), while those in the psychiatric unit were distinguished according to whether they were obtained at admission or discharge.
RESULTS
In the period July 2000 to March 2003, more than 1800 administrations of the HoNOS were recorded in ECC, GH or on admission to the psychiatric inpatient unit (ADM), and of these, 1582 contained no missing items. The numbers of administrations of complete HoNOS, and the mean total scores are shown in Table 1.
The mean HoNOS total score obtained in ECC is significantly higher than that obtained in the GH (t(1493) = 3.66, p < 0.001). The mean score at ADM is also significantly higher than that of the GH (t(1150) = 3.29, p = 0.001) but it is not significantly different from that of the ECC (t(515) = 1.32, NS). The mean scores in all three of these setting were very significantly higher (p < 0.0001) than that obtained at discharge from the acute inpatient unit. For this reason, we shall henceforth dispense with assessments at discharge from the acute inpatient unit.
Local activity statistics maintained by the C-L service indicated that a large proportion of patients seen in the emergency department were transferred to a psychiatric inpatient unit. This was usually in the patient's area of origin, which in most cases was not this hospital. Using this information source, it was possible to identify 79 instances of a HoNOS conducted in the emergency department in which the recorded separation was to a psychiatric inpatient unit. The mean total score of these was 22.1 (SD = 6.0), which is significantly higher than the 18.0 noted for all acute psychiatric admissions (t(164) = 3.92, p = 0.0001).
Mean HoNOS total scores, by setting
As a guide to overall severity, the mean of all (>4500) complete HoNOS assessments conducted at review(i.e. not the start or close of an episode of care) in the community in the period July 2000 to January 2002 in the four agencies that began routine outcome measurement in 2000, was 9.9 (SD = 6.0) (Trauer T: unpubl. data, 2003). Thus, the mean HoNOS total scores in the three settings of present interest are approximately 1 SD above those obtained at case review in the community, and for those transferring from emergency department to acute inpatient unit, approximately 2 SD. We may extend this analysis to examine ratings on the 12 individual HoNOS items. This is shown in Table 2. The differences between the three settings are significant for all items apart from items 7 (depressed mood) and 8 (other psychological problems). The ECC means are higher than those of GH for items 1 (aggressive/ overactive/disruptive/agitated behaviour), 2 (deliberate self-harm), 3 (alcohol/drug problem), 6 (hallucinations/ delusions), 9 (relationship problems), 11 (accommodation problems) and 12 (occupation/leisure problems), while the mean scores on items 4 (cognitive problems) and 5 (physical problems) were significantly higher in GH than ECC. There were few differences between ECC and ADM: the latter were higher on item 6 (hallucinations/ delusions) and 10 (activities of daily living). The ADM means were higher than the GH for items 1, 2, 3, 6, 9, 10 and 12, while the reverse was true for items 4 and 5.
The differences reported here need to be understood in terms of other systematic differences between the patients involved. It has been shown, for example, that there are differences in HoNOS scores according to age, sex and principal psychiatric diagnosis. 16 We proceed therefore to examine whether there are consistent differences between the three service settings on these three patient characteristics.
Mean HoNOS item scores, by setting
The mean ages of patients in ECC, GH and ADM were 39.3 years, 52.3 years and 34.8 years, respectively. These differences are highly significant (F(2,1575) = 97.7, p < 0.0001). Post-hoc comparisons (Scheffé's tests) showed that the mean age in GH was significantly higher than the other two locations, which were not significantly different from each other. The HoNOS total score was significantly negatively correlated with age in the GH (r = −.15, p < 0.0001) but not in ECC nor ADM (r = −.03 and .03, respectively, both p > 0.4).
The proportions of patients who were male in the ECC, GH and ADM were 44.7%, 47.9% and 47.7%, respectively, these differences being not significant. Mean total HoNOS scores of women were higher than those of men in all three settings, significantly so in ECC (t(425) = 4.19, p < 0.0001) and GH (t(1062) = 3.61, p < 0.001), but not at ADM (t (84) = 0.3, p > 0.7).
Principal psychiatric diagnosis was coded according to the main categories of International Classification of Diseases (10th revision; ICD-10). 17 Allowance was made for the recording of a secondary diagnosis where applicable, using the same system. Across the whole group, psychiatric diagnoses were missing from the database in approximately 8% of cases; of those with diagnoses, 68.4% had a single, principal, diagnosis and 31.6% had a secondary diagnosis as well. Patients were assigned to each of the main diagnostic categories if either the principal or secondary diagnosis had been applied. The numbers of patients with principal or secondary diagnoses in the main categories, and with no missing items in their HoNOS assessments, are shown in Table 3.
No. patients analysed by diagnosis and setting
The numbers in Table 3 sum to greater than the numbers of patients involved, because approximately one-third of patients had both principal and secondary diagnoses. Considering only the six most numerous diagnostic categories, it may be seen that the diagnostic profiles of the three settings are quite different (χ2(10) = 219.6, p < 0.001). Organic disorders were more common in the GH, substance abuse disorders were approximately twice as common in ECC and ADM than GH, psychoses were much more common in ADM than in the C-L settings, mood disorders were fairly evenly distributed, anxiety dis-orders were more common in the GH, and personal-ity disorders were approximately twice as common in ECC as in GH or ADM.
To the extent that different diagnoses attract differ-ent mean HoNOS scores, comparisons of the settings need to take this into account. The mean total HoNOS scores across all three settings of the six most numerous diagnostic categories were: organic, 17.0; substance abuse, 16.6; psychosis, 13.8; mood disorders, 15.4; anxiety/stress disorders, 12.9; and personality disorders, 16.9. These differences are highly statistically significant (F(5, 1635) = 11.98, p < 0.0001).
As a prelude to adjusting for age, sex and diagnostic differences, HoNOS total scores were regressed against age, sex and diagnostic category. The six most numerous diagnostic categories were represented by six indicator (binary) variables that took a value of 1 if either the principal or secondary diagnosis applied, and zero otherwise. This analysis showed that approximately 13% of the variance in HoNOS total scores could be accounted for by the joint effects of age, sex and diagnosis, a result that was highly significant (F(8,1564) = 29.81, p < 0.0001). On the basis of this regression analysis, a predicted HoNOS total score was calculated for each patient with complete data on the predictors. The mean predicted scores of patients in ECC, the GH and ADM were 16.9, 15.6 and 17.3, respectively, these differences being highly significant (F(2,1570) = 58.90, p < 0.0001). Comparison with the unadjusted means presented in Table 1 shows that the mean scores in ECC and the GH have hardly changed, but the adjustment produced a small lowering of the ADM scores. Post-hoc comparisons show that the differences between ECC and the GH, and between the GH and ADM, remained highly significant, while the difference between ECC and ADM remained non-significant. Similar adjustments were performed for each of the HoNOS item scores. The results were very similar to those of the unadjusted scores.
Across the three settings studied and where the profession of the HoNOS rater was known, between 93% and 97% of assessments were conducted by medical staff, with all but four of the remaining assessments being conducted by nursing staff. Of the assessments completed by medical staff, 94% were conducted by registrars, compared to 6% by psychiatrists.
DISCUSSION
The present study has found that overall severity levels, as measured by the HoNOS, of patients seen by a C-L service in the hospital emergency department were comparable with those at admission to the psychiatric unit. Some of the similarity is due to overlap of clientele, but those cases identified as moving from the emergency department to an acute
psychiatric inpatient unit had even higher severities than the average for the local acute psychiatric inpatient unit. Severity levels of patients seen in the general hospital were somewhat lower, but were still well above the typical level found at case review of psychiatric patients seen in the community. This supports the proposition, put by Smith, that C-L psychiatry deals with serious problems. 18
A comparative study of this nature needs to take into account pre-existing risk factors for severity, such as age, sex and psychiatric diagnoses. Large differences between settings were observed for two of these three factors. Patients seen in the general hospital were significantly older, there were more women that men in all three settings, and the diagnostic profiles were quite different: there were many more patients with organic and anxiety disorder diagnoses in the general hospital, but many fewer with substance abuse and psychotic diagnoses. Forty per cent of psychiatric inpatient admission assessments were of patients with psychotic dis-order diagnoses. However, differences between settings remained virtually unchanged after age, sex and diagnosis had been adjusted for statistically.
Sharrock and Happell described the role of the nurse in the C-L service in another teaching hospital in the same city as the one in which the present study was conducted. 19 They reported that three of the four most frequent referral problems were risk of self-harm, aggression/hostility, and substance abuse. Interestingly, it is these three problem areas that constitute the Behaviour subscale of the HoNOS, and the present findings confirm that these are at high levels in patients seen in the emergency department. In contrast, they are at much lower levels in that other area in which C-L psychiatry operates: the general hospital. Also consistent with Sharrock and Happell, the most common psychiatric diagnoses in both the emergency department and the general hospital were mood disorders. It is understandable that the highest scores on the two HoNOS Impairment items (items 4 and 5) were found in the general hospital, because these have been found to be strongly correlated with age (Trauer T: unpubl. data, 2003), 16 and they had physical problems of sufficient severity to warrant admission to a medical or surgical ward. It is similarly understandable that the highest ratings on item 6 (hallucinations and delusions) should occur in the specifically psychiatric setting.
It was found that HoNOS assessments were performed predominantly by medical staff and, in particular, registrars. Many, but not all, of these would have attended in-service training in the use of the instru-ment. Other work has shown that this group of staff were able to achieve high levels of accuracy and completeness, using a different reporting system, and that the use of a systematic reporting framework could contribute significantly to their training experience. 20
Finally, we comment on the validity of the HoNOS in the C-L setting, given that it was developed for use in the public psychiatric system without regard to potential applications in other settings. The C-L service studied here responded to referrals from the emergency centre and general hospital, where there was a presumptive significant mental health problem with which clinicians wanted help. To this extent, the HoNOS has face validity, and the fact that it yields high scores supports this, as does the fact that the C-L staff have found it useful in their work.
Footnotes
Acknowledgements
Dr M Salzberg, psychiatrist, Ms J Sharrock, mental health nurse coordinator, Dr R Yewers, director, Consultation-Liaison Service, St Vincent's Mental Health Service, Melbourne, and the numerous psychiatric registrars who conducted the majority of the HoNOS assessments.