Measuring NHS performance 1990–2009 using amenable mortality: interpret with care

Introduction

There has been a longstanding interest among policymakers and researchers to better understand whether and to what extent healthcare contributes to improving population health. One approach to measure this is mortality from causes considered amenable to healthcare (‘amenable mortality’) (Box 1). ¹

However, one challenge of using amenable mortality as a means to assess health system performance internationally is the difficulty to disentangle factors other than healthcare that are important determinants of population health against the diversity of healthcare systems. The NHS in the UK provides an important natural experiment with which to study the relationship between health system factors and health outcomes as some of the contextual factors can be considered as controlled for because of the common institutional, organizational and financial origins of the NHS in the four countries. At the same time, although the basic principles of the NHS have historically been similar throughout the UK, there have been important differences. Thus, the NHS in Scotland and Northern Ireland has been funded more generously than in England and Wales, with allocations governed by the 1978 Barnett Formula. However, recognizing the low spending on the NHS in international terms, in 1997 the incoming Labour government committed to increasing NHS spending across the UK to the European average and, although this has not yet been achieved, it did result in marked increase in per capita spending in England and Wales relative to the other countries from 1999 onwards ² and a closing of the gap with Scotland (and narrowing that with Northern Ireland) (Figure 1). ³

OPEN IN VIEWER

Figure 1

Changes in relative expenditure per capita on the NHS in the constituent parts of the UK: ratios compared to Scotland

Although both this spending increase and the performance of the NHS in the by now devolved Scottish and Northern Irish administrations, have been studied, research ⁴ has paid limited attention to health outcomes. ^5,6 In this paper, we ask whether and to what extent the changes of the NHS in each part of the UK, including the increase in spending in England, are reflected in health outcomes, using the concept of amenable mortality. This is an important question given that the concept was proposed for routine use within the new NHS Outcomes Framework in England as a means of capturing, ‘at a high level, how successful the NHS is’ in meeting the objective of preventing people from dying prematurely. ⁷ Under the proposed reforms, it is important that healthcare providers and commissioners, including the proposed Commissioning Consortia, understand fully what amenable mortality is able to tell us.

Methods

We obtained data on deaths by five-year age group, sex and cause of death for the years 1990–2009 from the World Health Organization's (WHO) mortality database. ⁸ Deaths were classified according to the ninth and 10th revisions of the International Classification of Diseases (ICD). Population data were obtained from the same source. Data were extracted for England and Wales, Scotland and Northern Ireland. Separate data for Wales are not available from this source.

We calculated age-standardized death rates from causes considered amenable to healthcare by sex, using the list of conditions applied in previous analyses (Appendix 1 – available online at http://jrsm.rsmjournals.com/cgi/content/full/104/9/370/DC1). ⁹ In keeping with that work, we considered half of deaths from ischaemic heart disease to be amenable to healthcare. Age-standardized death rates by sex were calculated for single causes and cause groups and for all ages (0–74 years) by direct standardization to the European standard population. ¹⁰

We included deaths under 75 years only, primarily because of the difficulty of reliably assigning a single cause of death to the often multiple conditions present among those dying at older ages. ¹¹ This includes almost 50% of mortality in the populations in question. In line with earlier work, we applied lower age limits for some causes, ¹² such as diabetes (<50 years).

We adjusted for known discontinuities related to the introduction of automated cause of death coding and the application of selection rule 3 to assigning underlying cause of death in England and Wales in 1993, which resulted, among other things, in an over-estimation of deaths assigned to pneumonia and an under-estimation of those assigned to cerebrovascular disease between 1993 and 1999; we used comparability ratios proposed by the Office for National Statistics. ^13,14

We assessed the change in age-standardized death rates in England and Wales, Northern Ireland and Scotland in two time periods: 1990–1999 and 1999–2009. The second period coincided with the beginning of increased spending in England and the creation of the Scottish parliament. As the latter came into being on 1 July 1999, we consider 1999 as the year of transition, and include it in both periods. We show both relative and absolute changes as each conveys complementary information. We estimated the absolute annual change (‘slope’) between 1990 and 1999 and between 1999 and 2009 by fitting a linear regression function to each of the two time periods. The relative change was estimated as the average annual percent decline throughout 1990–2009. Calculations were carried out using Microsoft Excel® 2007.

Results

In 1990, amenable mortality, for each sex, was highest in Scotland (male deaths: 238.6 per 100,000 population; female deaths: 172.3/100,000), followed by Northern Ireland (male deaths: 223.0/100,000; female deaths: 165.6/100,000) and England and Wales (male deaths: 187.0/100,000; female deaths: 140.0/100,000) (Table 1). By 2009, amenable mortality had fallen substantially in all countries, to 105.4 per 100,000 population among men/boys in Scotland (women/girls: 82.3), 87.0 per 100,000 population in Northern Ireland (women/girls: 74.0) and 83.5 in England and Wales (women/girls: 67.9).

For completeness, the tables show both the annual absolute change in mortality and the annual percentage change. While both are important, they provide different but complementary information. The same absolute reduction in two time periods of overall decline will represent a greater percentage reduction in the second period as the starting point is lower. However, the scope for further declines may be diminished in the second period as the easiest gains have already been made. Consequently, to allow the reader to interpret the data fully, we show both but in our discussion we focus on the latter.

The pace of change differed between countries and over time. In England and Wales the annual percentage rate of decline in amenable mortality was initially less than in Scotland and Northern Ireland (male deaths 2.68% versus 2.79% and 3.12%, respectively; female deaths 2.53% versus 2.9% and 3.39%, respectively). However, its rate of decline increased during the second period (1999–2009), with the percentage annual decline among men/boys overtaking that in Scotland while that for women/girls overtook the two other nations (Table 2; Figure 2). The rapid rate of decline seen for women/girls in Northern Ireland in the first period slowed subsequently.

OPEN IN VIEWER

Figure 2

Annual rate of decline in amenable mortality in the years 1990–1999 and 1999–2009

The annual percentage decline in death rates from causes other than those amenable to healthcare (last column of Tables 1 and 2) was somewhat slower (about two-thirds that of amenable mortality for men and about half for women) but it did increase in the second period everywhere except among women in Northern Ireland.

Aggregate data are not easy to interpret so it is necessary to examine specific causes (Table 2). Deaths from treatable cancers declined in male and female patients in each nation during 1990–1999. Among women they continued to do so, but more slowly in each nation in 1999–2009. Scottish men experienced an accelerating decline in these deaths in the second period. The annual percentage decline from ischaemic heart disease accelerated in each nation, for both sexes, after 1999, as did deaths from other circulatory diseases, which include hypertensive disease, cerebrovascular disease and chronic rheumatic heart disease. Progress in reducing deaths from respiratory diseases has been mixed, with an initial worsening trend among English men being reversed but a slowing of progress in Scotland and Northern Ireland. Death rates from diabetes worsened in Scotland in both periods, as in Northern Ireland after 1999.

Discussion

The principal findings of this study provide new insights into the performance of the NHS in each part of the UK over the past two decades. We already know that the rate of decline in amenable mortality in the United Kingdom as a whole, from 1998 onwards, has been substantially greater than in almost all other industrialized countries, which is consistent with the substantial increase in overall NHS funding from historically low levels. ^9,15 The faster rate of decline observed in Scotland and Northern Ireland in the period before 1999 is consistent with the hypothesis that the higher levels of spending may have allowed it to benefit disproportionately from the expansion in effective treatments in the late 1980s and early 1990s. The acceleration in the rate of improvement in England and Wales from 1999 onwards is consistent with a process of catching up, suggesting that the NHS in England did indeed convert some of the additional resources into better care.

However, the picture is complicated when individual causes of death are considered. Trends in treatable cancer improved more among women than men, with the greatest gains in the first period in all countries. Cancer deaths among women considered treatable are dominated by breast cancer and the early 1990s saw both the introduction of a screening programme across the UK coupled with significant advances in treatment that were introduced everywhere. ¹⁶ By 1999, these were widely implemented and, it would appear, a large proportion of those who might benefit were doing so, limiting the scope for additional gains. Thus, it would not be expected that the increased spending in England and Wales would differentially affect this cause of death. As the example of breast cancer shows, it is always difficult to disentangle the precise contributions of different factors to changes in specific causes of death. There are many things going on at once, so the accelerated reduction in deaths from cardiovascular disease occurred at a time when not only health spending was increasing but also smoking was being banned in public places and new systems of paying general practitioners, with an emphasis on meeting unmet needs, were being introduced. Perhaps the most that can be said is that, collectively, these measures seem to have worked. More worrying, however, is the increasing death rate (or among English and Welsh women, slowing in the rate of decline) from diabetes. This illustrates the main value of amenable mortality, as a high level indicator of progress that points to areas requiring more detailed study.

This analysis has strengths and weaknesses. Its main strength is that it captures the experience of the entire population in the relevant ages in each of the four countries. In addition, there is now a broad international consensus on the conditions included within the definition of amenable mortality, facilitating comparisons with other similar studies.

Its weaknesses relate to certain inevitable caveats that must be observed when seeking to interpret the data further. First, it is necessary to take account of variable lag periods between clinical interventions and outcomes. The impact of differences in the initial treatment of cancer on subsequent mortality may only become fully apparent after five years. In contrast, the results of surgery for major trauma will be apparent at once.

Second, it is important not to over-interpret short-term fluctuations in deaths from specific causes in territories with small populations. Thus, the small increase in death rates from treatable cancer in Northern Ireland, with a population of only 1.7 million, must be treated with considerable care.

Third, while our results have been adjusted for known problems with coding of causes of death, it will be important to be aware of any discontinuities in other countries where comparisons are to be undertaken. The effects of introducing new versions of the ICD are well-known; less so are changes to the rules for assigning ICD codes to causes of death, such as the introduction of automated coding in the UK in the late 1980s that we adjusted for in our analysis.

Fourth, it is necessary to take account of changes in incidence of underlying disease. This can never be done precisely, given the very limited data on disease incidence, but knowledge of general trends should be used to aid interpretation. ¹² Thus, it might be expected that the rising levels of obesity in the UK ¹⁷ would increase deaths from diabetes, all else being equal. This is addressed, in part, by restricting deaths from diabetes to those under the age of 50 years, at which ages most deaths will be associated with type 1 diabetes. However, the situation is more complex with ischaemic heart disease as there are countervailing forces at work. The increasing prevalence of obesity will increase the risk in the population while reduced smoking will decrease it, illustrated by the marked fall in hospital admissions with myocardial infarction following the introduction of smoking bans, ¹⁸ as mentioned above. Disentangling this complex picture is a priority given evidence of a flattening out in the rate of mortality decline among young people. ¹⁹

Fifth, the representation of the UK in international health statistics remains problematic, and we were unable to extract data separately for Wales for the entire period under analysis.

Finally, the attribution of an outcome to a particular aspect of healthcare is intrinsically difficult because of the multifactorial nature of most outcomes. As a consequence, when interpreting findings, a degree of judgement is needed, based on an understanding of the natural history and scope for prevention and treatment of the condition in question. This is explicitly recognized in the decision to include only 50% of deaths from IHD as evidence suggests that IHD is only partly amenable to healthcare ⁹ although it is recognized in the literature on amenable mortality that is a somewhat arbitrary choice.

The study has important implications for the ongoing debate on NHS reform and, in particular, the consequences of the increase in funding for the English NHS after 1999. While the interpretation of our findings must be laced with some caveats, it does seem that the rate of decline in deaths amenable to healthcare in England and Wales had been slower than in Scotland or Northern Ireland before 1999, at a time when it was relatively less generously funded. However, the subsequent injection of additional funds did allow it to accelerate. This is consistent with the evidence from international comparisons of the remarkable progress achieved by the United Kingdom as a whole in recent years.

The study also raises some questions that it has not been possible to address here. In the space available it has not been possible to examine in detail the reasons for the changes in individual causes in each country. This is a fertile area for future research.

DECLARATIONS