Abstract
The fissure-sealing of newly erupted molars is an effective caries prevention treatment, but remains underutilized. Two plausible reasons are the financial disincentive produced by the dental remuneration system, and dentists’ lack of awareness of evidence-based practice. The primary hypothesis was that implementation strategies based on remuneration or training in evidence-based healthcare would produce a higher proportion of children receiving sealed second permanent molars than standard care. The four study arms were: fee per sealant treatment, education in evidence-based practice, fee plus education, and control. A cost-effectiveness analysis was conducted. Analysis was based on 133 dentists and 2833 children. After adjustment for baseline differences, the primary outcome was 9.8% higher when a fee was offered. The education intervention had no statistically significant effect. ‘Fee only’ was the most cost-effective intervention. The study contributes to the incentives in health care provision debate, and led to the introduction of a direct fee for this treatment.
INTRODUCTION
“Oral diseases are a neglected area of international health.... the tools and best practices are there, and we need to ensure that they are applied and implemented.” [Dr. Margaret Chan, WHO Director-General (Chan, 2007)]
Dr. Chan’s comment is particularly relevant to children—dental decay remains the most common chronic disease of childhood. By age 15, 51% of UK children have experienced decay (defined as D3cvMFT, i.e., including visual caries) in their first permanent molars (Pitts and Harker, 2004). The epidemiology of dental decay is well-known, and is closely linked to social deprivation. In Scotland, 69% of five-year-olds in the least affluent districts have caries experience so defined, compared with 27% of those from the most affluent areas (Merrett et al., 2006).
Methods of effective prevention are also well-known. A recent Cochrane systematic review (Ahovuo-Saloranta et al., 2004) found that fissure sealant treatment, relative to none, reduced decay in children’s permanent dentition by 86% after 12 mos. Preventive sealant treatment for children at risk of caries is supported by the American Association for Pediatric Dentistry, the European Academy of Paediatric Dentistry, and the British Society of Paediatric Dentistry (Nunn et al., 2000; AAPD, 2004; Welbury et al., 2004). Although sealant application is inexpensive, easy, and long-lasting, adoption appears sub-optimal. Fewer than 20% of Scottish 11-year-olds have any first molars sealed (Merrett et al., 2005). Furthermore, treatment does not appear to be targeted at high-caries-risk individuals (Southwick et al., 1999; Tickle et al., 2007).
One reason may be the way National Health Service (NHS) dentists are paid (Chalkley and Tilley, 2006). NHS Scotland uses a hybrid system involving capitation (intended to cover preventive treatment, including sealing molars) and fee-for-service (per treatment item completed). Fee-for-service payments may encourage too much restorative treatment (there being no cost for children), while capitation payments offer no incentive for preventive treatment (Southwick et al., 1999; Scottish Executive, 2003; Scottish Dental Practice Board, 2005).
Remuneration arrangements based on financial incentives—for example, through target payments—are increasingly important internationally, and are the basis of the new UK National Health Service general medical practice contract (Guthrie et al., 2006). However, recent systematic reviews have concluded that the impact on patients’ health status is unclear (Giuffrida et al., 1999; Gosden et al., 2000; Grimshaw et al., 2005).
A second plausible reason for non-implementation is clinicians’ lack of knowledge of the evidence base of treatments. As with incentives, the effectiveness of teaching evidence-based practice skills is unclear (Feder et al., 1995; Freemantle et al., 1999; Parkes et al., 2001).
The purpose of the present study was to evaluate means of improving the sealant rate. Investigation of the effects of a direct financial incentive and of educational sessions on sealant treatment therefore appeared to be warranted. The primary hypothesis was that implementation strategies based on remuneration or training in evidence-based healthcare would produce a higher proportion of children receiving sealed second permanent molars than standard care. Study objectives were to compare the proportions of fissure-sealed second permanent molars of 12- to 14-year-olds attending dentists exposed to the two implementation strategies (separately and combined) or acting as controls, and to estimate their economic implications. The 12- to14-year-old age group was chosen to coincide with the eruption of second permanent molars.
The study was designed to follow recommendations of the UK Medical Research Council Complex Interventions Framework (Medical Research Council, 2000) and Eccles et al.(2005) to evaluate interventions aimed at supporting change in specific theoretical constructs. Such systematic evaluation of interventions to change the knowledge, skill, or behavior of clinicians is increasingly important in health care (Walker et al., 2003)—albeit largely within medical rather than dental care (McGlone et al., 2001).
MATERIALS & METHODS
Study Design
The 2x2 factorial design cluster RCT compared interventions delivered at the dentist level. Randomization was to one of four arms: (i) a fee for applying sealant (fee arm), (ii) a one-day educational workshop on evidence-based practice (education arm), (iii) both interventions (both arm), or (iv) no intervention (control arm). This design allowed the main effects of each intervention to be determined, as well as any potential interaction. Masking group allocation was not possible for either dentists or the research team. Children were not aware of their dentist’s allocation. The main clinical outcome was the percentage of children having at least one sealant treatment by study end.
The theoretical framework for the fee intervention is based on, but does not test, theories of behavior change from both psychology and economics. In psychology, Skinner’s theory of operant conditioning (Boeree, 2006)—in particular, the principle of reinforcement—posits that a reinforcing stimulus following a behavior results in an increased probability of that behavior recurring. The proposed mechanism of behavior change is contingent reinforcement, not a cognitive process. In economics, it is posited that a fee-for-service increases the ratio of benefits to costs for that activity, relative to other activities, making it more likely to occur (Prendergast, 1999).
These theoretical approaches were translated into a practical intervention by the introduction of a financial reward for the service of placing a fissure sealant, thereby reinforcing preventive care management. Dentists could claim a fee of £6.80 for each second permanent molar fissure sealed during a six-month period. The fee was identical to that for restorative sealant application, and therefore represented a realistic revision to the fee scale. The fee, payable immediately on receipt of the dentist’s claim, did not affect NHS capitation payments (£2.76 per month to age 12, £4.01 thereafter).
The education intervention was designed to influence knowledge about evidence-based practice as a way of encouraging preventive care, using current adult learning practice (O’Brien et al., 2001), which advises an interactive approach. The aims were to provide skills to implement an evidence-based approach to clinical practice and to raise awareness of research methods in primary care. Guidelines on targeted caries prevention were used as working examples. Non-attending dentists were mailed the course material and retained in the study on an intention-to-treat basis.
Since sealant is recommended for higher-risk children, and caries is associated with socio-economic deprivation, the following dentist eligibility criteria were applied: (a) working in the four more deprived categories (out of seven) in the Scottish post-code-based system defining deprivation; and (b) treating at least four children aged 12 to 14 yrs per mo. One dentist was selected per practice. Sampling, randomization, and analysis were conducted at arm’s length from the study base by the Health Services Research Unit, University of Aberdeen. Recruitment strategies followed recommendations of recent reviews (Edwards et al., 2003; Foy et al., 2003). Patient eligibility criteria were that the children had to have attended a study dentist during the six-month period, and that they were between the ages of 12 and 14. Dentist Informed consent was obtained prior to baseline questionnaire completion. Patient data were kept confidential, and NHS data custodians confirmed that neither child nor parental consent was required. Research Ethics Committee approval was obtained.
Recruitment and randomization at the practitioner level while measuring patient level outcomes required a cluster design. Consideration of the cluster design effect, recruitment rates, and power calculations meant recruitment of 150 dentists, each supplying data on 25 children (see Appendix). Practitioner-level minimization variables were used to control for possible study arm differences (Appendix Table 1).
Data Collection and Analysis
All 12- to 14-year-olds seen by study dentists during the six-month intervention period were identified through NHS records. A random sample of 25 per dentist was taken, and data collection forms were sent to each dentist for completion from practice records 12 mos post-intervention. This delay allowed for the recording of any continuing effects of the interventions on clinicians’ decisions to fissure-seal. These data were used to obtain the outcome measures presented in Tables 1 and 2 and related text.
The primary analysis estimated the two main effects at the dentist level through Analysis of Covariance (ANCOVA) on the intention-to-treat principle, weighted by number of children seen per dentist. Percentage of sampled children with a sealed first permanent molar was used as baseline treatment control. The study was not powered to detect clinically significant interaction effects, although it allowed for their investigation. Statistical significance for all endpoints was based on 2p < 0.05 (two-sided). Sensitivity analysis with only post-outcome sealant rates (unadjusted) was conducted to test primary analysis results.
The economic evaluation took a societal perspective. Costs used dentists’ time taken to seal two teeth, cost of consumables obtained from clinical guidelines and expert opinion of two experienced practitioners, cost of hygienist or dental nurse time (Netten and Curtis, 2004), and costs to parents. To avoid double-counting, we excluded fees paid to dentists, since they represent transfer payments, and the cost of dentists’ time was already included (see Appendix).
RESULTS
Elimination of duplicate names left a sample of 337 GDPs, further reduced to 284 after non-eligible dentists were excluded. The 149 dentists recruited represent 52% of those eligible. Fifty-five (75%) of 73 allocated to the education intervention attended a workshop, while 48 (63%) of 76 in the fee arms made a claim. Outcome data were provided by 133 (89%) (Fig.). Of a sample of 3682 individuals, usable data were obtained on 2833 (77%). There was a lower baseline of sealant treatment of second permanent molars in the fee and both arms (Table 1). No other significant baseline differences in practice or practitioner characteristics were found (Appendix Tables 1, 2).
Cluster-level analysis shows a significant increase in sealant treatment in the fee arms compared with the other arms (adjusted risk difference, 9.8%; CI 1.8%–17.8%) (Table 2). The increase in sealant treatments in the education arms was not statistically significant (4.1%, CI −3.9%–12.2%). Sensitivity analysis with an unadjusted analysis based only on post-intervention data confirmed similar differences between fee and education. There was no evidence of an interaction between education and fee (interaction effect: −5.6%, CI −20.7%, 9.5%). Incremental cost-effectiveness (% change in outcome per £) relative to controls was: 0.10 (fee), 0.02 (education), 0.06 (both), confirming the fee-only intervention as the most cost-effective.
DISCUSSION
This study was the first experimental study in primary dental care to use a rigorous design to compare the effectiveness of fee-for-service and educational interventions in promoting evidence-based practice. It was also unusual in that it evaluated theory-linked interventions using a clinical outcome. However, three points are relevant to its interpretation. First, despite minimization of several practice-level characteristics, there was a slight imbalance between arms in the number of sealants placed pre-intervention. This was a chance finding and not indicative of the introduction of bias. The primary analyses adjusting for the imbalance and the secondary unadjusted analyses corresponded. Second, the clinical outcome was based on dentists’ reports. While it is thus possible that bias was introduced, this would have applied equally across study arms (see Appendix for a fuller discussion). Third, no follow-up of sealant retention or cost savings are reported here.
In line with theoretical expectations and previous research findings (Gosden et al., 2000; Bonetti et al., 2006), the introduction of a financial reward to reinforce sealant placement increased its likelihood. By study end, 33% of children with erupted second permanent molars seen by ‘fee arm’ dentists had received sealant treatment, representing, after adjustment for baseline differences, 10% more than among children in the other arms. The difference is statistically significant, and, in the context of the high level of caries among Scotland’s children, clinically significant. The fee intervention was also the most cost-effective.
However, only two-thirds of eligible dentists claimed a fee. Possible reasons for this (and indeed for the modest impact of the incentive overall) are that: children seen were not believed to be at risk of caries, or, conversely, already had caries in these molars; it was too much hassle to claim a possibly small amount from the research team, rather than via the routine NHS channel; or the level of the fee was unattractive. A higher fee may have evoked greater take-up, but at the cost of being unrealistic vis à vis the existing fee structure.
The education intervention also showed a positive trend, and although this did not reach statistical significance, we did not rule out possible clinically significant effects. The study was not powered to detect significant interactions between the two variables, and none was found. It cannot therefore be argued that the education intervention in combination with the fee intervention had a demonstrably different effect than either intervention alone.
Our study may provide an early guide to the effects of incentives on treatment decisions and patient care. As such, its results have implications for medical as well as dental remuneration arrangements (Shekelle, 2003; Roland, 2004). Although it has been argued that fee-for-service may be the most effective Pay-for-Performance incentive arrangement (Wodchis et al., 2007), our findings support Epstein’s observation that Pay-for-Performance may prove to be “fundamentally a social experiment likely to have only modest incremental value” (Epstein, 2007).
Findings regarding the education intervention suggest that teaching an evidence-based approach to primary care dentists may not produce readily detectable changes in clinical practice. This result is consistent with the “modest changes in behavior that are seen with traditional educational approaches to the improvement of quality” (Roland, 2004).
Postscript
In November, 2005, a new £7.40 fee-for-service for preventive sealant application on child permanent molars was introduced in Scotland. In its first year (2006–7), dentists claimed for the preventive sealing of 120,989 molars in over 37,000 children—representing 24% of NHS-registered Scottish children in the relevant age groups.
Descriptives of Child-level Characteristics at Baseline (children without erupted 2nd permanent molars excluded)
Primary Outcome Data – Cluster-level Analysis
Consort Cluster Trial Diagram of ERUPT Study. Note: 7s: 2nd permanent molars.
Footnotes
Notes
Acknowledgements
The ERUPT study was funded by the Chief Scientist Office, Scottish Executive, and the Scottish Higher Education Funding Council. The Dental Health Services Research Unit and the Health Services Research Unit receive Scottish Executive core funding. Workshop contributors were: Anne-Marie Glenny, Lee Hooper, Richard Ibbetson, Maggie Leggate, Brian McKinstry, Derek Richards, Chris Southwick, and David Wray. The authors thank participating dentists and practice staff. Views expressed are those of the authors.
Other ERUPT Study Group members were: Frank Sullivan, Head, Tayside Centre for General Practice Research and Development in Primary Care, University of Dundee; Jim Rennie, Postgraduate Dental Dean, NHS Education for Scotland; Daffyd Evans, Senior Lecturer/Consultant Paediatric Dentist, University of Dundee; Elaine Humphreys, National Audit Co-ordinator in Dentistry; Chris Southwick, GDP; Madeleine Murray, Senior Lecturer, Glasgow Dental Hospital and School, University of Glasgow; Brian Cowie, Blair Millar, Andy Kernan, Practitioner Services Division, NHS Scotland; Chris Deery, Edinburgh Dental Institute, University of Edinburgh; Mary McCann, Deputy Chief Dental Officer, Scottish Executive; Sally Wyke, University of Stirling; and Louise Cardno, Diane Lynas, Marilyn Laird, DHSRU, University of Dundee. All study group members listed participated in the present study. National Research Register ID: N0470119687.
This paper is dedicated to the memory of Diane Lynas, ERUPT Study Administrator 2002-5.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
