Reducing uncertainty in orthodontic treatment: The wheel keeps turning

Introduction

One of the characteristics of human life is that we continuously face uncertainty. This permeates in all that we do, and we all tolerate or manage the existence of uncertainty to different degrees. This may range from merely wondering about the weather to managing significant life crises. Importantly, risk is present in all our clinical decisions. While we may feel that we are taking the ‘correct’ decision most of the time, it is a unique and possibly poor clinician who is 100% certain in all of their treatment choices. It is clear that uncertainty in clinical decision-making is reduced by experience, but it is also potentially reduced by research evidence. In effect, we research to overcome uncertainty about our treatment. We hope that research evidence will reduce our risk to minimal or acceptable levels (Logan and Scott 1996); however, we are unlikely to eliminate all uncertainty.

This then makes us consider that if we are to practice evidence-based care, we need to combine our clinical experience with research evidence and patient preferences (Sackett et al. 1996). Importantly, when research evidence is present, this should be more relevant to the final treatment decision than clinical experience. As a result, we need to think about how we get evidence and the factors that may make its interpretation difficult.

Confidence

One way of considering uncertainty has been put forward by Michael Rawlins (Chair of the National Institute for Health and Care Excellence) who states that ‘uncertainty is fraught with misinterpretation and that he prefers to consider our level of confidence in a finding or decision’ (Sense about Science, 2013). I think that this is a clinically relevant way of addressing the problem and that we should review the level of confidence that we have in our ‘evidence-based’ decisions. This brings us to the interpretation of research papers and the statistics that indicate levels of confidence (or uncertainty). Importantly, these show whether a finding is not merely a random event.

One method of evaluating confidence is to interpret confidence intervals (CIs), and we are only just getting round to including these in orthodontic research. One way of understanding this is to consider a study in which we want to identify the average overjet of 11-year-old children in the UK. We cannot make this measurement on every 11-year-old child, so we select a sample and come up with a mean overjet measurement. Because this is only a sample, we are uncertain on the accuracy of this measurement, so we calculate the 95% CI. This will indicate the range of values that we would expect the overjet to fall into for 95 out of 100 repeats of the data collection. The narrower the CI, the less the uncertainty. This information is vital to understanding the results of a piece of research.

I want to illustrate this with the results of a recent systematic review into methods of moving molars distally (Jambi et al. 2013). In this review, they included four studies in their meta-analysis. This showed that for a total sample of 75 patients, intraoral distalising appliances were more effective than headgear in moving molars distally, by 1.45 mm. It is clear that this difference is small and may not be clinically relevant. We also need to look at the CIs. These range from −2.74 to −0.15, which means that if we repeated this study 100 times, then 95 times out of 100 the ‘true mean’ would fall between −2.74 to −0.15. We can interpret this as representing a high degree of uncertainty in this area of our treatment. This is because the values represent a wide range and include a value of 0.15, which is not clinically significant. Therefore, we can conclude that the mean difference between treatments is not significant and that the finding has a high level of uncertainty. As a result, we should base our decisions on other factors. For example, our tolerance of risk in providing headgear with its inherent serious, although rare, risks to the patient.

We also need to remember that if we are so sure about a treatment, we may not need to carry out any similar research. One example of this may be the question of the effectiveness of early treatment of class II malocclusion. Again, I would like to consider a Cochrane Review (Batista et al. 2018). The results are clear and the meta-analysis of three large trials reveals that there is no evidence to suggest that treatment when a child is approximately aged 8 years old provides any advantage over one course of treatment provided in adolescence in terms of avoiding treatment, reducing the complexity of Phase II treatment, the need for extractions and the final result of treatment. It may be likely that further research will not markedly change these conclusions.

Nevertheless, we found less clarity on the issue of reduction in the incidence of incisal trauma when we looked at the effect of providing treatment early with a functional appliance. We found that there was a reduction in the incidence of new incisal trauma at the end of all treatment. This appeared to be clinically significant, with 30% (51/171) of participants reporting new trauma incidence in the late treatment group compared to only 19% (31/161) in the early treatment group. The odds ratio was 0.56 (95% CI = 0.33–0.95). This means that the odds of incisal trauma in the early treatment group are 56% less than the odds of trauma in the group that did not get early treatment. The CIs are still wide, representing uncertainty, but they include clinically significant values. As a result, we can give this information to the patient and parent as part of informed consent. Importantly, it is clear that if researchers are considering repeating these studies, perhaps the primary outcome should be the incidence of different types of trauma to the incisors?

Absence of evidence

I sometimes feel frustrated when I read a trial, and the authors report that there is no difference between the treatments and further research is needed. However, again, the interpretation of these negative findings is far from straightforward. It is easy to interpret ‘negative’ findings by suggesting that the treatment did not have an effect. This has been discussed over many years, and various researchers have stated that ‘absence of evidence does not mean evidence of absence’ (Altman and Bland 1995). In other words, if we do not find a difference in a study, then it is not correct to state that the treatment ‘does not work’. Our only conclusion is that the study did not detect any differences between the treatments. There are several main reasons for this. Firstly, the new treatment may, indeed, be no better than the other treatments under investigation. Alternatively, the study may not have sufficient power to detect a difference, even if it existed. That is, the study was not well designed.

If we look back at the conclusion of the class II review that I mentioned earlier, we cannot conclude that ‘early treatment does not work’. We can only decide that we have no evidence that it ‘works’. Nevertheless, we can explain to our patients that a particular treatment does not have an advantage over another. Again, this information helps them make an informed decision.

Key opinion leaders: gurus and turning the wheel

I hope that I have outlined a precise method of approaching uncertainty in research. Unfortunately, obtaining and interpreting clinical research information is not always straightforward. In an ideal world, the best way to get accurate information about new treatments is the refereed literature. Unfortunately, in the ‘real world’ this is difficult because of the large number of published papers, difficulty in accessing the journals behind paywalls and, perhaps most importantly, a potential lack of understanding of increasingly complex scientific methods. As a result, we tend to get information from conferences, word-of-mouth, advertising, social media and key opinion leaders (KOLs)/orthodontic gurus.

A KOL is defined as ‘A physician who influences their peers’ healthcare practice, including their prescribing behaviour’. Moynihan, somewhat harshly, outlined this role further (Moynihan 2008) when he stated:

‘These influential doctors are engaged by industry to advise on marketing and help boost sales of new medicines. Across all specialities, in hospitals and universities everywhere, many leading specialists are being paid generous fees to peddle influence on behalf of the world’s biggest drug companies’.

KOLs provide input to developments: they test the products and then give presentations on their impressions about their clinical performance. Not all of them receive payment. Nevertheless, there is a risk of a conflict of interest that adds to the uncertainty of the treatment that they promote.

The conference

KOLs mostly influence our conferences, but they are increasingly using social media. At the meetings, they use two main avenues to disseminate information. One is the main scientific programme and the other is the trade exhibition. If they are on the main programme, they present the information as part of a lecture. Similarly, in the trade exhibition, they give short lectures and are found on the stands talking to delegates who gather like bees around a honeypot. At the periphery of these exhibition stands I have frequently heard the phrase ‘Dr **** is recommending this treatment, so I shall give it a go’. Some companies even hold their conferences, at which many of their KOLs speak to rows of entranced delegates worshipping at the feet of their gurus. They have a similar influence on social media.

We now need to consider whether this is a problem. First, we need to remember that companies employ KOLs to influence clinical practice. In short, they promote products. We have all seen this promotion in the absence of scientific evidence. Paradoxically, we do not see or hear them mentioning the scientific research that shows that the treatment/philosophy/method of speeding up treatment does not work!

Solutions?

There is a solution. First, conference organisers should insist that all presenters declare if they are a KOL at the start of their presentations. Second, the KOLs should be aware that they have an ethical duty to both clinicians and patients. In this respect, they need to be clear about the level of evidence that they are using when they promote a new treatment. Furthermore, when research is published that does not support the claims that they have been pushing, they should hold up their hands and state that they were ‘wrong’. Finally, as clinician scientists, do we need to listen to the KOL or should we learn to interpret the uncertainty in clinical research?