Superiority Versus Noninferiority

Much of the learning that we obtain at scientific meetings comes over coffee or lunch. A recent topic at lunch was the methodology of how the United States Food and Drug Administration (FDA) approves a new drug or device. For example, Drug A may be a very successful established treatment for a disease, but it may also negatively impact quality of life or have such a short half-life that it must be taken 4 times daily. A new treatment (Drug B) may be considered to replace Drug A if it has similar properties to Drug A but only needs to be taken once weekly. One design that aptly compares these 2 drugs is a noninferiority design. The objective of such a design is to determine whether a new treatment (Drug B) is at least as good as an established treatment (Drug A). Equivalently, a demonstration of noninferiority simply means that Drug B is no worse than Drug A.

Statistical superiority is the paradigm for most orthopedic clinical research. In fact, a recent review of 237 randomized controlled trials (RCTs) in 4 highly regarded orthopedic journals found that 97.5% of published RCTs used superiority designs. The goal of a superiority study is to show that an experimental treatment is statistically superior to a control treatment. Almost half of these investigations had negative results, meaning the researchers failed to reject the null hypothesis that 2 competing treatments were similar. However, a common mistake is to assume that a negative result means that these 2 treatments are equivalent. Such methodology does not allow us to infer that 2 treatment arms are equal in the population. That is, the 2 treatments may very well be different in the population, and the researchers have simply failed to detect that difference. In clinical trial designs, this is called a type II error.

Only 2.5% of the clinical trials use noninferiority designs. This type of methodology is used to demonstrate that one treatment arm is not worse than another by a predetermined amount. This amount is referred to as the noninferiority margin (or delta). This delta is an acceptable clinical threshold such that, if the new treatment is worse than the established treatment, the difference is not clinically meaningful. That is, the new treatment’s response is still sufficiently close to the established treatment’s response such that we remain comfortable saying that the new treatment is no worse than the established treatment. The noninferiority margin must be established at the outset of a noninferiority study and is commonly set by a group of experts in the field. The smaller the delta, the harder it is for a new treatment, drug, or device to achieve statistical parity. A statistically significant result in a study with a delta of 15%, for example, means that the study treatment, drug, or device is not more than 15% inferior. Had the delta been set at 10%, the treatment, drug, or device may not have achieved statistical significance.^1,2

We have developed evidence-based clinical practice guidelines and appropriate use criteria to guide us in applying the best available evidence when offering treatment for our patients. As we become more sophisticated in comparing treatment options, we need to be even more aware of the statistical methodology that is being employed. Superiority methodology is designed to identify whether one treatment, drug, or device is better than the alternatives. The lack of statistical significance in a superiority model does not imply that the treatment arms are equal. Noninferiority statistical methodology is not comparable to superiority methodology. It has a lower requirement to achieve statistical significance and is highly dependent on the delta.

Understanding the complexities of modern statistics, Foot & Ankle International has a consultant statistician to help your editors provide the best available evidence. The FootForum would love to hear your comments. Send your comments to FAIjournal@aofas.org.