Abstract
Please consider that you have recently advanced to the finals of a major professional tennis tournament. Congratulations! Unfortunately, in the setting of a viral pandemic, no one will be allowed into the tennis arena except for the 2 athletes and 1 videographer. There will be no umpire, no human line judges, and no technicians to run computer line judges. Players will call the lines on their side of the court. There will be no live audience.
As you prepare for the match, you consider the fairness of your opponent. Many shots in tennis land close to the lines that divide “in” from “out.” Will your opponent be tempted to consciously call a ball “out” despite seeing it as “in”? Furthermore, is it possible that your opponent will actually unconsciously see a ball as “out” when the ball really is “in”? What can you expect for the really close calls during critical points? After all, the prize money difference between the winner and runner-up is about $1,000,000. Close calls may be biased in favor of the player making the call. “When in doubt, call it out!”
Just as calls in a tennis match should reflect the underlying truth, findings in a study of actual subjects with actual measurements should reflect the underlying truth in the universe of a target population with phenomena of interest. 5 Bias has been defined as sources of variation that distort the study findings in one direction, resulting in systematic error. 5 Bias can be introduced during the design, conduct, and analysis of a study. The systematic error attributed to bias is often substantially greater than the random error attributed to chance. 8 How does the critical reader recognize bias? While dozens of sources of bias have been named, the following 4 paragraphs introduce a few common types of bias with hypothetical examples in orthopaedic sports medicine.
Bias can be introduced when taking measurements or when selecting subjects from the population. With respect to taking measurements, observer bias is a distortion, conscious or unconscious, in the perception or reporting of a measurement by the observer. 5 Observer bias is the type of bias that concerned the tennis athletes calling their own lines in the hypothetical example. Consciously calling a ball “out” despite seeing it as “in” is cheating, which may be minimized by the presence of the videographer. However, unconsciously seeing a ball as “out” when the ball is really “in” also occurs commonly, which will not be influenced by the presence of the videographer. In orthopaedic sports medicine, observer bias may represent systematic errors in a way that an instrument is operated, 5 such as a tendency to round down goniometer measurements of extension loss after knee surgery. Alternatively, observer bias may be introduced when a postoperative research interview via telephone is conducted with leading questions.
Another bias related to taking measurements is subject bias, which occurs when a distortion of the measurement is made by the subject. 7 Perhaps the athlete wishes to please the team physician to be cleared to play. Or perhaps the young subject tends to report more improvement after receiving a high-tech surgical treatment, followed by physical therapy, than after attending physical therapy alone. Regardless of reason, the subject is systematically distorting the study findings away from the underlying truth.
With respect to selecting subjects from the population, selection bias occurs when patients are chosen in a manner that systematically influences the outcome of the study. 7 For example, subjects who are referred to a tertiary care center for shoulder instability may have more severe disease— more extensive labral tearing, larger Hill-Sachs deformities, and more bony loss about the glenoid—than the “average person” with shoulder instability. Enrolling subjects in a study without attention to the spectrum of disease may lead to a predominance of severe disease, systematically influencing outcomes to favor more extensive surgical treatment. However, milder cases that can be more easily treated may not benefit from this more extensive surgery.
A special case of bias in selecting subjects is nonrespondent bias, which occurs when people who do not respond to a study are fundamentally different from those who do respond. 7 Consider a survey distributed to all athletes in a particular sport that thoughtfully asks the respondent about the number of athlete exposures over the last year and any specific diagnoses of injuries sustained. Injured athletes are more likely to respond than noninjured athletes, leading to an overestimation of the incidence of injury. Similarly, an internet-based survey may further distort these findings, resulting in younger patients responding more commonly than older patients.
How does the critical reader take bias into account when interpreting the findings of a study? Remember: bias is a result of an error in the design, conduct, or analysis of a study. 4 One major effort to reduce or eliminate bias is blinding (or masking). Blinding can involve the subjects not knowing which group they were assigned to, minimizing subject bias. Blinding can also involve the data collectors and data analysts not knowing which group the subjects were assigned to, minimizing observer bias. Use of both these techniques is called double blinding. For example, in a study comparing a regimen of eccentric exercises combined with either ultrasound-guided leukocyte-rich platelet-rich plasma injection plus dry needling or ultrasound-guided dry needling alone in the treatment of patellar tendinopathy, both the treating orthopaedic surgeon and the patient remained blinded with respect to treatment group at least until the 26-week outcomes had been reported. 3 Interestingly, the patients were literally blindfolded throughout the dry needling and platelet-rich plasma procedure. 3
In addition, authors may work to alleviate concerns of bias by fully describing methods for selecting patients and for making measurements. As an example, in a nested cohort of the MOON longitudinal prospective cohort study, predictors of radiographic joint space narrowing in the lateral compartment after anterior cruciate ligament reconstruction were studied. 6 The Methods section described the radiographic technique in great detail, with a training process for standardizing these measurements. 6 The clearly defined protocol that resulted from this process does not seem susceptible to measurement bias.
Actual and perceived bias may result from conflicts of interest. 4 Disclosure of conflicts of interest is required by most medical journals, revealing possible incentives and perhaps deterring investigators from ethically problematic practices. The critical reader spends a bit of time reviewing these conflicts and proposing possible bias that could result from overt or subtle pressures, especially when considering issues such as occupational hazards. 4 As a recent example, a study on the survivorship of meniscal allograft transplantation demonstrated that clinical outcomes did not show differences according to age. 9 The authors declared that they had no conflict of interest. Consequently, the critical reader cannot propose any sources of bias attributed to financial incentives.
In summary, bias leads to systematic error. 5 When looking at bias within a study, I suggest the following approach:
Review the study design, conduct, and analysis. If the study involves double blinding, fully described proper methods, and no conflicts of interest, then perhaps bias is minimal. However, please remember that systematic error attributed to bias is only one factor that can lead a researcher to the wrong conclusions. Chance—that is, the random occurrence of events without systematic influence—can lead to errors in accepting observed associations as reflecting underlying truths. Confounding, because of extra variables correlated to predictors and outcomes, can also lead to wrong conclusions. 4 (Chance and confounding were already addressed in dedicated Critical Reader Editorials.1,2)
Propose potential sources of bias. Content expertise and experience may guide the critical reader to ask, “What could systematically influence these observed findings?” Again, bias has been defined as sources of variation that distort the study findings in one direction, resulting in systematic error. 5
Check consistency with other studies. If multiple studies have similar findings, then the reader should consider that the potential bias may not have actually occurred. 5
In a tennis match, bias in line calling is entirely unwanted. A search for the truth every time the ball bounces has led to line calling by human line judges complemented by computer line judges and an umpire. Even in that setting, sometimes there is evidence of a bad call—a mark on the clay courts, “chalk flying up” from lines on grass courts, or an instant replay on hard courts. In a medical publication, bias in design, conduct, or analysis should be recognized as equally unwanted. There are a lot of close calls in medicine, and there may be little evidence of a bad call. The critical reader does not assume that all calls have been made correctly when interpreting a study in orthopaedic sports medicine research. Bias must be entertained prior to accepting observations in a study as a reflection of truth in the universe.