Abstract
Background:
Spin is a reporting bias that misrepresents research. Ultimately it can affect surgeon decision making and patient care. Anterior cruciate ligament (ACL) reconstruction is common, but debate continues over optimal treatment modalities.
Purpose:
To identify the prevalence of spin in meta-analysis and systematic review abstracts regarding the treatment of ACL injuries with quadriceps tendon graft.
Study Design:
Cross-sectional study.
Methods:
Electronic libraries (MEDLINE, Embase, Web of Science, Google Scholar) were searched for meta-analyses and systematic reviews regarding the treatment of ACL tears with quadriceps tendon graft. The 9 most severe types of spin commonly found in abstracts were used as an evaluation tool to assess the articles. Two reviewers each performed a blinded assessment of each article for spin. A third reviewer helped after review was done to address any discrepancies between the original reviewers. Further evaluation included year of publication, number of citations, journal impact factor, and A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR-2) score.
Results:
The electronic database search resulted in 986 articles, of which 13 met our inclusion criteria. After review, we found that 53.8% (7/13) of the included articles contained 1 of the 9 most severe forms of spin. Of the 13 articles, 15.4% (n = 2) contained 2 types of spin, and 38.5% (n = 5) contained 1 type of spin. No studies contained ≥3 types of spin. Of the types of spin evaluated, the most prevalent (n = 4; 30.8%) was type 3 (“selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention”). All studies, regardless of the presence of spin, were found to be low or critically low quality according to the AMSTAR-2 assessment.
Conclusion:
This study demonstrated the presence of spin in 53.8% of meta-analysis and systematic review abstracts pertaining to quadriceps tendon graft for ACL reconstruction. Orthopaedic surgeons should learn to recognize spin as they review articles when deciding the treatment course for ACL injuries. Additionally, strict criteria should be considered to reduce the prevalence of spin in orthopaedic literature.
Anterior cruciate ligament (ACL) rupture is a common knee injury with a yearly US incidence between 100,000 and 200,000.1,15,20 The 3 most common autografts for ACL reconstruction are patellar tendon, hamstring tendon, and quadriceps tendon (QT). 19 Of those 3, QT has historically been utilized the least but is becoming more popular in recent literature and in practice.1,17,19 This growing popularity makes it critical to evaluate the data and studies regarding QT graft efficacy and safety to guide orthopaedic surgeons in clinical decisions for their patients.
Systematic reviews and meta-analyses are often used as the standard in evidence-based medicine. As such, it is vital to carefully analyze the results of these reviews to ensure that they provide the most accurate conclusions possible to maximize the quality of care that patients are provided. A major aspect that affects the quality of a study is bias. Bias can distort data and lead to false conclusions. One common type of bias that is often found in literature is “spin,” which Yavchitz et al 32 defined as “a specific way of reporting, intentional or not, to highlight that the beneficial effect of the experimental treatment in terms of efficacy or safety is greater than that shown by the results.” Spin in the conclusion of abstracts has been shown to affect how physicians interpret the findings of studies. 5
One metric that is used to determine the methodologic quality of studies is A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR-2). 29 AMSTAR-2 focuses on the critical steps that must be taken for a systematic review and meta-analysis to maximize quality and minimize overall bias.
The primary objective of this study was to investigate meta-analyses and systematic reviews on ACL reconstruction using QT grafts for the 9 most critical forms of spin found in abstracts as enumerated by Yavchitz et al. 32 The present study includes articles that compare QT grafts with other types of grafts as well as articles that compare different surgical techniques utilizing QT grafts. The secondary objective was to evaluate whether there is a relationship between spin and other study characteristics, such as AMSTAR-2 score, year of publication, number of citations, and journal impact factor. We hypothesized that spin would be present in ≥25% of the abstracts of meta-analyses and systematic reviews discussing QT graft usage for ACL reconstruction.
Methods
A systematic search was conducted on April 27, 2021, by one of the authors (M.T.G.) using MEDLINE, Embase, Web of Science, and Google Scholar. The following keywords were used: “ACL” or “anterior cruciate ligament” and “quadriceps” or “quadricep” and “meta-analysis” or “systematic review.”
Inclusion Criteria
All systematic reviews and meta-analyses regarding the treatment of ACL tears with QT autograft were reviewed. Articles addressing primary and revision ACL reconstruction were included. Only articles written in English regarding human participants were included in this study. There was no exclusion based on the date of publication. Articles that were not systematic reviews or meta-analyses were excluded.
Review
Two independent authors (T.M.T. and J.G.P.) reviewed the initial 22 abstracts to determine if they met inclusion criteria. During this review, they determined 4 articles did not pertain to QT graft use in ACL reconstruction; 2 articles had no abstract available; 1 article had no manuscript available; and 2 articles were not systematic reviews or meta-analyses. After these exclusions, 13 studies were included in the final analysis (Figure 1). Of these, 10 were systematic reviews and 3 were meta-analyses. To determine the types of spin found in the abstract, a thorough review of each article in its entirety was required. The authors who were included in the review process were trained on the topic of spin and AMSTAR-2 grading systems by individually reading articles on the topic of spin2,5,7,32 and AMSTAR-2.28,29 After this training, reviewers read the 13 articles in their entirety multiple times to give each an AMSTAR-2 grade and to assess if the findings of the article were spun in the abstract.

Flow diagram of study selection.
The article by Yavchitz et al 32 entitled “A New Classification of Spin in Systematic Reviews and Meta-analyses Was Developed and Ranked According to the Severity” utilized 122 experts in the field of scientific research to create a list of the 9 most severe forms of spin found in abstracts. This list was used in the current study to evaluate each article for spin. AMSTAR-2, a 16-question survey used to critically examine the methodology of systematic reviews, was used to analyze the 10 systematic reviews in our study. 29 AMSTAR-2 provides a rating system and ranks the quality of the methodology of each study as critically low, low, moderate, or high. 29 The number of times that each article had been cited was recorded on May 5, 2021. Furthermore, the following data of each article were obtained for analysis: the year of publication, the number of yearly citations, and the impact factor of the journal in which the article was published. Given that 2 authors independently decided on the spin and AMSTAR-2 rating, when discrepancies were encountered, a third author (M.T.G.) reviewed the article to yield a consensus.
Although any grading system is subjective by nature, each item within the 9 most severe forms of spin, as presented in Table 1, can be viewed as relatively objective. Objectivity was the goal in mind as the 2 independent reviewers meticulously read each article. The types of spin found in the left column of Table 1 were used as a constant reference such that the reviewers could independently determine the type of spin while reading the abstract and article.
Nine Most Severe Types of Spin per Yavchitz et al 32
Statistical Analysis
Data were organized into an Excel (Microsoft Corp, Version 16.56) spreadsheet that was then imported into R studio (Version 4.0.4). A Fisher exact test was used to independently examine the association of AMSTAR-2 classification and the presence of spin in abstracts. Given the scarcity of meta-analyses and systematic reviews on the topic of QT graft for ACL reconstruction, this study was underpowered, and overall associations between spin and study characteristics could not be determined (eg, journal impact factor, AMSTAR-2 grade, year of publication, and mean number of yearly citations).
Results
The electronic database search resulted in 986 articles, of which 13 met our inclusion criteria (Figure 1). †† The year of publication ranged from 2015 to 2021. The journals that published the articles were Arthroscopy (5 articles), Knee Surgery, Sports Traumatology, Arthroscopy (3 articles), Clinical Journal of Sports Medicine (2 articles), Journal of Knee Surgery (1 article), Revista Brasileira De Medicina do Esporte (1 article), and European Journal of Orthopaedic Surgery and Traumatology (1 article).
Spin was present in 53.8% (7/13) of the abstracts reviewed. Of the 9 most severe types of spin found in abstracts, the most prevalent (30.8%; 4/13) was type 3 (“selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention”). Other common types of spin identified included type 5 (“conclusion claims the beneficial effect of the experimental treatment despite high risk of bias in primary studies”) and type 9 (“conclusion claims the beneficial effect of the experimental treatment despite reporting bias”); both occurred in 15.4% (2/13) of the abstracts reviewed. Table 1 presents a detailed accounting of the prevalence of each type of spin. Appendix 1 (available in the online version of this article) provides a comprehensive breakdown of the 13 reviewed articles by type of spin, year of publication, publishing journal, journal impact factor, AMSTAR-2 score, and annual citations.
Of the 13 articles analyzed, 15.4% (n = 2) contained 2 types of spin, and 38.5% (n = 5) contained 1 type of spin. No studies contained ≥3 types of spin. When studies were reviewed by AMSTAR-2 rating, all 13 were either low or critically low. Other characteristics of the studies that did and did not contain spin are presented in Table 2.
Data Regarding the Characteristics of Studies That Did and Did Not Contain Spin a
Data are presented as No. or mean ± SD. AMSTAR-2, A Measurement Tool to Assess Systematic Reviews 2.
The number of citations among articles ranged from 0 to 153, with the mean being 26.5. The mean number of citations per year ranged from 0 to 24, with the mean being 9.1. The impact factor of the journals ranged from 0.6 to 4.8, with a mean of 3.6. Furthermore, 76.9% (10/13) of articles mentioned that the project adhered to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. 4
Of the 13 articles, 7 compared QT with bone–patellar tendon–bone and hamstring tendon autograft; 2 compared QT with hamstring tendon autograft; 1 compared QT with bone–patellar tendon–bone autograft; and 3 compared different types of QT options (bone plug vs no bone plug, suspensory vs aperture, full vs partial QT).
Discussion
New interest has grown in the use of QT graft for ACL reconstruction, with several meta-analyses and systematic reviews finding equivalent or superior short- and long-term results. ‡‡ With this growing enthusiasm, care should be taken to identify spin within the literature to truly assess the effectiveness of QT graft treatment. Assessing spin in scientific research is a relatively new topic. Abstract spin is recognized to bias interpretation of the results in favor of a particular outcome. 5 By assessing spin in frequently performed surgery, we aim to enhance transparency and validity in study conclusions. Our study identified 1 of the 9 most severe forms of spin in 53.8% (7/13) of the included meta-analysis and systematic review abstracts.
Although no previous studies have examined the prevalence of spin in ACL research, our finding is similar to that of other orthopaedic literature regarding spin bias. Arthur et al 2 and Checketts et al 7 evaluated orthopaedic randomized controlled trials (RCTs) for spin and reported rates of 44.8% in RCTs in specific orthopaedic journals with the highest impact factors and 58.7% in total joint replacement RCTs, respectively. Jones et al 14 identified spin in 34.2% of abstracts of systematic reviews and meta-analyses pertaining to proximal humeral fractures. Spin has also been recognized within orthopaedic trauma literature and rotator cuff literature at rates of 66% and 35%, respectively.25,30 Our assessment of spin within orthopaedic surgery is lower than or equivalent to that in other specialties, such as colorectal surgery 24 and otolaryngology, 9 with recent evidence identifying spin in 82% and 70% of studies, respectively.
Of the different types of spin evaluated, type 3 (“selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention”) was the most encountered (31%) in our analysis. One example of type 3 spin occurred in the abstract of a study by Migliorini et al, 18 which focused on comparing ACL reconstruction with QT versus patellar tendon autograft. The authors reported lower graft failure and anterior knee pain in QT tendon grafts as compared with patellar tendon grafts in the abstract; however, in the results of the study, there was an overlapping confidence interval suggesting no statistical significance when comparing graft failure and anterior knee pain as individual variables. 18 Within the Results section, the authors indicated that QT had a failure rate of 1.39 (odds ratio; 95% CI, 0.49-2.29) as compared with 1.54 (95% CI, 0.76-2.32) for patellar tendon. 18 The rate of anterior knee pain for the QT graft was 2.23 (odds ratio; 95% CI, 0.94-3.63) as compared with 3.75 (95% CI, 2.37-5.14) for the patellar tendon graft. 18
The second and third most common types of spin identified during the analysis were type 5 (“conclusion claims the beneficial effect of experimental treatment despite high risk of bias in primary studies”) and type 9 (“conclusion claims the beneficial effect of the experimental treatment despite reporting bias”), both in 15% of the included studies. One example of type 5 spin occurred in a study by Kanakamedala et al. 16 The article’s abstract reported comparable results between the success of ACL reconstruction using partial- and full-thickness QT autografts even though the study noted the inclusion of studies that contained significant methodological flaws (eg, small sample size, retrospective evaluation of data, limited reporting of technique, and lack of a control group for comparison). An example of type 9 spin occurred in the study by Belk et al, 3 who stated in the abstract conclusion that “QT patients experienced less knee laxity postoperatively as compared with [hamstring tendon] patients.” However, this study also did a bias assessment of the data obtained from their meta-analysis, and it revealed a high or unclear risk of bias in a significant portion of the data used.
The effect of spin on one’s impression of the study results may lead to faulty clinical and operative decision-making and possibly affect patient outcomes. This reality in biomedical research was discussed by Boutron et al, 5 who created an RCT to determine the effects of spin on interpretation of results, and Chiu et al, 8 who explored the nature and prevalence of spin in biomedical research. Boutron et al reported that treatments described in abstracts with spin were perceived as significantly more beneficial and less rigorous (P = .030 and .034, respectively). Additionally, Boutron et al observed that clinicians were more interested in reading the full text of articles with spin in their abstracts as compared with abstracts with no spin (P = .029). As Chiu et al pointed out, not only does spin affect clinical decision-making, but it also leads to research into clinical interventions without strong supporting evidence, resulting in misallocation of resources away from other, possibly more beneficial, scientific research. In a recent study, Otte et al 22 examined >500,000 RCTs and found that >49,000 contained wording to try to express a statistical relationship even when one did not exist, a variant of type 3 spin.
Elimination of spin is a multifaceted obstacle and should be dealt with by educating the authors, readers, and journal editors. Boutron and Ravaud 6 discussed the need to change the perception of spin from “an acceptable practice” to a “detrimental practice.” Although the scientific community takes research misconduct, such as data falsification, fabrication, or plagiarism, seriously and has built systematic methods to filter such research, misrepresentation and distortion of results are less recognized and more often encountered in published literature. Page et al 23 researched the selective inclusion and reporting of specific studies in systematic reviews and revealed this to be a common practice. Therefore, it is critical to educate peer reviewers and journal editors to recognize such distortion of true results to provide a filter within the publication process. Additionally, raising awareness and educating authors and readers will encourage more critical analysis of scientific research while decreasing the influence of research with spin.
Limitations
Even though spin was explicitly defined in the study by Yavchitz et al 32 and our protocol aimed to adhere to the definition described by Yavchitz et al 32 , there remains a subjective aspect of determining if and how much spin exists. Although we attempted to mitigate this subjectivity by independently assessing spin and following the predetermined protocol for disagreements, the nature of the subjectivity remains. Additionally, although our literature search included 986 articles from 4 databases (MEDLINE, Embase, Web of Science, Google Scholar), our strict inclusion criteria limited our study to a small sample size of 13 studies. For this reason, the study’s results may under- or overestimate the true prevalence of spin in systematic reviews and meta-analyses regarding QT graft in ACL reconstruction surgery. This small number of studies also rendered our study underpowered in determining associations between spin and study characteristics. Furthermore, owing to the nature of the focus of our study, the prevalence of spin determined here should not be extrapolated to other specialties or different areas of orthopaedic surgery. Likewise, given the cross-sectional design of our study, the prevalence of spin should not be extrapolated to different time frames.
Conclusion
This study demonstrated the presence of spin in 53.8% of meta-analysis and systematic review abstracts pertaining to QT graft for ACL reconstruction. Orthopaedic surgeons should learn to recognize spin as they draw treatment conclusions from the published literature. Additionally, stricter criteria should be considered to reduce the prevalence of spin in orthopaedic literature.
Supplemental Material
sj-pdf-1-ajs-10.1177_03635465231169042 – Supplemental material for Spin in the Abstracts of Meta-analyses and Systematic Reviews: Quadriceps Tendon Graft for Anterior Cruciate Ligament Reconstruction
Supplemental material, sj-pdf-1-ajs-10.1177_03635465231169042 for Spin in the Abstracts of Meta-analyses and Systematic Reviews: Quadriceps Tendon Graft for Anterior Cruciate Ligament Reconstruction by Matthew T. Gulbrandsen, Taha M. Taka, Joshua G. Peterson, Jun Ho Chung, Hasan M. Syed, Nirav H. Amin, Austin V. Stone, John W. Xerogeanes and Joseph N. Liu in The American Journal of Sports Medicine
Footnotes
Submitted October 28, 2022; accepted March 3, 2023.
One or more of the authors has declared the following potential conflict of interest or source of funding: N.H.A. has received consulting fees from Pacira Pharmaceuticals, Smith & Nephew, Baudax Bio, AcelRx Pharmaceuticals, Trice Medical, Medical Device Business Services, Bioventus, KCI USA, and DePuy Synthes Products; royalties from Smith & Nephew; and hospitality payments from PAVmed and Acumed. A.V.S. has received consulting fees from AlloSource, Bioventus, and Smith & Nephew; research support from AlloSource and Flexion Therapeutics; support for education from Legacy Ortho, Arthrex, and Medwest Associates; and hospitality payments from Stryker. J.W.X. has received consulting fees from Arthrex and Trice Medical, royalties from Arthrex, and support for education from United Orthopedics. J.W.X. also holds stock or stock options from My-Eye. J.N.L. has received support for education from Empire Medical, Smith & Nephew, and Arthrex and hospitality payments from Stryker Corporation and Smith & Nephew. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
