Abstract
Background:
Randomized controlled trials (RCTs) significantly influence clinical decision-making, necessitating comprehensive reporting of trial outcomes. However, previous studies have demonstrated that reporting of harms among RCTs is often inadequate.
Purpose/Hypothesis:
The purpose of this study was to evaluate reporting of harms among RCTs cited within the American Academy of Orthopaedic Surgeons (AAOS) clinical practice guidelines (CPGs) for anterior cruciate ligament (ACL) injuries, with an emphasis on study characteristics associated with adherence. We hypothesized that >50% of these trials would have incomplete adherence to the checklist items outlined in the Consolidated Standards of Reporting Trials (CONSORT) extension for harms and that a larger sample size would be associated with greater compliance.
Study Design:
Cross-sectional study.
Methods:
We identified RCTs cited in the AAOS CPGs for the management of ACL injuries and assessed compliance with the 18-item CONSORT extension for harms checklist. Descriptive statistics were used to summarize overall adherence to the checklist items, and linear regression analysis evaluated adherence over time.
Results:
Our analysis included 116 RCTs, the majority of which were single-center studies (81.0%). Most trials had sample sizes of 50-100 patients (45.7%) or 101-500 patients (37.9%). On average, trials adhered to 9.25 of 18 CONSORT items (51.4%), with the most compliant study meeting 15 of 18 items (83.3%) and the least compliant meeting 3 of 18 items (16.7%). Overall, 18 RCTs (15.5%) reported ≤33% of items, and 22 RCTs (19.0%) reported ≥67% of items. Studies with >500 patients had significantly higher adherence compared with studies with <50 patients (P = .046). There was no significant difference in adherence based on the disclosure of funding sources (P = .85) or the implementation of blinding (P = .37). Interrupted time series regression analysis demonstrated no significant change in reporting both before (R2 = 0.017; P = .62) and after (R2 = 0.21; P = .16) the release of the checklist in 2004.
Conclusion:
On average, RCTs cited within the AAOS CPGs for the management of ACL injuries adhered to 51.4% of CONSORT checklist requirements, with considerable variability across studies. Improvements in reporting practices are warranted to support evidence-based treatment decisions.
More than 200,000 anterior cruciate ligament (ACL) injuries occur each year in the United States, 18 most commonly affecting male patients in their early twenties and female patients in their late teens. 24 Given the high incidence, lengthy recovery, and significant cost associated with treating ACL injuries, it is essential that physicians are employing appropriate treatment modalities for their patients.4,10,12,30 In 2022, the American Academy of Orthopaedic Surgeons (AAOS) released clinical practice guidelines (CPGs) for the management of ACL injuries 1 as a means of providing evidence-based recommendations, with a focus on identifying high-quality evidence in the form of randomized controlled trials (RCTs).
RCTs have a significant effect on clinical decision-making. As such, these studies must be thorough and transparent in their methodology and reporting. Efforts to improve the quality of trials is exemplified by the Consolidated Standards of Reporting Trials (CONSORT) statement, which was released in 1996, providing a set of guidelines that should be followed by authors conducting RCTs. 25 In 2004, a 10-item extension checklist was added to emphasize the need for adequate reporting in regard to adverse events (AEs) and complications that occur over the duration of a study. 15 This information is critical in educating patients on associated risks, with the intent of modifying susceptibilities to complications and ultimately improving outcomes.
Previous investigations have found both an overall lack of adherence and wide variability in reporting of harms among RCTs in a variety of fields, including adult reconstruction,3,22,32 prompting efforts for improvement. Most recently, the AAOS CPGs for the management of ACL injuries were evaluated for compliance, with the authors finding an adherence rate of only 22%. Given that this is one of the largest and newest collections of RCTs within the AAOS CPGs, we viewed this as an optimal opportunity to investigate whether study characteristics, such as sample size, are associated with rates of reporting of harms.
Our primary aim was to re-evaluate the reporting of AEs among the RCTs comprising the AAOS CPGs for the management of ACL injuries, with an emphasis on RCT factors that may be associated with adherence. We hypothesized that >50% of these trials would have incomplete adherence to the checklist items listed in the CONSORT extension for harms and that a larger sample size would be associated with greater compliance.
Methods
Data Collection
Institutional review board approval was not required for this study, as human participants were not involved. There were 2 authors (R.D.S., R.T.) who identified all RCTs cited in the 2022 AAOS CPGs for the management of ACL injuries. 1 RCTs that were not in English or did not have a full text available were excluded from review. The following information was also collected for each RCT: publication year, journal of publication, number of participating centers, sample size, blinding methodology, and funding type. The impact factor of each journal was recorded using the 2022 Journal Citation Report. 9 The 18-item CONSORT extension for harms checklist was then applied to each study to assess the adequacy of reporting of AEs. 15 For each item, all studies received a binary rating of “yes” (1 point) or “no” (0 points) depending on whether the requirement was satisfied, with the highest possible score being 18.
Statistical Analysis
Descriptive statistics were used to assess overall adherence and compliance with individual checklist items. Associations between study characteristics and adherence were investigated via an independent t test and analysis of variance, with a P value <.05 considered significant. Interrupted time series regression analysis was performed to evaluate adherence over time, specifically before and after the release of the CONSORT checklist in 2004. Regression analysis was also performed to assess the relationship between sample size and adherence.
Results
The AAOS CPGs for the management of ACL injuries includes 342 references, of which 117 are RCTs. The full text for all but 1 RCT was available, resulting in a total of 116 RCTs included for review and data extraction (Figure 1).

Flow diagram of included and excluded studies.
Study Characteristics
The included studies were published between 1990 and 2022, and 16 of them were published before the release of the CONSORT checklist in 2004. Overall, 94 RCTs were conducted at a single center (81.0%), and 22 RCTs (19.0%) were conducted across multiple centers. The total number of participants across the included trials was 27,312 (mean, 235 [range, 22-4564]). About half of the RCTs were unblinded (n = 59 [50.9%]), 41 were single blinded (35.3%), and 16 were double blinded (13.8%). A funding statement was provided by 57 studies (49.1%); the source of funding was private for 24 RCTs (42.1%) and public for 20 RCTs (35.1%), and 13 explicitly stated that no funding was received (22.8%). The study characteristics of included trials are presented in Table 1.
Study Characteristics of Included Randomized Controlled Trials
Adherence to CONSORT Harms Checklist
The mean number of checklist items met per study was 9.25 of 18, for an overall adherence rate of 51.4% (Table 2). No study demonstrated full compliance, with the highest score being 15 (n = 1) and the lowest being 3 (n = 1). Overall, 18 trials (15.5%) reported ≤33% of items, and 22 trials (19.0%) reported ≥67% of items. In addition, 5 checklist items (4a, 4b, 7a, 8a, 10a) had a compliance rate of ≥80%, and 6 items (3b, 4d, 5a, 6a, 6b, 9a) had a compliance rate of ≤20% (Table 2).
Study Compliance With CONSORT Checklist Items a
AE, adverse event; CONSORT, Consolidated Standards of Reporting Trials.
Interrupted time series regression analysis demonstrated no significant change in reporting both before (R2 = 0.017; P = .62) and after (R2 = 0.21; P = .16) the release of the checklist in 2004 (Figure 2).

Interrupted time series regression analysis evaluating checklist compliance before and after 2004.
Regression analysis did not demonstrate a linear association between adherence and sample size (R2 = 0.026; P = .08). However, further analysis indicated significant differences when samples were grouped into subcategories. Notably, studies with >500 patients had significantly higher adherence compared with studies with <50 patients (P = .046). No significant differences were found between other sample size groups in terms of overall adherence. Association analysis between sample size and reporting of item 9a (inclusion of subgroup analyses) demonstrated that trials with >100 patients were more likely to report subgroup analyses than those with ≤100 patients (P = .04).
No significant associations were found between compliance and whether a study disclosed funding information (P = .85). Of studies that disclosed information regarding funding, the compliance rate was significantly higher for those that received funding as opposed to those that stated that no funding was received (P = .001). There was no significant association between reporting of harms and the presence or absence of blinding (P = .37) or between studies that were single blinded and those that were double blinded (P = .31). There was also no significant association between adherence and the impact factor of the journal in which the RCT was published (P = .22).
Discussion
Our findings demonstrate that reporting of AEs was incomplete among the RCTs included in the AAOS CPGs for the management of ACL injuries, with an overall adherence rate of 51.4% across the 116 included studies. Although several checklist items were well reported, including information regarding the timing and mode of AE data collection, one-third of items had reporting rates of ≤20%. The lowest reported items were 3b (distinguishes between expected and unexpected AEs), 4d (includes harm-associated monitoring information), and 9a (describes subgroup exploratory analyses). This information may be helpful in determining where future RCTs can set reporting efforts.
Inadequate reporting of harms has been highlighted across previous studies in a variety of disciplines, including pain medicine, ophthalmology, and oncology.17,20,32 Similar analyses have also been conducted within orthopaedic surgery, primarily focused on other AAOS CPGs, with varied results.2,3,7,19,29 For example, the overall adherence rate among RCTs within the CPGs for carpal tunnel management was only 19%, 19 while that for clavicular fractures was more promising at 52%. 29 Since the implementation of the AAOS CPGs in 2009, the list continues to grow. 31 The current guidelines for ACL injuries reflect one of the most recent developments and expansive documents, with a total of 342 references. A recent evaluation of adherence to its reporting standards demonstrated a compliance rate of 22%, which is much lower than our finding of 51.4%. While a standardized CONSORT template has been utilized for these types of investigations, it remains a subjective measure of evaluation, leading to varied results. Regardless, the present study confirms that reporting of harms was still incomplete among these RCTs, and there remains an opportunity for substantial improvement, as past studies have highlighted.2,3,7,19 As CPGs continue to be published, with the inclusion of larger bodies of recent evidence, continued analyses of reporting of harms become more relevant.
While previous studies within orthopaedics have specifically commented on the poor reporting of AE severity,19,29 our study found overall higher compliance rates, with 52.6% of studies noting information regarding AE grade or severity and 33.6% discussing a validated instrument to report the severity of expected complications. This is a promising finding, given the importance of emphasizing where a complication falls on the spectrum of being relatively benign versus serious enough to readmit a patient or necessitate revision surgery. A common pitfall of reporting of harms is the use of vague language regarding descriptions of AEs, including statements such as “the complications may be well tolerated,” as well as summing overall complication rates, without drawing attention to differences in expected complications among particular patient cohorts. 15 Therefore, while it is encouraging to see an increase in the acknowledgment of AE severity within our study, this is one area that could be improved upon with new CPGs.
A strength of the present study is the investigation of potential factors that may be associated with greater reporting, such as sample size. While we did not find a linear association between these 2 metrics overall, studies with particularly large samples (>500 patients) did exhibit significantly higher adherence compared with studies with <50 patients. This finding may reflect the higher resource availability and methodological oversight associated with larger trials. Additionally, reporting of harms may be substantially lower among very small trials because of their more limited statistical power. Given that these studies are powered on benefits and not harm, they may not be able to sufficiently detect data on harms collected in larger studies, decreasing the ability of authors to report their findings in alignment with CONSORT standards. Even though we might expect limited reporting of harms among smaller trials, these studies do still hold value as part of subsequent meta-analyses. Aside from sample size, this investigation did not demonstrate an association between adherence and other RCT qualities, such as funding disclosure or blinding methodology. For example, a previous study focusing on the treatment of glenohumeral osteoarthritis found that funding disclosure and double blinding were associated with higher adherence. 7 In a similar fashion, our investigation found that studies that disclosed that they had received funding had significantly better rates of reporting of harms compared with studies that explicitly stated that no funding was received. This suggests that improved reporting may be in part caused by greater resource availability and stricter outcome reporting standards imposed by funding bodies.
The importance of adequate reporting of harms may be further exemplified by examining the potential effect of items that were substantially underreported. Item 3b instructs authors to distinguish between expected and unexpected AEs at the time of patient consent and was followed by only 5.2% of included trials. If patients are made aware of the potential complications associated with their management before being enrolled in a study, they may be more likely to report them. 21 As such, adherence to this item may introduce a potential source of bias and therefore affect the complication profiles observed throughout the study period. Further, the manner in which participants are questioned plays a significant role in reporting rates, as administering a checklist-type questionnaire of potential AEs elicits greater reporting than that obtained from open-ended questioning. 5 Therefore, including a limited number of expected events on a checklist before a study, while omitting several others, would likely skew the resulting complication rates that are reported. For these reasons, many authors might instead choose to rely on unprompted reporting of complications to reduce bias, which may explain the markedly reduced adherence to this requirement. Item 9a, which encourages the inclusion of subgroup analyses, was also heavily underreported, with only 16.4% of studies complying. By providing a more focused risk assessment, subgroup analyses are very useful for practitioners when deciding if a certain intervention is optimal for a specific patient. 23 One possible explanation for this underreporting may be sample size. Given that trials with >100 patients were significantly more likely to include subgroup analyses compared with those with ≤100 patients, these smaller studies simply may not have been statistically able to support such analyses.
The implications of inadequate reporting of harms within RCTs are significant and have the potential to negatively influence patient outcomes.6,11,26 An essential skill as a practitioner is the ability to interpret the benefits and harms associated with existing evidence and then apply this risk assessment to individual patients. 17 However, if reporting of complications is incomplete, or based on generic populations without subgroup analyses, then both the conclusions of the study and the readers’ interpretations may be inherently skewed toward the efficacy and traditionally proposed success of an intervention.13,23,33 If practitioners are not aware of the comprehensive harms associated with a treatment, they may be more willing to recommend it to a patient, putting them at risk of unknown or underreported complications.6,26 It also becomes more challenging for physicians to discern between treatment modalities if there is variation and incompleteness regarding reporting of AEs between interventions for the same indication.8,28 Lastly, given that RCTs form the building blocks of systematic reviews and meta-analyses, the damage from incomplete data disclosure may be propagated to future works.14,16
This study did not detect a change in adherence over time since the release of the CONSORT checklist in 2004, a finding that has varied in occurrence among previous investigations2,7 and prompts the need for an intervention. To mitigate these downstream effects of inadequate reporting, we propose several recommendations. The most effective intervention exists at the level of scientific journals. An analysis of 168 high–impact factor journals revealed that CONSORT guidelines, especially the extension items for harms, were infrequently discussed within the instructions for authors. 27 Simply including the checklist on the journal’s website may help to raise awareness of these expectations. Taking it a step further and enforcing adherence as being required for publication would likely elicit more drastic improvements in reporting. This would also address the hesitancy that authors often have to allocate more discussion space to the downsides of an intervention rather than focusing on the benefits. 20 Beyond the level of journals, orthopaedic societies such as the AAOS can play a role in increasing awareness and adherence of these guidelines at their conferences and events, including a requirement to emphasize the reporting of harms during the abstract submission process.
Limitations
Our study focused on reporting of harms, specifically for RCTs cited within the AAOS CPGs for the management of ACL injuries. These findings may differ if adherence was assessed in non-RCTs or in those that were not included in the AAOS CPGs. Lastly, some included studies were published before the release of the 2004 CONSORT extension for harms. Therefore, these trials were at a disadvantage in that their authors would have been less aware of the criteria we used to score the studies compared with authors who had published after 2004.
Conclusion
On average, RCTs cited within the AAOS CPGs for the management of ACL injuries adhered to 51.4% of CONSORT checklist requirements, with considerable variability across studies. This poses challenges for clinicians in accessing comprehensive safety data for making informed treatment decisions. Encouraging improvements in reporting practices can better support evidence-based practice.
Supplemental Material
sj-pdf-1-ajs-10.1177_03635465251321100 – Supplemental material for A Smaller Sample Size Is Associated With Lower Rates of Reporting of Harms in Randomized Controlled Trials Cited in the AAOS Clinical Practice Guidelines for Anterior Cruciate Ligament Injuries
Supplemental material, sj-pdf-1-ajs-10.1177_03635465251321100 for A Smaller Sample Size Is Associated With Lower Rates of Reporting of Harms in Randomized Controlled Trials Cited in the AAOS Clinical Practice Guidelines for Anterior Cruciate Ligament Injuries by Ryan D. Stadler, Ryan Terrany, Suleiman Y. Sudah, Kathryn Whitelaw, Mariano E. Menendez, Charles J. Gatt and Ryan Plyler in The American Journal of Sports Medicine
Footnotes
Submitted July 31, 2024; accepted December 10, 2024.
One or more of the authors has declared the following potential conflict of interest or source of funding: C.J.G. has received consulting fees and research support from the Musculoskeletal Transplant Foundation, has received support for education from Sea Pearl, and owns stock or stock options in Novopedics. R.P. has received support for education from Arthrex. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
