Abstract
Background:
Presentations at scientific meetings are often used to influence clinical practice, yet many presentations are not ultimately published in peer-reviewed journals. Previously reported publication rates for orthopaedic specialties have varied from 34% to 52%. In addition, the publication rate of accepted abstracts is a strong indicator of meeting quality, and it has a potential effect on clinical practice. To date, no studies have investigated publication rates in the field of sports medicine, and specifically for abstracts presented at American Orthopaedic Society for Sports Medicine (AOSSM) meetings.
Purpose:
To determine the overall publication rate of abstracts presented at AOSSM annual meetings and whether there were differences in publication rates between poster and podium presentations.
Study Design:
Descriptive epidemiology study.
Methods:
A comprehensive search was performed using PubMed and Google Scholar for all published manuscripts pertaining to abstracts presented at the 2006 to 2010 AOSSM annual meetings. Abstracts were classified according to presentation type (podium, poster) and subsequently were categorized into subspecialty area and study design. For published abstracts, the journal and publication date were recorded.
Results:
A total of 1665 abstracts were submitted to AOSSM annual meetings from 2006 to 2010, with 444 abstracts accepted (26.7% overall acceptance rate); there were 277 podium presentations and 167 posters. Of these 444 abstracts, 298 (67.1%) were published within 3 years in peer-reviewed journals. The overall publication rates for podium and poster presentations were 73.3% and 56.9%, respectively. For the combined years of 2006 to 2010, podium presentations were 2.08 (95% CI, 1.39-3.11) times more likely to be published compared with poster presentations.
Conclusion:
The overall publication rate of abstracts presented at AOSSM annual meetings (67.1%) was much higher than that reported for other orthopaedic meetings (34%-52%), highlighting the overall educational value and information quality of AOSSM meetings. In addition, there was a significant difference in the overall publication rates for podium and poster presentations. These data suggest that the quality and type of poster and podium presentations may not be equal, and these potential differences should be kept in mind when considering changes in clinical practice according to type of meeting presentation. Furthermore, AOSSM annual meeting program planners should consider these results when investigating ways to further improve the quality of research presented.
Paper and poster presentations at scientific meetings facilitate the dissemination of current research. Importantly, novel findings and results of clinical studies presented at such meetings often affect clinical practice. Although all accepted abstracts undergo a rigorous selection process, not all presented abstracts ultimately are published in peer-reviewed scientific journals. Previous studies on publication rates of presentations across several medical subspecialty meetings have reported rates ranging from 36% to 66%.3,9-11,16 One of the highest measures of the quality of presentations at annual meetings is the eventual publication rate of accepted abstracts; thus, efforts to improve meeting quality are inextricably linked to abstract quality and potential for peer-reviewed publication.
The American Orthopaedic Society for Sports Medicine (AOSSM) is a prominent sports medicine group that hosts an annual meeting, which includes, among other things, the presentation of novel research. It is often difficult to select the research that is most likely to affect clinical practice, and the issue has been the subject of review for other orthopaedic surgery meetings. For example, in light of studies by Bhandari et al, 3 who reported a 5-year publication rate of 34% for all abstracts presented at the 1996 American Academy of Orthopaedic Surgeons (AAOS) meeting, and Donegan et al, 5 who reported a 5-year publication rate of 47% for posters and 52% for papers presented at the 2001 AAOS meeting, the Program Committee of AAOS has enhanced efforts to improve the quality of meetings through more stringent abstract selection criteria. 1 Part of the original concept for the AOSSM was “to provide a scientific outlet for presentation and publication of new ideas . . . [and] proposals for future progress in sports medicine.” 2 To enhance the effect of the meeting on evidence-based practice, a central goal should be to select abstracts that are of the highest quality and therefore are most likely to be published. Although several studies have examined the publication rates of presentations at other national meetings, to our knowledge there has not been a published study investigating these outcomes in the field of sports medicine.
To improve the effectiveness of AOSSM meetings regarding the selection of research abstracts for presentation, and to enhance the influence of these meetings on clinical practice, we sought to investigate historical publication rate characteristics of abstracts presented at AOSSM meetings from 2006 to 2010. In light of previous studies demonstrating that the vast majority of abstracts are published within 3 years of presentation,5,14-16 as well as to gain insight into more recent meetings, we chose to study this 5-year sample. The objective of our study was to determine the overall publication rates of abstracts presented during the AOSSM annual meetings. We also sought to determine whether there were differences in publication rates and time to publication according to type of presentation (podium vs poster).
Methods
We performed a comprehensive literature search for all abstracts presented at the 2006 to 2010 AOSSM annual meetings. The years 2006 to 2010 were selected to allow for the investigation of publication rates over a 3-year period after each respective meeting. The submission rates and digital copies of each meeting’s abstract booklet were obtained courtesy of the AOSSM. Those abstracts that were accepted for presentation were then further analyzed.
Abstracts were classified according to presentation type: podium vs posters. Abstracts were subsequently subcategorized into specific orthopaedic areas, including basic science, lower extremity, upper extremity, and miscellaneous (ie, infection, neoplasm, etc), as well as by study design. A method similar to the strategy employed by Bhandari et al 3 was used to search for each abstract listed in the AOSSM annual meeting program using PubMed and Google Scholar, which is in line with the method used in prior similar studies.4,6,8,14,15,19 Abstract searches included the author’s name, presentation titles, and key words found in the title. A successful match indicated substantial similarities between the published manuscript and the manuscript that appeared in the meeting proceedings in terms of the title of the paper, authorship, and abstract. If the initial search did not reveal a publication, the search was expanded to include the last name and first initial of every author and was manually cross-referenced with the key words from the title until a match was found or it was determined that there was no match.
When the search revealed a match, we recorded time to publication from presentation at AOSSM, area within sports medicine, journal name of publication, study design, and study level of evidence. A database of published presentations was created taking into account the total number of presentations, total poster versus total podium presentations, and each paper’s subspecialty category by meeting year. To ensure completeness and accuracy of data abstraction, 2 authors independently searched for 10% of the abstracts in a small pilot phase. After initial pilot abstraction, authors repeated searches in cases where publications could not be found.
Data were analyzed by use of Stata Statistics, version 12.1 (StataCorp LP). Descriptive statistics, including means, SDs, frequency counts, and percentages, were calculated for each of the variables. Further analysis included logistic regression for predicting the outcomes of categorical dependent variables (eg, publication outcome) based on predictor variables (eg, podium status, year), and χ2 tests with Pearson χ2 values for comparisons involving categorical dependent variables. A P value of ≤.05 was considered statistically significant.
Results
From the 1665 abstracts submitted to the AOSSM meeting from 2006 to 2010, 444 were selected for presentation, consisting of 277 podium presentations (62.3%) and 167 poster presentations (37.6%). The overall 3-year publication rate of all accepted abstracts (n = 298) over 2006 to 2010 was 67.1%. Only 14 abstracts were published in years 4 and 5 after presentation for meetings with such time points available. The overall 3-year publication rates for podium and poster presentations were 73.3% and 56.9%, respectively (Tables 1 and 2). There was a small, statistically significant, positive difference in total publication rates when we compared 2006 to 2010 (P = .037). Yearly publication rates are also shown in Tables 1 and 2. There was significant increase in poster publication rate in 2010. Further analysis demonstrated a statistically significant difference in publication rate for podium presentations compared with posters (73.3% vs 56.9%; P < .0001). Podium presentations were 2.08 (95% CI, 1.39-3.11) times more likely to be published than poster presentations (P < .0001) (Table 3). The most common journal for publication (59.4% of publications) was The American Journal of Sports Medicine (AJSM) (Table 4).
Podium Presentations: 3-Year Publication Rate, 2006-2010
Poster Presentations: 3-Year Publication Rate, 2006-2010
Podium Versus Poster Presentations at AOSSM Meetings, 2006-2010
Journals of Publication
Discussion
Our data suggest an overall publication rate of 67.1% for abstracts presented at the AOSSM meetings between 2006 and 2010, which is much higher than that reported in other studies of orthopaedic surgery meetings. For example, with respect to AAOS annual meetings, Hamlet et al 9 reported a publication rate of 46% for presentations from the 1990 to 1992 meetings, Murrey et al 11 reported a publication rate of 44% from the 1993 meeting, Bhandari et al 3 reported a publication rate of 34% from the 1996 meeting, and Donegan et al 5 reported a publication rate of 49% from the 2001 meeting.
Notably, our study found that there was a significant difference in the overall publication rates for podium and poster presentations from the 2006 to 2010 AOSSM meetings. In fact, podium presentations were twice as likely to be published compared with posters. These data suggest that the quality of poster and podium presentations is not the same, a finding noted in other publication-rate studies 1 and one that can aid meeting organizers in highlighting presentations more likely to be published in peer-reviewed journals. Of note, there was a significant increase in poster publication rate in 2010 (also the year of highest podium presentation rate), which could be a product of random variation, or it could reflect the increasing competition for podium positions. This latter point will be interesting to follow in upcoming years. Regardless, physicians who attend AOSSM meetings should be more critical in accepting data presented as a poster, rather than as a podium, in light of the decreased likelihood of the project being published.
In fact, a previous study 5 cautioned against allowing meeting presentations to guide clinical practice. This is based on the low publication rate of abstracts presented at meetings, which reflects an inability of such abstracts to pass the scrutiny of peer review. As a result, it would be helpful to have a keen eye for meeting abstracts more likely to be published (such as those that are podium presentations). Although our study reports higher overall publication rates than many other studies, the publication rate is far from 100%, and the low overall poster publication rate must serve as a caution to physicians who attend AOSSM meetings. In addition, 1 study 3 reported important inconsistencies between abstracts presented at meetings and final manuscript data, while another 12 noted similar conclusions despite changes in details such as sample size. Recent studies5,18 on barriers to publication reported the 3 main reasons for failure to publish after an AAOS meeting to be insufficient time for preparation of manuscript, study ongoing, and relationships with coauthors. Perhaps moving forward, the selection committee could specifically ask submitting authors to comment on plans for manuscript preparation, ongoing data collection, and author relationships beyond the meeting time. Another important observation in our study was that the most common journal for publication was AJSM (56.9%), which is certainly influenced by right of first refusal for AOSSM presentations, as it is the official publication of the society. Keeping these factors in mind, AOSSM committee members can improve selection of abstracts in light of those most likely to be published, therefore having a greater effect on clinical practice.
While our study, to the best of our knowledge, is the first to show a significant difference in publication rates of poster and podium presentations from AOSSM meetings, it has several limitations. First, we limited our search to articles found in PubMed, Medline, and Google Scholar using the method outlined by Bhandari et al. 3 Because our study did not use EMBASE or Cochrane databases, we may have failed to identify publications indexed only in either of those 2 databases. That being said, although it is possible that we underestimated publication rates, the reported publication rates remain higher than those reported in other studies. EMBASE is a large biomedical and pharmacologic database containing more than 28 million records from 1947 to the present, 7 compared with Medline and PubMed, which contain more than 21.6 and 23.5 million records dating to 1950 and 1966,6,13 respectively. Although some studies in other areas of biomedical research have suggested that the use of EMBASE yielded more results compared with Medline or PubMed, 7 the trend may not be the case in orthopaedics. For example, a recent study by Gavazza et al 8 comparing the use of Medline, Google Scholar, and EMBASE to determine orthopaedic hip abstract publication rates found that Medline and Google Scholar identified significantly more publications than did EMBASE. Furthermore, Slobogean et al 17 found that searching Cochrane databases revealed only 36 titles not indexed in Medline or EMBASE. 20 As a result, although our study design may underestimate the true publication rate, it is unlikely that the number is significant. In addition, our study design is consistent with that used in earlier studies on the topic that limited searches to Medline and Google Scholar.1,4-6,8
Second, we limited our literature search to a 3-year period after each AOSSM meeting. We recognize that some studies could be published more than 3 years after initial presentation; however, for the 2006, 2007, and 2008 meetings, for which later time points were available, a minority of abstracts were found to have been published more than 3 years after presentation. In addition, studies from other meetings have reported that the majority of published abstracts are published within 3 years of presentation.14-16 Finally, because the searches were conducted by the authors, it is possible that subjectivity in the search biased the results, limiting comparison with other studies, even though we based the search criteria on those outlined by Bhandari et al, 3 which have been used successfully in many similar studies.
Despite the fact that we found AOSSM meeting abstract publication rates to be higher than the rates reported for many other meetings, it remains notable that a significant number of abstracts are not ultimately published. Furthermore, poster presentations are much less likely to result in peer-reviewed publication than are podium presentations. Readers should be cautious in interpreting the potential effect of all research presented at meetings, whether presented as a poster or on the podium. A critical evaluation of the data, including its potential biases, generalizability, and clinical significance, is paramount when assessing the overall quality of a study. Characterization of those abstracts most likely to result in publication improves not only abstract selection but also meeting quality and potential effect on clinical practice. It is a challenging endeavor to meet this goal while at the same time organizing a balanced program that includes novel work as well.
Footnotes
Acknowledgements
The authors acknowledge the editorial assistance of Lynn S. Cain.
The authors declared that they have no conflicts of interest in the authorship and publication of this contribution.