Do AAO-HNSF CORE Grants Predict Future NIH Funding Success?

Abstract

Objective

To determine (1) whether academic otolaryngologists who have received an American Academy of Otolaryngology— Head and Neck Surgery Foundation (AAO-HNSF) Centralized Otolaryngology Research Efforts (CORE) grant are more likely to procure future National Institutes of Health (NIH) funding; (2) whether CORE grants or NIH Career Development (K) awards have a stronger association with scholarly impact.

Study Design and Setting

Historical cohort.

Methods

Scholarly impact, as measured by the h-index, publication experience, and prior grant history, were determined for CORE-funded and non–CORE-funded academic otolaryngologists. All individuals were assessed for NIH funding history.

Results

Of 192 academic otolaryngologists with a CORE funding history, 39.6% had active or prior NIH awards versus 15.1% of 1002 non–CORE-funded faculty (P < .0001). Higher proportions of CORE-funded otolaryngologists have received K-series and R-series grants from the NIH (P-values < .05). K-grant recipients had higher h-indices than CORE recipients (12.6 vs 7.1, P < .01). Upon controlling for rank and experience, this difference remained significant among junior faculty.

Conclusions

A higher proportion of academic otolaryngologists with prior AAO-HNSF CORE funding have received NIH funding relative to their non–CORE-funded peers, suggesting that the CORE program may be successful in its stated goals of preparing individuals for the NIH peer review process, although further prospective study is needed to evaluate a “cause and effect” relationship. Individuals with current or prior NIH K-grants had greater research productivity than those with CORE funding history. Both cohorts had higher scholarly impact values than previously published figures among academic otolaryngologists, highlighting that both CORE grants and NIH K-grants awards are effective career development resources.

Keywords

scholarly productivity scholarly impact research output research productivity academic promotion academic advancement academic appointment National Institutes of Health funding AAO-HNSF CORE grants CORE

Introduction

Maintaining active research interests and having an impact on scholarship within the field play important roles in the hiring and promotions process in academic otolaryngology, as well as other fields of medicine.^1-11 Identifying and retaining faculty with the ability to secure funding from external sources is advantageous for an institution and takes on special importance in light of recent cutbacks in research support.^12,13 With these issues in mind, several grant programs are focused on providing support for junior practitioners to help them mature into advanced researchers.

One of the grant types offered by the National Institutes of Health (NIH) is the Career Development Award, designated as “K-series” grant. Some of these, such as the mentored clinical scientist development awards (K01, K08, K23), are usually focused on supporting junior faculty early in their careers¹⁴; additionally some of these, such as K12 awards, are institutional awards. Other career development grants include K18 awards (Career Enhancement Award for Stem Cell Research, aimed at midcareer individuals) and K24 (Midcareer Investigator in Patient-Oriented Research) awards. Research (R) Grants are the most common funding mechanism offered by the NIH, particularly the R01 program, which is open to researchers at all levels of their career.

An alternate source of support for early-career otolaryngology investigators includes grants awarded by the American Academy of Otolaryngology—Head and Neck Surgery Foundation (AAO-HNSF) Centralized Otolaryngology Research Efforts (CORE) grants program. Created in 1985, CORE grants are intended to promote career development for investigators hoping to make basic, clinical, or translational science a component of their careers. This program was designed with an eye toward future participation in the NIH peer review and funding process.¹⁵ A previous analysis demonstrated that procurement of AAO-HNSF CORE grant is associated with greater scholarly impact among academic otolaryngologists.¹⁶ A primary objective of the present analysis was to determine whether receiving a CORE grant ultimately leads to future NIH funding success. Furthermore, we were interested in evaluating whether CORE grant recipients in academic otolaryngology receive Career Development (K) NIH grants, Research (R) NIH grants, and multiple NIH grants at a higher rate than non–CORE-funded academic otolaryngologists. Additionally, we were interested in exploring whether CORE grants or Career Development (K) NIH grants have a stronger association with greater scholarly impact.

Materials and Methods

This analysis compares two cohorts: (1) CORE-funded academic otolaryngologists and (2) non–CORE-funded academic otolaryngologists, comparing the proportion of individuals in each of these cohorts who have received NIH funding. We obtained a listing of all CORE grant recipients from 1985 to 2012 from the AAO-HNSF. This listing contained 494 grants awarded to 432 different individuals. An Internet search of these individuals with the term otolaryngology was conducted to determine their practice setting. Only individuals confirmed to be full-time faculty in an academic otolaryngology department were included in this analysis, while the following practitioners were excluded: those not possessing an MD or MD-equivalent degree, part-time faculty, and voluntary faculty. The NIH Reporter website (http://projectreporter.nih.gov/reporter.cfm) was searched to determine whether these individuals had active or previous NIH grants and what types of grants they had received. This resource contains NIH grants dating back to 1989.

In further comparison of CORE-funded faculty versus K-grant recipients, we considered only individuals receiving one of these grants (not both) to facilitate comparison of independent groups, unless mentioned otherwise. In other words, “CORE-funded faculty” includes those who received CORE funding but not K-grants (unless otherwise mentioned), while individuals receiving K-series grants do not include CORE-funded faculty for the purposes of this analysis.

The Scopus database was used to determine publication experience and scholarly impact, as measured by the h-index, of all faculty included in this analysis. An individual’s h-index is the number (h) of their papers that have been cited in the literature at least h times each.^17,18 Its definition, strengths, and weaknesses have been widely discussed in the literature.^11,17-31 While other databases such as Publish or Perish and ISI’s Web of Knowledge contain this information, Scopus has been used in a multitude of analyses utilizing the h-index, and results from this database have been shown to correlate highly with other commonly used bibliometric databases, such as Google Scholar.^{2,9-11,19,24-26,32-43} Publication experience was defined as the number of years from an individual’s first publication to the present.

NIH funding status was similarly determined for practitioners included in the non–CORE “control” cohort. The American Medical Association’s FREIDA database was used to find a list of academic otolaryngology departments, and an Internet search of online faculty profiles was used to gather all available academic otolaryngologists meeting inclusion criteria as specified previously. Any individuals with a CORE funding history were excluded from this group, as they were already counted in the “CORE-funded” academician cohort. Academic rank, scholarly impact (as measured by the h-index), and publication range were determined for NIH-funded individuals. Data collection was completed in May 2013. This study qualified as nonhuman subject research and was thus exempted from needing Institutional Review Board approval per the standing policy of the Institutional Review Board of Rutgers New Jersey Medical School.

Statistical Analysis

Mann-Whitney U-tests and Pearson’s chi-square tests were used for comparison of continuous and categorical data, respectively, with threshold for significance set at P < .05 (two-tailed). SPSS version 20 (an IBM Company, Chicago, Illinois) was used for statistical calculations.

Results

Out of 432 individual CORE grant recipients since 1985, 192 were confirmed to be current full-time MD faculty serving in an academic otolaryngology department as per our methodology. This group was used for comparison to the non–CORE-funded cohort of 1002 academic otolaryngologists included in this analysis ( Table 1 ).

Table 1.

Sizes of Cohorts Included in This Analysis.^a

	Number Non–CORE-Funded	Number CORE-Funded	Number NIH-Funded	Number K-Grant-Funded
Overall	1002	192	227	79
Assistant	439	90	49	27
Associate	240	50	60	23
Professor	323	41	115	28

Among CORE-funded academicians, 11 full-time academic faculty did not have a designated rank.

As previously stated, the present analysis compares 2 cohorts, specifically (1) CORE-funded academic otolaryngologists and (2) non–CORE-funded otolaryngologists. Seventy-six of 192 (39.6%) academicians with a CORE-funding history have received NIH funding, a statistically greater proportion than the 15.1% (152 out of 1002) of non–CORE-funded otolaryngologists who have received NIH funding (P < .01) ( Table 2 ) (P < .0001). Out of these 76 academicians with both a CORE and NIH funding history, 71 (93.4%) received at least 1 NIH grant after receiving a CORE award.

Table 2.

Comparison of NIH Funding Success between CORE and Non–CORE-Funded Academicians.

	CORE-Funded (n)	Non-CORE (n)	Difference (95% CI)	P Value
NIH (any) funding history	39.6% (76)	15.1% (151)	24.5% (17.5%-31.9%)	< .0001
K-series grants	15.6% (30)	4.9% (49)	10.7% (6.0%-1.7%)	< .001
R-series grants	20.8% (40)	9.6% (96)	11.2% (5.7%-17.8%)	< .001
Multiple NIH grants	8.9% (17)	5.4% (54)	3.5% (–0.02% to 8.5%)	.06

In addition to comparing overall NIH funding success between CORE and non–CORE-funded otolaryngologists, the authors were interested in seeing whether differences in NIH success were present between these 2 groups upon further examination of specific grant types. Therefore, the proportion of CORE-funded otolaryngologists who have received multiple NIH grants, any K-grants, and any R-series grants were compared to the proportions of non–CORE-funded who have received multiple NIH grants, any K-grants, and any R-series grants. A statistically greater proportion of CORE-funded otolaryngologists received K-grants (15.6%) than the proportion of non–CORE-funded otolaryngologists receiving K-grants (4.9%) (P < .001) ( Table 2 ). Similarly, a statistically greater proportion of CORE-funded otolaryngologists received R-grants (20.8%) than the proportion of non–CORE-funded otolaryngologists receiving R-grants (9.6%) (P < .001) ( Table 2 ). The proportion of otolaryngologists receiving multiple NIH grants did not differ upon comparison of the CORE-funded and non–CORE-funded cohorts (9.4% vs 5.4%), P = .06, 95% CI of difference, 0.02%-8.5% ( Table 2 ).

Upon assessment of scholarly impact, individuals receiving AAO-HNSF CORE grants (but not receiving NIH Career Development K awards) had a statistically lower h-index than those receiving K-grants (P < .0001) ( Figure 1 ). We considered only individuals receiving 1 of these grants to facilitate comparison of independent groups. When considering the 30 academicians who have received both CORE and K-grants, there was a statistical trend toward higher scholarly impact in these individuals than those receiving only CORE grants (P = .09); additionally, individuals receiving both CORE and K-grants had lower h-indices than individuals in the K-grant cohort (P = .01).

Figure 1.

Scholarly impact, as measured by the h-index, organized by career development resource. K represents NIH K-series career development awards, CORE represents AAO-HNSF CORE awards. Both represents individuals who have received both of these grant types. Error bars represent standard deviations.

We further compared the relative scholarly impact of these groups by controlling for academic rank and publication experience. CORE recipients at the assistant professor level had lower scholarly impact, as measured by the h-index, than their K-grant counterparts (H = 7.1 ± 4.9 SD vs 12.6 ± 7.5 SD, P < .01). These differences disappeared upon comparison of promoted faculty, as there was no statistical difference in scholarly impact among associate professors (H = 13.5 ± 5.9 SD vs 14.4 ± 5.5 SD, P = .69) and professors (H = 21.2 ± 9.8 SD vs 23.2 ± 9.4 SD, P = .39). Seventy-five of 162 (46.3%) ranked individuals who received CORE grants (but no K-grants) were senior faculty (ie, associate professors or professors), significantly lower than the 71.4% of faculty with K-grants at these ranks. Overall, R-series grants recipients were the individuals with the highest scholarly impact (H = 19.2 ± 10.2 SD, P-values < .05).

Individuals with a CORE funding history (but no K-grants) had a statistically lower mean publication experience (15.5 years) than those who had received NIH career development grants (but no CORE funding) (20.5 years) (P < .001). As K-series grants and CORE grants are both intended to promote career development for investigators, the authors felt it to be logical to compare scholarly impact values among individuals in these 2 cohorts. Upon controlling for publication range, individuals receiving NIH Career Development awards had higher scholarly impact among individuals with 10 or less years of experience (P = .0002) ( Figure 2 , Table 3 ). These differences disappeared among faculty beyond this point, with no statistical difference noted among K-funded and CORE-funded individuals upon comparison of 5- and 10-year experience intervals (P-value for comparison of individuals with 11-20 years of experience = 0.14, P-value for 21-30 years = .25) ( Figure 2 ).

Figure 2.

Scholarly impact, as measured by the h-index, organized by publication experience. Top (solid) line represents individuals who have received K-series career development awards from the NIH (but not AAO-HNSF CORE awards), bottom (dashed) line represents individuals who have received AAO-HNSF CORE (but not K) grants.

Table 3.

Comparison of Scholarly Impact between K-Grant Recipients and CORE Grant Recipients.

	CORE Grant^a	K-Grant^a	Difference (95% CI)	P Value
Overall H-index	11.7	18.2	6.5 (5.2-6.5)	< .0001
H-index at < 10 years of experience	4.9	16.0	11.1 (9.2-11.1)	.0002
H-index at 11-20 years of experience	12.7	15.6	2.9 (1.5-4.3)	.14
H-index at 21-30 years of experience	19.0	21.6	2.6 (0.15-5.0)	.25

To facilitate comparison of independent groups, this refers to individuals with only one type of these grants, not individuals with both.

Discussion

Individuals with current or prior NIH Career Development awards had greater scholarly impact, as measured by the h-index, than their colleagues with a CORE funding history ( Figure 1 ). Interestingly, a far higher proportion of K-grant recipients were senior faculty relative to CORE-grant recipients (71.4% vs 46.3%). As mentioned, these differences in scholarly impact were only present among junior faculty, emphasizing that differences in research production may trump the prestige or types of grants one procures in the academic advancement process in many departments. When controlling for years of experience ( Figure 2 ), K-grant recipients had marked and significantly higher h-indices than CORE-funded academicians among those in the first decade of their careers, likely related to the finding that a higher proportion are promoted faculty.

Despite an association with greater scholarly impact among K-grant recipients, a higher proportion of academicians with a CORE funding history successfully received K-series grants relative to their non–CORE-funded colleagues ( Table 2 ). This relationship persisted when considering R-grants and overall NIH funding ( Table 2 ). This suggests that the CORE program may be successful in its stated goals of preparing individuals for the NIH peer review application process.¹⁵ Further supporting this is that the vast majority (93.4%) of academicians receiving support from both organizations had CORE grants that preceded NIH grants. However, the present analysis was not a prospective study that can attribute causation. In other words, it is not clear if individuals who are more likely to pursue future NIH funding opportunities are the ones drawn toward applying for CORE funding or if the CORE program process facilitates the skills and research maturation necessary for a career that includes NIH-supported projects. For example, individuals in the former scenario may have had a more comprehensive research background, collaborated with highly effective (and well-funded) mentors, or simply had more dedicated research time set aside. These possibilities were beyond the scope of our study design but present interesting issues for further exploration in a prospective and/or survey-based study.

In addition to deficits in scholarly impact, CORE-funded individuals (without K-grants) had a statistically lower mean publication experience than their peers with K-grants (P < .001). There are several potential explanations for this finding, an obvious one being the monetary value of NIH awards, with one analysis reporting this to average over $500,000 for otolaryngologists receiving grants.⁴¹ Although fiscal concerns in recent years have resulted in declining NIH funding,¹³ the size of these awards is still many times the magnitude of CORE funding, as the median CORE award in the database provided to use was $10,000. This perspective is valuable in reinforcing the CORE program’s primary goal of preparation for the rigorous NIH peer review process. Additionally, it should be emphasized that CORE grants are aimed toward trainees and early-career investigators, which may partially explain the discrepancies in publication experience we report.

Individuals receiving K-grants but no CORE grants (H = 18.2) ( Figure 1 ), CORE grants but no K-grants (H = 11.7) ( Figure 1 ), both K and CORE grants (H = 13.1) ( Figure 1 ), and R-grants (regardless of other grants; H = 19.2) all had far higher scholarly impact than previously cited values for otolaryngologists. One study of scholarly impact in various surgical subspecialties reported a mean h-index of 7.8 among academic otolaryngologists.⁴³ Comparison with another analysis of otolaryngologists organized by academic rank puts the scholarly impact of NIH- and CORE-funded individuals in this analysis above associate professors (previously reported to have an h-index of 8.13) and for some groups, above the level of professors (previously calculated to be 15.6).¹¹

We have previously explored the relationship between scholarly impact, as measured by the h-index, and NIH funding among faculty from the top 20 NIH-funded otolaryngology departments.⁴¹ That analysis demonstrated a strong relationship between NIH funding and scholarly productivity, as NIH-funded colleagues had far higher h-indices than their non-funded peers at the same 20 departments. Our present analysis builds on this analysis by analyzing the relationship between procuring CORE grants and future NIH funding success, as well as comparing 2 grant programs intended as career development research support tools, namely, CORE funding and NIH Career Development (K) grants. This analysis highlights that individuals benefitting from either of these programs have greater scholarly impact. While K-grant recipients have higher h-indices, receiving CORE funding may play a role in preparing young investigators for future success with the NIH application process.

A limitation inherent to this analysis is our use of the h-index to assess scholarly impact. Departments and institutions may take into account a variety of factors, subjective and objective, when considering research output. Subjective factors, such as an individual’s reputation or the number of one’s “significant” publications, have obvious limitations in that they are not uniform and are considered differently by different individuals.¹⁷ Commonly used objective factors, such as number of publications, also have limitations. Number of publications does not relay the strength, relevance, or quality of a person’s research. One’s total number of citations in the literature may be disproportionately influenced by a single or a handful of papers. We chose to look at the h-index because it is an objective bibliometric that indicates the consistency at which an individual is producing impactful work and has been shown to have a strong association with scholarly impact, academic promotion, NIH funding, and even fellowship training in a multitude of specialties, including otolaryngology.^{2,9-11,19,24-26,29-39,41,43-53} Its definition, strengths, and weaknesses have been discussed extensively in the literature.^{2,9,17,18,21,24,26,31,35,54,55}

This analysis evaluated the 192 CORE-funded practitioners confirmed to be currently in academic practice in an attempt to compare practitioners in similar practice settings. It is likely that the 230 other CORE-funded persons may possibly not be as productive in terms of scholarly impact, whether using measurements such as NIH funding procurement or objective bibliometrics such as the h-index. Another limitation is the retrospective design of this analysis, which makes it impossible for us to attribute causation. A future analysis of interest may include a prospective design focusing on AAO-HNSF CORE grant applicants and other otolaryngologists-in-training along with their subsequent research trajectories. In addition, an attempt was made to create a more robust prediction model based on some of the predictive factors for NIH funding and h-index suggested in this article (including K-grants, CORE grants, years of publications); however, analysis suggested that these factors were not independent using the Durbin-Watson statistic.

Another limitation in this analysis relates to the NIH RePORTER database only reporting awards dating back to 1989, representing a 4-year gap since the start of the CORE funding program in 1985 in which CORE-funded individuals may have received NIH funding that was not included in this analysis. Finally, this analysis was not able to nor was it intended to take into account numerous other sources of support for research in the field, including local organizations, intra-institutional funding opportunities, and even internal departmental mechanisms. These potential sources may have value in encouraging individuals to incorporate research into their careers and providing the skills necessary to mature into advanced researchers.

Conclusions

A higher proportion of academic otolaryngologists with an AAO-HNSF CORE funding history have received NIH funding relative to their non–CORE-funded peers. This is true when further considering individuals awarded Career Development (K) awards and Research (R-Series) grants from the NIH, as a higher proportion of CORE-funded academicians are successfully awarded these grants than their non–CORE-funded academic colleagues. This suggests that the CORE program may be successful in its stated goals of preparing individuals for the NIH peer review application process, although further prospective study is needed to evaluate whether this association represents a cause and effect relationship. Upon further comparison of relative scholarly impact, individuals with current or prior NIH K-grants had greater research productivity than those with a CORE funding history. Both of these cohorts have higher scholarly impact values than previously published figures among all academic otolaryngologists, highlighting that both AAO-HNSF CORE grants and NIH K-grants awards are effective career development resources.

Author Contributions

Jean Anderson Eloy, conception, design, analysis, and interpretation; revision; final approval; Peter F. Svider, conception, design, data acquisition, analysis; drafting; final approval; Vivek V. Kanumuri, data acquisition; revision; final approval; Adam J. Folbe, analysis; revision; final approval; Michael Setzen, analysis; revision; final approval; Soly Baredes, analysis; revision; final approval.

Disclosures

Competing interests: Michael Setzen, speaker for TEVA and MEDA on their Speakers Bureau (not related to current subject).

Sponsorships: None.

Funding source: None.

Footnotes

Acknowledgements

The authors would like to thank Stephanie Jones and the CORE grant committee members for their assistance in providing a database of previous grant recipients.

Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article.

References

Carr

Friedman

Moskowitz

Kazis

Weed

. Research, academic rank, and compensation of women and men faculty in academic general internal medicine. J Gen Intern Med. 1992;7:418-423.

Colaco

Svider

Mauro

Eloy

Jackson-Rosario

. Is there a relationship between National Institutes of Health funding and research impact on academic urology? J Urol. 2013;190:999-1003.

Hauptman

Chow

Martin

Itagaki

. Research productivity in neurosurgery: trends in globalization, scientific focus, and funding. J Neurosurg. 2011;115:1262-1272.

Atasoylu

Wright

Beasley

. Promotion criteria for clinician-educators. J Gen Intern Med. 2003;18:711-716.

Dixon

. Publishing and academic promotion. Singapore Med J. 2009;50:847-850.

Holden

Rosenberg

Barker

. Bibliometrics: a potential decision making aid in hiring, reappointment, tenure and promotion decisions. Soc Work Health Care. 2005;41:67-92.

Sanfey

. Promotion to professor: a career development resource. Am J Surg. 2010;200:554-557.

Sanfey

Hollands

. Career development resource: promotion to associate professor. Am J Surg. 2012;204:130-134.

Svider

Mady

Husain

. Geographic differences in academic promotion practices, fellowship training, and scholarly impact. Am J Otolaryngol. 2013;34:464-470.

10.

Svider

Blake

Setzen

Folbe

Baredes

Eloy

. Rhinology fellowship training and its scholarly impact. Am J Rhinol Allergy. 2013;27:e131-e134.

11.

Svider

Choudhry

Baredes

Liu

Eloy

. The use of the h-index in academic otolaryngology. Laryngoscope. 2013;123:103-106.

12.

Jacob

Lefgren

. The impact of research grant funding on scientific productivity. J Public Econ. 2011;95:1168-1177.

13.

Garrison

Drehman

. NIH research funding trends: FY1995-2013. http://www.faseb.org/Policy-and-Government-Affairs/Data-Compilations/NIH-Research-Funding-Trends.aspx#sthash.PW3i8pW4.dpbs.

14.

K Kiosk—information about NIH Career Development Awards. Accessed June 29, 2013. http://grantsnihgov/training/careerdevelopmentawardshtm. 2013

15.

Centralized Otolaryngology Research Efforts (CORE) Program. http://www.entnet.org/EducationAndResearch/COREGrants.cfm.

16.

Eloy

Svider

Folbe

Setzen

Baredes

. AAO-HNSF CORE grant acquisition is associated with greater scholarly impact. Otolaryngol Head Neck Surg. 2014;150:53-60.

17.

Hirsch

. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A. 2005;102:16569-16572.

18.

Hirsch

. Does the H index have predictive power? Proc Natl Acad Sci U S A. 2007;104:19193-19198.

19.

Benway

Kalidas

Cabello

Bhayani

. Does citation analysis reveal association between h-index and academic rank in urology? Urology. 2009;74:30-33.

20.

Castillo

. Measuring academic output: the H-index. AJNR Am J Neuroradiol. 2010;31:783-784.

21.

Engqvist

Frommen

. The h-index and self-citations. Trends Ecol Evol. 2008;23:250-252.

22.

Gaster

. A critical assessment of the h-index. Bioessays. 2012;34:830-832.

23.

Jeang

. H-index, mentoring-index, highly-cited and highly-accessed: how to evaluate scientists? Retrovirology. 2008;5:106.

24.

Pagel

Hudetz

. H-index is a sensitive indicator of academic activity in highly productive anaesthesiologists: results of a bibliometric analysis. Acta Anaesthesiol Scand. 2011;55:1085-1089.

25.

Pagel

Hudetz

. Bibliometric analysis of anaesthesia journal editorial board members: correlation between journal impact factor and the median h-index of its board members. Br J Anaesth. 2011;107:357-361.

26.

Pagel

Hudetz

. Scholarly productivity of United States academic cardiothoracic anesthesiologists: influence of fellowship accreditation and transesophageal echocardiographic credentials on h-index and other citation bibliometrics. J Cardiothorac Vasc Anesth. 2011;25:761-765.

27.

Poynard

Thabut

Jabre

. Ranking hepatologists: which Hirsch’s h-index to prevent the “e-crise de foi-e”? Clin Res Hepatol Gastroenterol. 2011;35:375-386.

28.

Quigley

Holliday

Fuller

Choi

Thomas

Jr.

Distribution of the h-index in radiation oncology conforms to a variation of power law: implications for assessing academic productivity. J Cancer Educ. 2012;27:463-466.

29.

Rad

Brinjikji

Cloft

Kallmes

. The H-index in academic radiology. Acad Radiol. 2010;17:817-821.

30.

Rezek

McDonald

Kallmes

. Is the h-index predictive of greater NIH funding success among academic radiologists? Acad Radiol. 2011;18:1337-1340.

31.

Sharma

Boet

Grantcharov

Shin

Barrowman

Bould

. The h-index outperforms other bibliometrics in the assessment of research performance in general surgery: a province-wide study. Surgery. 2013;153:493-501.

32.

Lee

Kraus

Couldwell

. Use of the h index in neurosurgery. Clinical article. J Neurosurg. 2009;111:387-392.

33.

Pagel

Hudetz

. An analysis of scholarly productivity in United States academic anaesthesiologists by citation bibliometrics. Anaesthesia. 2011;66:873-878.

34.

Eloy

Svider

Chandrasekhar

. Gender disparities in scholarly productivity within academic otolaryngology departments. Otolaryngol Head Neck Surg. 2013;148:215-222.

35.

Eloy

Svider

Cherla

. Gender disparities in research productivity among 9952 academic physicians. Laryngoscope. 2013;123:1865-1875.

36.

Eloy

Svider

Kovalerchik

Baredes

Kalyoussef

Chandrasekhar

. Gender differences in successful NIH grant funding in otolaryngology. Otolaryngol Head Neck Surg. 2013;149:77-83.

37.

Eloy

Svider

Mauro

Setzen

Baredes

. Impact of fellowship training on research productivity in academic otolaryngology. Laryngoscope. 2012;122:2690-2694.

38.

Eloy

Svider

Folbe

Couldwell

Liu

. Comparison of plaintiff and defendant expert witness qualification in malpractice litigation in neurological surgery. J Neurosurg. 2014;120:185-190.

39.

Eloy

Svider

Patel

Setzen

Baredes

. Comparison of plaintiff and defendant expert witness qualification in malpractice litigation in otolaryngology. Otolaryngol Head Neck Surg. 2013;148:764-769.

40.

Eloy

Svider

Patel

Setzen

Baredes

. In response to “comparison of plaintiff and defendant expert witness qualification in malpractice litigation in otolaryngology.” Otolaryngol Head Neck Surg. 2013;149:649-650.

41.

Svider

Mauro

Sanghvi

Setzen

Baredes

Eloy

. Is NIH funding predictive of greater research productivity and impact among academic otolaryngologists? Laryngoscope. 2013;123:118-122.

42.

Svider

Husain

Folbe

Couldwell

Liu

Eloy

. Assessing National Institutes of Health funding and scholarly impact in neurological surgery. J Neurosurg. 2014;120:191-196.

43.

Svider

Pashkova

Choudhry

. Comparison of scholarly impact among surgical specialties: an examination of 2429 academic surgeons. Laryngoscope. 2013;123:884-889.

44.

Pashkova

Svider

Chang

Diaz

Eloy

. Gender disparity among US anaesthesiologists: are women underrepresented in academic ranks and scholarly productivity? Acta Anaesthesiol Scand. 2013;57:1058-1064.

45.

Agarwal

Clark

Svider

Couldwell

Eloy

Liu

. Impact of fellowship training on research productivity in academic neurological surgery. World Neurosurg. 2013;80:738-744.

46.

Svider

Lopez

Husain

Bhagat

Eloy

Langer

. The association between scholarly impact and National Institutes of Health funding in ophthalmology. Ophthalmology. 2014;121:423-428.

47.

Eloy

Svider

Setzen

Baredes

Folbe

. Does receiving an American Academy of Otolaryngology—Head and Neck Surgery Foundation Centralized Otolaryngology Research Efforts grant influence career path and scholarly impact among fellowship-trained rhinologists? Int Forum Allergy Rhinol. 2014;4(1):85-90.

48.

Svider

Husain

Mauro

Folbe

Baredes

Eloy

. Impact of mentoring medical students on scholarly productivity. Int Forum Allergy Rhinol. 2014;4(2):138-142.

49.

Sunaryo

Svider

Jackson-Rosario

Eloy

. Expert witness testimony in urology malpractice litigation. J Urol. 2014;83(4):704-709.

50.

Kasabwala

Morton

Svider

Nahass

Eloy

Jackson-Rosario

. Factors influencing scholarly impact: does urology fellowship-training affect research output? [published online December 30, 2013]. J Surg Educ. doi:http://dx.doi.org/10.1016/j.jsurg.2013.10.010

51.

Tomei

Nahass

Husain

. A gender-based comparison of academic rank and scholarly productivity in academic neurological surgery [published online December 8, 2013]. J Clin Neurosci. doi:10.1016/j.jocn.2013.11.006

52.

Eloy

Mady

Svider

. Regional differences in gender promotion and scholarly productivity in otolaryngology. Otolaryngol Head Neck Surg. 2014;150(3):371-377.

53.

Folbe

Svider

Setzen

Zuliani

Lin

Eloy

. Scientific inquiry into rhinosinusitis: who is receiving funding from the National Institutes of Health? [published online November 15, 2013] Laryngoscope. doi:10.1002/lary.24525

54.

Purvis

. The h index: playing the numbers game. Trends Ecol Evol. 2006;21:422.

55.

Kelly

Jennions

. H-index: age and sex make it unreliable. Nature. 2007;449:403.