Abstract
Publications, citations and h-index are three quantitative measures of scholarly productivity commonly used to evaluate university academic staff members. The quantity and quality of an individual's publication record is often an integral part of promotion and tenure decisions as well as post-tenure notoriety and awards. In this study, the Scopus database was queried to provide information on the scholarly productivity of 83 mining engineering academic staff members at accredited mining engineering schools in the United States. The data retrieved in this study include the number of publications, number of citations and h-index, for each individual academic staff member. Data for individual academic staff members was then classified by academic level/rank, institution and primary research sub-discipline to produce meaningful comparative results. The data show that the average number of publications per academic increases from 4 to 22 to 49 for assistant professors, associate professors and professors, respectively. Citations (9–83–290) and h-index (1–3–6) follow similar trends. Further analysis indicates that mineral processing and mine electrical systems are the most highly cited sub-disciplines. When aggregated by academic institution, the data show that the productivity of academic staff is linked to Carnegie Research classification. Finally, the data show a clear Pareto tendency with the highest 20% of academic staff members contributing 80% of the overall citations.
Introduction
At colleges and universities in the US, the research productivity of faculty/academic staff members is often assessed by the individual's publication record in peer-reviewed journals. The axiom ‘publish or perish’ has proven true as the number of peer-reviewed publications is often used as an integral component of internal promotion and tenure decisions as well as post-tenure annual evaluations and merit-based pay increases. Externally, an academic staff member's publication record promotes appreciation and notoriety among peers. An established record of peer-reviewed publications creates recognised expertise and can be a significant factor in research grant competition as well as other external honours and awards. As a result, college-level administrators and department chairs are routinely analysing inter-college and inter-departmental statistics to establish goals and benchmarks.
Most academic staff evaluation criteria includes language that acknowledges both the quantity and quality of scholarly publications. While the raw number of publications is relatively easy to measure and document, the quality of an academic staff member's publications is much more difficult to assess. Several metrics have been proposed, but two of the most broadly accepted and widely utilised are the number of citations and the h-index. In a 2013 survey of the Society of Mining Professors membership, 70% of the respondents indicated that their universities consider number of citations when going through the promotion process (Saydam and Kecojevic 2014). Similarly, 52% of the respondents indicated that the h-index was also used in the promotion process. These trends were especially noteworthy and significant for Australian universites.
While assessing citation records was once a cumbersome process, several online databases, including Google Scholar, Web of Science and Scopus, automatically record scholarly productivity data. Noting the limits of raw publications and citations in assessing and individual's scholarly impact, Hirsch (2005) developed the h-index as a sole indicator of research output. Per Hirch, ‘A scientist has index h if h of his/her papers have at least h citations each, and the other papers have no more than h citations.’ Therefore, in order to attain a high h-index, a scientist must publish many papers which are heavily cited.
Evaluation of the scholarly productivity of various academic subsets has been done for several academic units, including: accounting (Hasselback, Reinstein and Schwan 2000), library and information science (Meho and Spurgin 2005), computer science (Meho and Rogers 2008) and cross-discipline (Fairweather 2002). All of these studies show the need for specialised benchmarks, as the expectations and trends for individual disciplines varies widely. For example, Fairweather (2002) shows that the average number of publications over 2 years varied from 1.49 for academics in fine arts up to 5.08 for academics in health science. The US mining engineering programmes offer a unique opportunity for this type of study, as the total number of academic staff members is relatively small. As a result, the entire population, rather than a sample, can be assessed while still making meaningful comparisons between various sub-disciplines and academic ranks.
The status and direction of the US mining engineering schools have been reviewed and analysed in several recent publications (McCarter 2007; Poulton 2012; Sevim and Honaker 2012). National undergraduate degrees awarded at US mining engineering schools have dropped significantly from a high of 467 in 1981 to a low of 56 in 1990 (Fig. 1). From the period of 1981–1990, several mining engineering programmes were dissolved and ultimately assimilated into other engineering degrees. Nevertheless, since 2005, the remaining mining engineering programmes have seen a new resurgence. Nearly, all programmes have increased numbers individually, and nationally, the number of graduates has steadily increased since 2005. While many are uncertain if this trend will continue, the need for a new generation of junior and senior academic staff members is guaranteed. Many of the current academics (part of the pre-1990 era) are close to retirement. Since the cohort of new academics must undergo the tenure process, a thorough review of current productivity statistics may provide suitable benchmarks and goals for these individuals.
Bachelor's degrees awarded in mining engineering at US accreditation board for engineering and technology (ABET) schools from 1970 to 2013. Data after (Engineering Trends 2007; Gibbons 2009; Yoder 2014)
Scopus database
Scopus (Copyright, Elsevier B.V.) is an online abstract and citation database that covers peer-reviewed publications, including scientific journals, books, conference proceedings and patents. As of 2014, the database contains over 50m records from 5000 publishers dating back to 1823 (Scopus 2015). However, the citation analysis only covers recent articles published after 1995. The database may be queried using several individual or publication indicators, including author names, keywords, date ranges and affiliations. The database also produces reports for individual authors that provide analysis of publications, citations and h-index over time. For mining engineering academics, the Scopus database includes nearly all peer-reviewed journals and most major conference proceedings, including the national Society for Mining, Metallurgy and Exploration (SME) conference. The database generally does not include proceedings at local conferences.
Various articles have reviewed online citation analysis systems, including the Scopus database (e.g. Bakkalbasi, Bauer, Glover and Wang 2006; Meho and Yang 2007; Bar-Ilan 2008; Falagas, Pitsouni, Malietzis and Pappas 2008). While these prior studies cannot conclude if Scopus is universally better than other online alternatives, we have found that Scopus has a wider coverage of journal and conference papers when compared to Web of Science. Furthermore, Google Scholar only records statistics for those that have created an account.
Objectives of study
In this paper, the scholarly productivity of academic staff members at accredited US mining engineering schools is analysed. Here, scholarly productivity is defined as the quantity and quality of publications, as indicated by the number of publications, the number of citations and the h-index of individual academic staff members. This data will provide clear metrics and benchmarks for individual academic staff seeking promotion and tenure as well as college administrators who must assess mining engineering programmes. The data may also be used to compare the field, in general, to other similar scientific and engineering disciplines. To the authors’ knowledge, this type of rigorous individual assessment has never been reported for the academic staff of mining engineering programmes in the US.
Methodology
Scope of study
For this study, the scope of the analysis has been restricted to tenured and tenure-track academic staff members at schools that offer a Bachelor's degree in mining engineering and are accredited by the Accreditation Board for Engineering and Technology (ABET). Accreditation in the US is voluntary and typically conducted by independent, non-governmental agencies. Accreditation Board for Engineering and Technology is one such accrediting agency that reviews programmes in applied science and engineering. Accreditation Board for Engineering and Technology is considered the ‘gold standard’ for accreditation of US engineering programmes. They evaluate and periodically re-evaluate programmes to ensure that the educational standards and quality are maintained (ABET 2015). More information on the accreditation of US mining engineering programmes is given by Kecojevic, Grayson, Saperstein and Karmis (2008).
The distinction to examine academic staff members in mining engineering programmes provides a convenient and logical partitioning, but excludes several sister disciplines including: geological engineering, materials science and metallurgical engineering. In the US, ‘tenured and tenure-track’ faculty/academic staff members include those at the assistant professor, associate professor and professor levels. Occasionally, the term ‘full professor’ is also used to identify those at the professor level. This addition provides a clear distinction from the assistant and associate professor levels. Other academic staff, including those with the terms, ‘teaching,’ ‘research,’ ‘adjunct,’ and ‘emeritus’ in their title, were excluded from the analysis. The 2014 SME Guide to Mineral and Material Science Schools was used to identify schools and academic staff members (SME 2014).
Table 1 includes the schools names, total enrolment data and academic staff populations for the 13 schools identified in this study. The table also includes the Carnegie university designations. The Carnegie Classification™ uses self-reported empirical data to categorise universities in order to reflect the diversity of institutions of higher education. The Carnegie Classification™ is not a quality-based ranking system, but rather organises universities with respect to their primary function and overall mission. Since the US lacks a national organisation of the higher education system, the Carnegie Classification™ is often used as the de facto standard for taxonomical indexing of US institutions. While recent revisions have added more detail and flexibility to the indexing system, the classifications have traditionally differentiated research institutions (and the level of research activity) from liberal arts, and other special focus institutions (McCormick and Zhao 2005; Carnegie 2015). For this study of mining engineering schools, the most relevant distinctions are made between the very high research activity universities, the high research activity universities and other universities. Further details on the Carnegie Classification descriptions and methodology can be found on the classifications website (Carnegie 2015).
List of US accreditation board for engineering and technology (ABET) accredited mining engineering programmes
RU/H: Research University (high research activity); RU/VH: Research University (very high research activity); Bac/Diverse: Baccalaureate Colleges (diverse Fields); Spec/Engg: Special Focus Institutions (schools of engineering); P: number of professors; AP: number of associate professors; ASP: number of assistant professors; Fr: freshman, first year; So: sophomore, second year; Sr: senior, fourth year.
Data retrieval and analysis
Using the universities included in Table 1, a complete list of tenure and tenure-track academic staff in the mining engineering discipline was compiled. This population set includes 19 assistant professors, 27 associate professors and 37 professors, providing a total of 83 individual academic staff members. Data on the number of publications, citations and h-index were retrieved from the Scopus database (www.scopus.com). This data retrieval was conducted during August 2014, and the results represent a snapshot of the productivity at that time. Names of individual academic staff members were entered into the Scopus query system, and affiliations and other keywords were used to isolate authors of interest. Scopus was able to produce an individual author report for all but one individual (a full professor). For this academic staff member, Scopus could not isolate individual statistics, as the name and keyword search produced several researchers with the same name. This individual was thus omitted from any other downstream analysis. For the remaining 82 academic staff members, the data retrieval produced 2434 documents with a total of 12 844 citations.
Additional data, including the rank/level (assistant, associate or full professor) and primary expertise, was gathered from departmental websites and the SME Guide to Mineral and Material Science Schools. The expertise classifications used in this study include 11 primary topics widely considered traditional mining engineering sub-disciplines. This expertise classification was done for individual academics, rather than for each individual publication. While some academics may have publications in more than one expertise area, all publications were assigned based on the primary focal area of the individual. These topics are listed below with detailed descriptions:
Environmental: acid mine drainage, aqueous chemistry, sustainability Explosives: surface and underground blasting, rock fragmentation, non-mining explosive applications CBM/natural Gas: coal bed methane and natural gas extraction from mining, mine degasification, reservoir modelling Health and safety: occupational health and safety, training, dust, noise, risk analysis Mine electrical systems: mine power, mine control systems Geostatistics/mine valuation: exploration, mineral economics Mineral processing: mineral beneficiation, coal preparation, separations, comminution, surface chemistry Rock mechanics: numerical modelling, rock mass classification, experimental methods, slope stability, ground control Surface mining: surface mine design and planning, mining equipment, environmental and safety aspects specific to surface mines Underground mining: underground mine design and planning, tunnelling, underground construction Ventilation: modelling of ventilation systems, experimental methods.
In addition to these 11 topics, the research of four academic staff members did not correspond well to any of the traditional sub-disciplines. These topic areas included two professors specialising in geophysics, one in particle mechanics and one in big data and automation. For the analysis, these four academic staff members have been classified together, as a twelfth category: ‘other/non-traditional.’
For the data presented in this publication, the names of individuals and schools were removed to ensure confidentiality. While all of the data presented are publically available, this confidentiality ensures that the results and findings in this paper are unbiased and impartial. Finally, in order to make meaningful comparisons, the raw data have been organised by academic level, school and sub-discipline. When appropriate, the data are presented on a per academic staff member basis to negate the influence of large departments or sub-disciplines.
Results and discussion
Figure 2 shows the scholarly productivity data for all individuals organised by academic rank/level. In this figure, the productivity datasets are presented as box-and-whisker plots showing the minimum, maximum, median and first and third quartiles. Averages are also noted; however, in all cases, the averages are skewed upward owing to extremely high positive numbers for the highest 10% of the academic staff members. It should be noted that to preserve a meaningful scale, some of axes have been adjusted to exclude these maximum values. The numeric maximum values are instead noted on the respective graphs. Furthermore, the h-index plot for assistant professors does not show a third quartile, as the median and the third quartile are the same value.
Box and Whisker plot array showing number of publications (first row), number of citations (second row) and h-index (third row) for individual academic staff members organised in columns by academic rank. Note the varying vertical scales for the different academic levels
Figure 2 shows the general productivity increases associated with promotion at the various levels. For example, the average number of publications increases from 4 to 22 (+18) from the assistant to the associate professor level, and from 22 to 49 (+27) from the associate professor to the professor level. The average number of citations increases from 9 to 83 (+74) from the assistant to the associate professor level, and from 83 to 290 (+207) from the associate professor to the professor level. Finally, the average h-index increases from 1 to 3 (+2) from the assistant to the associate professor level, and from 3 to 6 (+3) from the associate professor to the professor level.
Figure 3 shows the scholarly productivity data organised by the schools. These schools have also been grouped by Carnegie classifications, which is indicated by the coloured data series. The data in these figures has been normalised on a ‘per-academic staff member’ basis to negate the influence of the department's size. Finally, averages for the specific Carnegie classifications are shown on the plot. A notable observation in these plots is the clear distinction of research productivity between the Carnegie classifications, especially with respect to the measures of research quality (i.e. number of citations and h-index).
Scholarly productivity organised by schools. Carnegie classifications include research university-high research activity (RU/H), research university-very high research activity (RU/VH) and other-Baccalaureate colleges and special focus institutions
Figure 4 shows the number of academics in each sub-discipline organised by level. The corresponding productivity data is shown in Fig. 5. The raw number of academic staff members (Fig. 4) indicates that mineral processing and rock mechanics are the most popular disciplines across all ranks. This figure also identifies ‘up-and-coming’ topics areas, such as environmental and coal bed methane/natural gas, which are predominantly covered by newer academic staff at the assistant and associate professor levels. Other disciplines, such as ventilation, underground mining, surface mining, geostatistics and mine electrical systems, can be identified as ‘critical needs’ in the US since they have fewer than two assistant professors.
Number of academic staff members sorted by sub-discipline and academic rank
Scholarly productivity organised by expertise area
Additionally, Fig. 5 shows the corresponding academic staff productivity data organised by sub-discipline. These figures provide the average publication values for individual academic staff members classified into the respective topics. As mentioned above, the classifications are representative of the academic staff members, not the individual papers. While an individual academic staff member may publish in several sub-disciplines, the reported values are indicative of the individual's primary interest. Nevertheless, the statistics do indicate a substantial range between the levels of publication and citation for various sub-disciplines. For instance, the average number of papers per academic staff member ranges from a low of eight for the CBM/Natural Gas category to a high of 61 for the mine electrical systems. Many of the remaining areas have similar average publication numbers, with about half being between 20 and 30 publications per academic staff member on average. Alternatively, the citation data show that mineral processing, mine electrical systems and other/non-traditional categories are much more frequently cited than the remaining categories. The lowest citation number for these three categories (325: other/non-traditional) is more than double the next closest category (132: rock mechanics).
Many of these trends may be explained by the levels and the productivity of individuals within the sub-disciplines. For instance, mine electrical systems is composed entirely academics at the professor level; therefore, the greater number of publications and citations is expected, given the requirements to reach that level and the longevity of the individual papers. Alternatively, fields like mineral processing have similar demographics to others, but still show substantially different statistics. This result may indicate that the journals in which these academic staff members publish in receive higher circulation in the broad scientific community.
Despite many of the apparent trends, much of the higher aggregated productivity values are the result of individual academics. Figure 6 shows the productivity data for the top 10 academic staff members in each of the three categories. This data show that for the entire population, a single mineral processing professor claims a large portion of the citations and publications (more than double the next ranking individual in both categories). The remaining nine fall into a closer range. It should be noted, that seven academics have an h-index of 8, creating a tie for the tenth place position. These academics include those with expertise in surface mining (two academics), mineral processing, mine electrical systems, ventilation, rock mechanics and other/non-traditional.
Scholarly productivity for top 10 ranking academic staff members in each category
Figure 7 extends these totals through the entire population as a Pareto graph (percentage of the total publications or citations plotted against the percentage of contributing individuals). The relationship between citations and academic staff members follows a typical Pareto curve, where 20% of the academic staff members account for 80% of the citations. While not considered in this study, a similar analysis between citations and individual papers would likely show a similar relationship (i.e. 20% of one's papers account for 80% of one's citations). The relationship between number of papers and academic staff members shows much less elitism as 20% of the academic staff members only account for 45% of the total documents.
Pareto curve showing cumulative citations and documents plotted against the cumulative percentage of contributing academic staff members
Conclusions
Publications, citations and the h-index are key indicators of scholarly productivity for academics. These parameters are often used in promotion and tenure decisions alongside other indicators of teaching, research and service. To the authors’ knowledge, no prior published study has reported research productivity data for US mining engineering academics. Mining engineering is a relatively small field and these statistics may help provide benchmarks for individual departments as well the colleges and university promotion and tenure committees.
In conclusion, this study has produced five notable findings:
The number of publications and citations increases with increasing level/rank. On average, associate professors have 18 more publications than assistant professors, while full professors have 27 more publications than associate professors. The level of scholarly productivity varies greatly between schools, and the difference is most significant for average citations per academic staff member. Schools in the higher research classification are more likely to have academic staff members that are well-cited. The number and rank of academic staff members identifying with various sub-disciplines also varies greatly. While some areas, such as environmental and CBM/natural gas, are entirely composed assistant and associate professors, others, including mine electrical systems, surface mining and explosives, have few if any assistant professors. Assuming that the higher ranked professors are trending towards retirement, US mining engineering departments should focus on these critical areas. Some mining sub-disciplines, such as mineral processing, mine electrical systems and non-traditional mining, tend to have higher citations per academic staff member and higher citations per paper than others. While this result can be somewhat attributed to the individual academic staff members within these areas, the result may also indicate that some areas are more likely to be cited by the larger scientific community.