Abstract
Dear Sir,
During the pre-internet era, libraries used to be very selective in subscribing to research magazines and journals from their limited institutional funds and hence only a few prestigious journals could make it onto their racks. This scenario prompted the need for a system to evaluate the quality of circulating journals in order to help both libraries and readers choose wisely. Historically, the quality and standing of a journal used to depend on the reputation of the society which published it, the members of its editorial board, the circulation size, the frequency of publication, the manuscript processing time and the quality of the peer reviewing process. With the advent of internet, another dimension of the potential for online search and retrieval of an article has also been introduced.
The first citation-based design for rating academic manuscripts was proposed by Eugene Garfield in 1955. 1 This was called the ‘Citation Index’. The term citation index refers to the number of acknowledgements or citations that an already published work (indexed by the Institute of Scientific Information or ISI) receives by other scientists in their research work, which again gets published in an ISI indexed journal. The Citation Index can be used for bibliometric purposes and for assessment of research evaluation in the form of the journal impact factor. The impact factor is a citation-based metric that has become an instrument for the assessment of the quality of journals and it is monitored every year by Thomson Reuters (formerly called ISI). It collects references from indexed journals in its database and calculates the impact factor for a journal by taking the ratio of the number of citations to articles published by that journal during the last two years to the total number of articles it published during that period. For example, this year’s impact factor for a journal can be calculated as:
The numerator in this equation includes original articles, review papers, editorials, commentaries, letters to the editor as well as meeting abstracts whereas the denominator includes the sum of original contributions, review articles and notes.
The impact factor has been widely accepted as an indicator to evaluate the impact and quality of a journal regardless of its size, frequency and content. Primarily intended to rate the journals, the impact factor is also used by the administrative authorities and funding agencies as a shortcut to assess the quality of individual articles and to rank different universities and research institutes.
There is a growing criticism from the research community that the impact factor should not influence decisions pertaining to an individual’s career, such as evaluating their scientific standing, determining their access to research grants and their hiring to prestigious research positions. 2 Some argue that use of algorithms to judge the quality of a scientific work is irrational and unjustified as quantitative methods cannot truly evaluate a qualitative measure such as the standard of research, 3 whereas others point to the intrinsic flaws of the metric that give advantage to some people, languages or areas of research over others. For example, the formula for the IF calculation takes into account only last two years citations and ignores the more lasting impact. This puts articles published in emerging fields with lesser readability at a disadvantage. Similarly, articles published in English receive more citations compared to those in other languages. Basic research journals generally get a higher IF than those of applied research.
There are also other shortcomings that have been exploited by publishers, editors and authors. A high proportion of review articles are added by publishers to improve the impact factor of a journal as reviews are bound to attract more citations than original contributions. Similarly, the articles published in open access journals get higher number of citations than those involving payments. Self-citation and citation on an editor’s or reviewer’s request can be used to improve the impact factor of a journal. Interestingly, even negative citations are treated the same way as positive citations.
As it is very obvious that assessing research quality has no standard solution, so evaluation of merit and impact of individual research needs to be separated from the impact factor ranking of a publishing journal.
‘Not everything that counts can be counted and not everything that can be counted counts’