The Impact Non-Factor

    loading  Checking for direct PDF access through Ovid


At the Annual Meeting of the American Society of Cell Biology in San Francisco held in December, 2012, a group of editors and publishers of medical and scientific journals crafted the San Francisco Declaration on Research Assessment (DORA), subsequently published in May, 2013.1 The DORA document was created as a result of the group’s consensus concern that a journal’s Impact Factor does not accurately represent the quality or scientific value of the content of the journal. Also of concern to the group was the increasing trend to citation of review articles or other secondary sources, as opposed to primary research sources, in biomedical publications. The first general recommendation in DORA was: “Do not use journal-based metrics, such as journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.” The document went on to make a further 17 specific recommendations, aimed at funding agencies, institutions, publishers, organizations that supply metrics, and investigators. Since the initial publication of the DORA document, more than 12,700 individuals (including the undersigned authors) and more than 850 organizations have signed on in support of DORA.
In July, 2016, the American Society for Microbiology (ASM) – responsible for 8 different biomedical journals – stated: “Despite widespread condemnation of journal Impact Factors to assess the significance of published work, these numbers continue to be widely misused in publication, hiring, funding, and promotion decisions.”2 The ASM leadership and Editors of ASM journals made the decision to remove reference to the Impact Factor from all ASM journals, and from related informational or promotional materials. They further noted: “The relentless pursuit of high Impact Factor publications has been detrimental for science.”
The May, 2017 issues of the Journal of Clinical Pharmacology and Clinical Pharmacology in Drug Development – two scientific journals owned by the American College of Clinical Pharmacology (ACCP) and published by Wiley – contained simultaneous editorials stating that reference to the Impact Factor would be removed from the journals and their web sites, and from any informational material made available from the ACCP.3
How did we get here?
The late Eugene E. Garfield passed away in February, 2017, at the age of 91. He was a prodigious intellect and a pioneering innovator in the field of citation science. His 1955 article in Science proposed the concepts of a citation index for science, and an Impact Factor for journals.4 The journal Impact Factor was actually anticipated to assist libraries in the selection of journals to which they would subscribe.4,5 Garfield went on to found the Institute for Scientific Information, and was behind the founding of Current Contents, Science Citation Index, Journal Citation Reports, and The Scientist. The current iterations of these enterprises are controlled by a single large media conglomerate.
Calculation of the Impact Factor is deceptively straightforward. For a given journal in a given calendar year, the Impact Factor is the total number of citations to publications in that journal for the two previous years divided by the total number of citable items appearing in that journal during the same two previous years. It is, essentially, an attempt to quantitate the average number of citations per article over the most recent two-year period.
Two major drawbacks are evident, and have been discussed for decades.6–9 First, most journal article citations happen well after the initial two years following publication – the Impact Factor will therefore fail to capture most of an article’s citations. Second, the citation pattern among articles is highly skewed – the majority of total citations can be attributed to a few highly-cited papers.
    loading  Loading Related Articles