COMMENTS

Does size matter? Considerations on the use of the impact factor

http://dx.doi.org/10.5546/aap.2013.6

When comparing scientific publications, we should ponder on how important it is for us to be able to compete, to play the game, to be able to be ranked and evaluated, condition we have accessed to thank to the increasingly renowned position of our Archivos Argentinos de Pediatría.
Since its creation and in the quest for being chosen by readers¹ our journal has undergone many changes. Particularly over the last ten years, Archivos Argentinos de Pediatría has celebrated its diamond wedding and has consolidated its frequency of publication, has made all of its contents available online for free, has been included in outstanding index databases (Scielo and Pubmed), and has recently started offering its contents in English.²
All this has led to an increase in the journal visualization; and in the last two years, Archivos Argentinos de Pediatría has been included in the more than 8200 journals considered by the Journal Citation Reports (JCR), together with its impact factor (IF) evaluation.³ After only two years of being evaluated, Archivos Argentinos de Pediatría is already in the third quartile of pediatric publications and reaching the top half of the list, and it would be reasonable to believe that the English version, together with the growing international collaborations, will rank us in a remarkably better position in the next evaluation.
At this stage, it would be advisable to reflect on the meaning and significance of the IF.
What is the Impact Factor? It is a measure of the importance of a journal initially developed for librarians to decide which regular publications had to be acquired for their library.⁴
How is the Impact Factor calculated? A journal's IF for a specific year is the result of dividing the number of citations received that year by the articles published in the two previous years in the same journal by the number of articles published that year in the same journal. For example, the IF of a journal in 2012 will be the number of citations received in 2012 for the articles published in that journal during 2010 and 2011.
Who estimates the Impact Factor? The calculation is made by Thomson-Reuters, a private company specialized in information services.
What is the current use of the Impact Factor? The IF has appeared in the Journal Citation Reports (JCR) for more than 35 years, and it has been extensively used to rank and evaluate not only scientific journals, but also articles and their authors.
As mentioned before, the IF was developed as a guideline for those in charge of selecting which publications to acquire and, in time, it was more and more used to make a value judgment of articles and, almost transitively, of their authors.⁵ Several national scientific research support systems use the IF as one of the items to be taken into account for the ranking of their researchers. Even social networks for researchers use the IF of journals where participants' results are reported to categorize them.⁶
However, this index has some weaknesses that should also be addressed. The calculation is based on the number of citations received by "citable" items in a journal (total citations/citable items). While this calculation includes all citations made to a journal articles (the numerator), the classification of an item as "citable" or not (the denominator) depends on the individuals in charge of the calculation process, which is not publicly available. In addition, it is usually believed that the IF represents the average number of citations to an article, when the distribution of citations per article is not normal. Finally, the IF might be manipulated by journal editors who could adopt some specific policies to boost their impact factor.⁷
However, the most serious problem is that the IF is sometimes used as a proxy for the quality of the article and, even worse, for the quality of the researcher.⁵ It is true that journals with the highest IF are usually viewed as the most prestigious ones⁸ and tend to receive the best contributions from the most outstanding researchers, thus creating the virtuous cycle that has been in place for a long time. Nevertheless, the contribution of information technology to the dissemination of scientific contents is changing this reality and, since the 1990s, the number of cited articles appearing in journals with a high IF is getting smaller and smaller.⁹ And this is very reasonable, since researchers may currently access numerous citable items online, with no need to search for them in the most prestigious journals.
In addition, in many cases it is possible to use information technology tools to evaluate the number of citations made to each article, which is a more accurate indicator of its scientific value than the very fact of having been published in a prestigious journal.
Finally, we should provide an answer to the question posed in the title: Does the IF matter? Even with its limitations and weaknesses, the IF is still a simple and powerful tool for the global evaluation of the scientific quality of a journal. We should not ask of it more than what it can offer, and we should be attentive to other sources for bibliometric measurement.¹⁰
Of course, we should always bear in mind the importance of thoroughly reading each article and categorizing and ranking it according to its scientific content, instead of doing so by taking into account the journal, the researcher or the institution involved. When it comes to science, only facts matter.

Fernando Ferrero

Assistant Editor

1. Otero P. El desafío de las publicaciones científicas... cómo seguir siendo elegidas por los lectores. Arch Argent Pediatr 2012;110(5):370-371.         [ Links ]

2. Ceriani Cernadas JM. El comienzo de un trascendente cambio en Archivos Argentinos de Pediatría. Arch Argent Pediatr 2012; 110(4):283-4.         [ Links ]

3. Journal Citation Reports. Available at: http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports/. Accessed on 11/20/2012.         [ Links ]

4. Garfield E. The history and meaning of the journal impact factor. JAMA 2006; 295(1):90-3.         [ Links ]

5. Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ 1997; 314(7079):498- 502.         [ Links ]

6. ResearchGate. Available at: https://www.researchgate.net/aboutus.AboutUs.html. Accessed on 11/20/2012.         [ Links ]

7. Campanario JM. El factor de impacto de las revistas académicas: preguntas y respuestas. Available at: http://www.uah.es/otrosweb/jmc. Accessed on 11/20/2012.         [ Links ]

8. Saha S, Saint S, Christakis DA. Impact factor: a valid measure of journal quality? J MedLibrAssoc 2003; 91(1):42-6.         [ Links ]

9. Lozano G, Lariviere V, Gingras Y. The weakening relationship between the Impact Factor and papers' citations in the digital age. Available at:http://arxiv.org/abs/1205.4328. Accessed on 11/20/2012.         [ Links ]

10. Falagas ME, Kouranos VD, Arencibia-Jorge R, Karageorgopoulos DE. Comparison of SCImago journal rank indicator with journal impact factor. FASEB J 2008;22(8):2623-8.         [ Links ]