The impact factor of a journal is a poor measure of the clinical relevance of its papers

Abstract

We evaluated the top 13 journals in trauma and orthopaedics by impact factor and looked at the longer-term effect regarding citations of their papers. All 4951 papers published in these journals during 2007 and 2008 were reviewed and categorised by their type, subspecialty and super-specialty. All citations indexed through Google Scholar were reviewed to establish the rate of citation per paper at two, four and five years post-publication. The top five journals published a total of 1986 papers. Only three (0.15%) were on operative orthopaedic surgery and none were on trauma. Most (n = 1084, 54.5%) were about experimental basic science. Surgical papers had a lower rate of citation (2.18) at two years than basic science or clinical medical papers (4.68). However, by four years the rates were similar (26.57 for surgery, 30.35 for basic science/medical), which suggests that there is a considerable time lag before clinical surgical research has an impact. We conclude that high impact journals do not address clinical research in surgery and when they do, there is a delay before such papers are cited. We suggest that a rate of citation at five years post-publication might be a more appropriate indicator of importance for papers in our specialty. Cite this article: Bone Joint J 2014;96-B:414–19.