Citation versus disruption in the military: Analysis of the top disruptive military trauma research publications

Abstract

BACKGROUND Bibliometric analysis of surgical research has become increasingly prevalent. Citation count (CC) is a commonly used marker of research quality, but may overlook impactful military research. The disruption score (DS) evaluates manuscripts on a spectrum from most innovative with more positive scores (disruptive [DR]) to most entrenched with more negative scores (developmental; DV). We sought to analyze the most DR and DV versus most cited research in military trauma. METHODS Top trauma articles by DS and by CC were identified via professional literature search. All publications in military journals were included. Military trauma-related keywords were used to query additional top surgical journals for military-focused publications. Publications were linked to the iCite NIH tool for CC and related metrics. The top 100 DR and DV publications by DS were analyzed and compared with the top 100 articles by CC. RESULTS Overall, 32,040 articles published between 1954 and 2014 were identified. The average DS and CC were 0.01 and 22, respectively. Most articles were published in Mil Med (68%). The top 100 DR articles were frequently published in Mil Med (51%) with a mean DS of 0.148. Of these, the most cited article was only the 40th most disruptive. The top 100 CC articles averaged a DS of 0.009 and were commonly found in J Trauma (53%). Only five publications were on both the top 100 DR and top 100 CC lists; 19 were on both the top DV and CC lists. Citation count was not correlated with DR (r = −0.134; p = 0.07) and only weakly correlated with DV (r = 0.215; p = 0.003). CONCLUSION DS identifies publications that changed military paradigms and future research directions previously overlooked by citation count alone. The DR and DV articles are distinct with little overlap between highly cited military articles. Multiple bibliometric measures should be employed to avoid overlooking impactful military trauma research. LEVEL OF EVIDENCE Diagnostic Test or Criteria; Level IV.