Validity of Research Based on Public Data in Sports Medicine: A Quantitative Assessment of Anterior Cruciate Ligament Injuries in the National Football League

Abstract

Background: Numerous researchers have leveraged publicly available Internet sources to publish publicly obtained data (POD) studies concerning various orthopaedic injuries in National Football League (NFL) players. Purpose: To provide a comprehensive systematic review of all POD studies regarding musculoskeletal injuries in NFL athletes and to use anterior cruciate ligament (ACL) injuries in NFL players to quantify the percentage of injuries identified by these studies. Study Design: Systematic review; Level of evidence, 4. Methods: A systematic review was conducted to identify all published studies utilizing POD regarding ACL injury in NFL athletes from 2000 to 2019. Data regarding player demographics were extracted from each publication. These results were compared with prospectively collected data reported by the teams’ medical staff to the NFL Injury Surveillance System database linked to the League’s electronic health record. An ACL “capture rate” for each article was calculated by dividing the number of ACL injuries in the POD study by the total number of ACL injuries in the NFL injury database occurring in the study period of interest. Results: A total of 42 studies were extracted that met the definition of a POD study: 28 evaluated a variety of injuries and 14 dealt specifically with ACL injuries, with 35 (83%) of the 42 studies published during or since 2015. POD studies captured a mean of 66% (range, 31%-90%) of ACL injuries reported by the teams’ medical staff. This inability to capture all injury rates varied by position, with 86% capture of ACL injuries in skill athletes, 72% in midskill athletes, and 61% in linemen. POD studies captured 35% of injuries occurring during special teams play. Conclusion: The frequency of studies leveraging publicly obtained injury data in NFL players has rapidly increased since 2000. There is significant heterogeneity in the degree to which POD studies correctly identify ACL injuries from public reports. Sports medicine research relying solely on publicly obtained sources should be interpreted with an understanding of their inherent limitations and biases. These studies underreport the true incidence of injuries, with a bias toward capturing injuries in more popular players.