Validity of Research Based on Publicly Obtained Data in Sports Medicine: A Quantitative Assessment of Concussions in the National Football League

Abstract

Context: Numerous researchers have leveraged publicly available internet sources to publish clinical research concerning incidence and recovery from injuries in National Football League (NFL) players. Objective: This study aims to (1) provide a comprehensive systematic review of all publicly obtained data studies (PODS) regarding concussions in NFL athletes and (2) quantify the percentage of injuries identified by these studies in comparison with published concussion data from the NFL injury database. Study Selection: A systematic review was conducted in accordance with PRISMA guidelines to identify all published studies utilizing publicly obtained data regarding concussions in NFL athletes. Study Design: Systematic review. Level of Evidence: Level 4. Data Extraction: Manuscript details, factors related to the athletes of interest (eg, study period, positions included), and results (eg, concussion rate, number of total concussions, return-to-play data) were extracted independently by 2 authors. Results were compared with incident concussions reported from 2015 to 2019 by each medical staff member to the NFL database linked to the League’s electronic health record (EHR). Results: A total of 20 concussion-focused manuscripts based on PODS were identified from 2014 to 2020. PODS captured between 20% and 90% of concussions (mean, 70%) reported by medical staff to the injury database. PODS reported that 55% of concussions occurred on offensive plays, 45% on defensive plays and <1% occurred during special teams plays, compared with 44%, 37%, and 18%, respectively, as indicated by published data from the NFL injury database. When analyzed by position groups, running backs and quarterbacks comprised the most over-represented positions concussed in PODS, while offensive linemen, defensive backs, and linebackers comprised the most under-represented positions. Conclusion: PODS captured approximately 70% of concussions reported by NFL medical staff to the NFL injury database. There is heterogeneity in the degree to which PODS were able to identify concussions, with a bias toward concussions among players at higher profile positions.