Australasian Institute of Digital Health Summit 2022–Automated Social Media Surveillance for Detection of Vaccine Safety Signals: A Validation Study

Abstract

Abstract Background Social media platforms have emerged as a valuable data source for public health research and surveillance. Monitoring of social media and user-generated data on the Web enables timely and inexpensive collection of information, overcoming time lag and cost of traditional health reporting systems. Objectives This article identifies personally experienced coronavirus disease 2019 (COVID-19) vaccine reactions expressed on Twitter and validate the findings against an established vaccine reactions reporting system. Methods We collected around 3 million tweets from 1.4 million users between February 1, 2021, to January 31, 2022, using COVID-19 vaccines and vaccine reactions keyword lists. We performed topic modeling on a sample of the data and applied a modified F1 scoring technique to identify a topic that best differentiated vaccine-related personal health mentions. We then manually annotated 4,000 of the records from this topic, which were used to train a transformer-based classifier to identify likely personally experienced vaccine reactions. Applying the trained classifier to the entire data set allowed us to select records we could use to quantify potential vaccine side effects. Adverse events following immunization (AEFI) referred to in these records were compared with those reported to the state of Victoria's spontaneous vaccine safety surveillance system, SAEFVIC (Surveillance of Adverse Events Following Vaccination In the Community). Results The most frequently mentioned potential vaccine reactions generally aligned with SAEFVIC data. Notable exceptions were increased Twitter reporting of bleeding-related AEFI and allergic reactions, and more frequent SAEFVIC reporting of cardiac AEFI. Conclusion Social media conversations are a potentially valuable supplementary data source for detecting vaccine adverse event mentions. Monitoring of online observations about new vaccine-related personal health experiences has the capacity to provide early warnings about emerging vaccine safety issues.