Scalable Surveillance of E-Cigarette Products on Instagram and TikTok Using Computer Vision

Abstract

Abstract Introduction Instagram and TikTok, video-based social media platforms popular among adolescents, contain tobacco-related content despite the platforms’ policies prohibiting substance-related posts. Prior research identified themes in e-cigarette-related social media posts using qualitative or text-based machine learning methods. We developed an image-based computer vision model to identify e-cigarette products in social media images and videos. Aims and Methods We created a data set of 6999 Instagram images labeled for 8 object classes: mod or pod devices, e-juice containers, packaging boxes, nicotine warning labels, e-juice flavors, e-cigarette brand names, and smoke clouds. We trained a DyHead object detection model using a Swin-Large backbone, evaluated the model’s performance on 20 Instagram and TikTok videos, and applied the model to 14 072 e-cigarette-related promotional TikTok videos (2019–2022; 10 276 485 frames). Results The model achieved the following mean average precision scores on the image test set: e-juice container: 0.89; pod device: 0.67; mod device: 0.54; packaging box: 0.84; nicotine warning label: 0.86; e-cigarette brand name: 0.71; e-juice flavor name: 0.89; and smoke cloud: 0.46. The prevalence of pod devices in promotional TikTok videos increased by 15% from 2019 to 2022. The prevalence of e-juices increased by 33% from 2021 to 2022. The prevalence of e-juice flavor names and e-cigarette brand names increased by about 100% from 2019 to 2022. Conclusions Deep learning-based object detection technology enables automated analysis of visual posts on social media. Our computer vision model can detect the presence of e-cigarettes products in images and videos, providing valuable surveillance data for tobacco regulatory science (TRS). Implications Prior research identified themes in e-cigarette-related social media posts using qualitative or text-based machine learning methods. We developed an image-based computer vision model to identify e-cigarette products in social media images and videos. We trained a DyHead object detection model using a Swin-Large backbone, evaluated the model’s performance on 20 Instagram and TikTok videos featuring at least two e-cigarette objects, and applied the model to 14 072 e-cigarette-related promotional TikTok videos (2019–2022; 10 276 485 frames). The deep learning model can be used for automated, scalable surveillance of image- and video-based e-cigarette-related promotional content on social media, providing valuable data for TRS. Social media platforms could use computer vision to identify tobacco-related imagery and remove it promptly, which could reduce adolescents’ exposure to tobacco content online.