Automatic detection of e-cigarette screens using object detection and vision language models

The e-cigarette market evolves faster than regulation and enforcement. In recent years, e-cigarette manufacturers have introduced vapes featuring not just addictive, flavored nicotine, but also technological features that appeal to youth. These products with digital screens have the potential to gamify the use of an addictive drug. To keep pace with these developments, automated tools capable of rapidly analyzing the online marketplace could inform regulatory enforcement.

The goal of this work is to automatically detect and classify e-cigarette devices with screens. To do this, we first use an object detection model to filter web-scraped images that contain e-cigarettes. Product images and descriptions are then provided to a vision language model (VLM) to determine if the e-cigarette contains a screen. We evaluated the pipeline with images and text descriptions from five e-cigarette online retailers and manually validated predictions.

The object detection model was trained on approximately 7000 images and tested on 3920 images achieving an accuracy, precision, recall, and F1 score of 0.95, 0.98, 0.94, 0.96 respectively in identifying e-cigarette devices. The VLM was tested on 2401 images and was able to detect screens on e-cigarette devices with an accuracy of 0.92, precision of 0.96, recall of 0.87, and F1 score of 0.92.

Object detection models and VLMs can be combined to automatically detect and classify e-cigarettes with integrated screens. This method can enable rapid-response pipelines to identify youth-appealing features in emerging e-cigarette products and provide policymakers insights into their proliferation in the online marketplace.

New e-cigarettes with potentially youth-appealing technological features, such as integrated screens, continue to be introduced to the US market, challenging timely and effective tobacco product surveillance and regulatory efforts. Digital screens on e-cigarettes offer greater user control and make products that deliver addictive, flavored nicotine resemble entertainment devices, rather than tobacco products. Our automated screen detection pipeline provides a scalable, rapid-response tool for monitoring online e-cigarette retailers in near real-time, quantifying the proliferation of products with specific features. This pipeline could be adapted to accommodate a variety of new features for detection, here we use the presence of screens as a test example. Rapid identification and classification of new tobacco products available online could help inform interventions before the sale and use of such products becomes widespread.