Leveraging web-scraped data to examine alcohol pricing: an Australian feasibility study with retail data

Area of Science:

• Public Health
• Data Science
• Alcohol Policy Research

Background:

• Comprehensive alcohol price data is vital for understanding purchasing behavior and harms, but is often inaccessible to researchers.
• Web scraping offers an automated solution for collecting extensive website data efficiently.
• This study addresses the need for reliable alcohol price data to inform public health policies.

Purpose of the Study:

• To demonstrate the utility of web scraping for generating alcohol policy-relevant data.
• To assess the consistency and reliability of web scraping for alcohol price data collection.
• To compare data from a commercial provider with a university-developed scraper.

Main Methods:

• Daily scraping of online alcohol catalogues from major Australian retailers (2020 onwards).
• Data collection across all Australian jurisdictions, matching products sold by multiple retailers.
• A university-developed scraper collected single-day data from the largest retailer for cross-validation.

Main Results:

• Excellent agreement between commercial and university datasets for 16,409 matched products (ICC=0.997).
• Visualizations revealed significant daily price variations over 12 months for major retailers.
• Jurisdictional differences in pricing, including deeper discounting in Victoria, were observed.

Conclusions:

• Web scraping provides a feasible, large-scale, and longitudinal method for collecting alcohol price data.
• The validated data enhances understanding of the alcohol retail market dynamics.
• This approach supports the development of evidence-based alcohol price policies.