Generating Contextual Variables From Web-Based Data for Health Research: Tutorial on Web Scraping, Text Mining, and...

Area of Science:

• Health Services Research
• Geospatial Health
• Computational Social Science

Background:

• Contextual variables for geographic areas are crucial for health and social research.
• Acquiring contextual data can be difficult due to a lack of monitoring systems or census data.

Purpose of the Study:

• To describe and implement an 8-step method (WeTMS) for transforming web text data into analyzable contextual datasets.
• To create datasets of health assets aimed at enhancing social connections for older adults in Catalonia.

Main Methods:

• The WeTMS method integrates web scraping, text mining, and spatial overlay analysis.
• Python and R were used for data extraction, text preprocessing, classification, and spatial analysis.
• The method was applied to identify health assets across 374 health jurisdictions from 2015-2022.

Main Results:

• Over 17,000 websites were analyzed, yielding 9,546 health assets (5,022 activities, 4,524 resources).
• Leisure and skill development activities, and leisure/cultural associations were the most prevalent asset types.
• Asset registration varied significantly across jurisdictions, with an agreement rate between 62.02% and 99.47%.

Conclusions:

• The WeTMS method provides a robust approach for generating contextual variables from internet text data.
• This methodology can assist health and social researchers in efficiently creating ready-to-analyze datasets.