Attack of the bots: Lessons from a compromised online MSM survey

Area of Science:

• Public Health
• Epidemiology
• Behavioral Science

Background:

• Anonymous online surveys with financial incentives are crucial for understanding sexual networks, behaviors, and risk factors.
• These surveys are susceptible to bot exploitation, compromising data integrity.
• The rise of advanced AI, including large language models, exacerbates the threat of sophisticated bot attacks.

Purpose of the Study:

• To assess geolocation application use among men who have sex with men (MSM).
• To characterize the role of app usage on sexual behavior and infection risk.
• To highlight the vulnerability of online surveys to bot exploitation and the need for enhanced security.

Main Methods:

• An in-person, limited audience survey was deployed via QR code in North Carolina.
• Descriptive statistics were used to analyze repeat responses, free-text length, and demographic consistency.
• The study unexpectedly went viral on social media, leading to a surge in responses.

Main Results:

• The survey received 4,709 responses between August 2022 and March 2023.
• A sharp spike in responses occurred after the survey went viral, with over 2,000 responses in one day.
• While free-text responses were sophisticated, many multiple-choice answers were inconsistent, indicating potential bot activity.

Conclusions:

• Online survey data quality is threatened by sophisticated bot attacks, particularly with advanced AI.
• Defensive techniques like response time validation, logic checks, and IP screening are essential for protecting research integrity.
• Proactive implementation of security measures is critical for reliable data collection in online research.