Intervention in Health Misinformation Using Large Language Models for Automated Detection, Thematic Analysis, and...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Natural Language Processing
• Public Health Communication

Background:

• Social media facilitates the rapid spread of health misinformation, impacting public health by causing confusion and eroding trust.
• Misinformation on social media leads to noncompliance with health guidelines and risky health behaviors.
• Understanding misinformation dynamics is crucial for effective public health strategies.

Purpose of the Study:

• To develop an automated approach using LLMs and machine learning to detect health misinformation on social media.
• To uncover the underlying causes and themes of health misinformation.
• To generate refutation arguments to control misinformation spread and inoculate the public.

Main Methods:

• Trained three LLMs (BERT, T5, GPT-2) to classify documents as misinformation or nonmisinformation.
• Employed topic modeling algorithms (LDA, Top2Vec, BERTopic) to identify misinformation topics and themes.
• Utilized prompt engineering to extract sentence-level topic representations and generate misinformation themes.

Main Results:

• The BERT model achieved 98% accuracy in misinformation classification with reduced false positives.
• BERTopic was the optimal topic modeling approach, showing strong performance metrics.
• A novel prompt engineering method achieved 99.6% appropriateness for generating topic representations and 82% accuracy for detecting misinformation themes.

Conclusions:

• A comprehensive, automated system using LLMs and prompt engineering effectively detects health misinformation and identifies themes.
• The system generates explanatory responses to combat misinformation spread on social media.
• The approach, tested on a COVID-19 dataset, shows promise for improving public health communication, though real-world evaluation is pending.