评估大型语言模型用于从社交媒体发布的疫苗帖子上的情感分析和犹分析:定性研究
Augustine Annan1, Amanda L Eiden2, Dong Wang2
1IMO Health, Rosemont, IL, United States.
JMIR formative research
|October 15, 2025
概括
大型语言模型 (LLM) 对分析社交媒体上的疫苗情绪和犹有前途. GPT-4表现出卓越的准确性,零射击学习是疫苗话语分析中最有效的方法.
科学领域:
- 计算语言学 计算语言学
- 公共卫生信息学 公共卫生信息学
- 医疗保健中的人工智能
背景情况:
- 社交媒体是公共卫生话语的关键平台,包括疫苗讨论.
- 准确的情绪分析和犹的检测对于了解公众论至关重要.
- 大型语言模型 (LLM) 为分析复杂的健康相关文本数据提供了先进的功能.
研究的目的:
- 评估和比较各种LLM在情绪分析和疫苗犹检测方面的表现.
- 确定最有效,最准确,最具成本效益的LLM来分析与疫苗相关的公众情绪.
- 评估不同学习范式 (零射击,一射击,少数射击) 的LLM绩效.
主要方法:
- 利用GPT-3.5,GPT-4,Claude-3 Sonnet和Llama 2来分析来自X,Reddit和YouTube的疫苗讨论.
- 在零射击,一射击和少数射击学习范式中测试了模型.
- 通过使用准确度,F1分数,精度和回忆来评估性能,并根据代币使用进行成本分析.
主要成果:
- GPT-4获得了最高的性能 (F1得分=0.85,准确度=0.83),超过了其他LLMs.
- 零射击学习被发现与少射击学习一样有效,计算成本更低.
- 在准确分类中性情绪,刺和间接表达犹方面仍然存在挑战.
结论:
- GPT-4是疫苗情绪和犹分析最准确的LLM,零射击学习提供了性能和效率的良好平衡.
- 将LLM与传统机器学习模型相结合的混合方法可以优化成本和性能.
- 在公共卫生传播策略中,LLM具有显著的潜力,但需要进行持续的评估和改进.
相关概念视频
Stereotype Content Model
15.3K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
15.3K
Vaccinations
51.2K
Overview
51.2K
Attitudes
32.7K
Attitude is our evaluation of a person, an idea, or an object. We have attitudes for many things ranging from products that we might pick up in the supermarket to people around the world to political policies. Typically, attitudes are favorable or unfavorable: positive or negative (Eagly & Chaiken, 1993). And, they have three components: an affective component (feelings), a behavioral component (the effect of the attitude on behavior), and a cognitive component (belief and knowledge;...
32.7K
Social Proof
31.5K
Social proof is a form of persuasion based on comparison and conformity. People compare their behavior and actions to what others are doing and will change to conform to do what their peers do.
31.5K
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
406
Biopharmaceutical studies constitute a vital field aiming to enhance drug delivery methods and refine therapeutic approaches, drawing upon diverse interdisciplinary knowledge. In research methodologies, the choice between controlled and non-controlled studies significantly influences the study's reliability and accuracy.
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
406
Cancer Vaccines
960
Cancer treatment vaccines are a rapidly evolving field that offers a promising approach to immunotherapy. Unlike traditional vaccines that prevent diseases, cancer treatment vaccines are designed to treat existing cancers by stimulating the immune system to recognize and attack cancer cells.
Cancer vaccines come in two categories: preventive (prophylactic) and treatment (active). Preventive vaccines, such as the Human Papillomavirus (HPV) vaccine, protect against viruses that cause certain...
Cancer vaccines come in two categories: preventive (prophylactic) and treatment (active). Preventive vaccines, such as the Human Papillomavirus (HPV) vaccine, protect against viruses that cause certain...
960

