大型语言模型可以在一个提示符中实现社交媒体集体的感应主题分析:人类验证研究
Michael S Deiner1, Vlad Honcharov2,3, Jiawei Li4
1Department of Ophthalmology and Francis I Proctor Foundation, University of California San Francisco, San Francisco, CA, United States.
JMIR infodemiology
|August 29, 2024
概括
大型语言模型 (LLM) 可以有效地分析社交媒体的健康内容,识别相关主题和合理的主题. 虽然不完全复制人类的深度,LLMs显示公共卫生社会倾听的希望.
科学领域:
- 计算语言学计算语言学
- 公共卫生信息学 公共卫生信息学
- 社交媒体分析.
背景情况:
- 对社交媒体的手动分析提供了公共卫生洞察力,但耗时.
- 生成型大语言模型 (LLM) 为总结和解释大量文本提供了潜力.
- 在识别社交媒体中微妙的健康含义方面,LLM的有效性仍然不清楚.
研究的目的:
- 评估LLM是否能够对社交媒体内容进行主题模型选择和主题分析.
- 为了比较LLM的表现与先前研究的人类主题专家.
- 确定LLM是否可以在社交媒体数据中合理地识别与健康相关的主题.
主要方法:
- 使用三个LLM (GPT4-32K,Claude-instant-100K,Claude-2-100K) 复制了先前研究的研究问题和社交媒体内容.
- 将LLM主题选择和主题识别与人工人体分析进行比较.
- 评估了LLM的一致性和模式间协议.
主要成果:
- 在LLM中,先前人类识别的话题的排名很高,超过了机会水平 (P<.001).
- 专家认为,LLM确定了具有低幻觉率的相关主题,这些主题被专家认为是合理的.
- 在不同LLM和重复分析之间观察到主题识别的变化.
结论:
- 简单的LLM有效地处理大型社交媒体健康数据集,并提取合理的主题.
- 在公共卫生领域,LLM显示了自动化社会倾听的巨大潜力.
- 需要进一步验证以匹配主题提取中人类专家分析的深度.
相关概念视频
Qualitative Analysis
22.1K
For solutions containing mixtures of different cations, the identity of each cation can be determined by qualitative analysis. This technique involves a series of selective precipitations with different chemical reagents, each reaction producing a characteristic precipitate for a specific group of cations. Metal ions within a group are further separated by varying the pH, heating the mixture to redissolve a precipitate, or adding other reagents to form complex ions.
For instance, group IV...
For instance, group IV...
22.1K
Stereotype Content Model
14.0K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.0K
Data Validation
5.0K
Data validation is an essential part of a comprehensive assessment. Validation is confirming or verifying and opening the door to gathering more assessment data as it clarifies vague or unclear data. The process of checking and verifying the collected information is called data validation. The primary purpose of data validation is to ensure data is as free from error, bias, and misinterpretation as possible.
Nursing assessment guides are generally based on holistic models rather than medical...
Nursing assessment guides are generally based on holistic models rather than medical...
5.0K
Longitudinal Research
11.9K
Sometimes we want to see how people change over time, as in studies of human development and lifespan. When we test the same group of individuals repeatedly over an extended period of time, we are conducting longitudinal research. Longitudinal research is a research design in which data-gathering is administered repeatedly over an extended period of time. For example, we may survey a group of individuals about their dietary habits at age 20, retest them a decade later at age 30, and then again...
11.9K
Higher Mental Functions of the Brain: Language
774
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
774
Aggregates Classification
309
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
309


