在定性研究中对GenAI内大型语言模型的评估
Supriya D Mehta1,2, Souvik Paul3, Enid Awiti4
1Division of Infectious Diseases, Department of Medicine, Rush University College of Medicine, Chicago, USA. Supriyad@uic.edu.
Scientific reports
|October 7, 2025
概括
大型语言模型 (LLM) 显示了定性研究的潜力,但在主题分析准确性方面存在困难. 当前的人工智能工具可以帮助主题识别,但对于严格的定性研究来说尚未可靠.
科学领域:
- 定性研究方法 定性研究方法
- 人工智能在健康研究中的应用
背景情况:
- 大型语言模型 (LLM) 为文本分析和响应生成提供了高级功能.
- 整合LLM的生成AI (GenAI) 为定性研究应用提供了潜在的实用工具.
- COVID-19大流行严重影响了青少年女孩和年轻女性 (AGYW) 的性和生殖健康.
研究的目的:
- 在定性研究中,将GenAI (GPT-4o) 的专题分析与人类分析进行比较.
- 在GenAI和人类编码者之间评估主题识别,引文选择和引文质量.
- 探索GenAI进行的定量和定性情绪分析.
主要方法:
- 关于COVID-19流行病对肯尼亚农村AGYW性和生殖健康的影响的定性研究.
- 在转录输入和预处理后,LLM (GPT-4o) 用于主题和情绪分析.
- 两位研究人员独立地使用标题对GenAI输出与定性研究标准进行了审查.
主要成果:
- 与人类分析相比,GenAI确定了相关的子主题,但有些人没有达到与人类分析相比完整主题的门.
- 在为主题和情绪分析选择支持性报价时,GenAI表现低且可变.
- 基因人工智能产生了幻觉,并确定了与培训数据和文化背景限制有关的偏见.
结论:
- 目前,GenAI (GPT-4o) 无法产生与人类研究人员无法区分的专题分析.
- 基因人工智能可以帮助识别定性数据中的主题,关键词和基本叙述.
- 为了让GenAI可靠地独立进行严格的定性主题分析,需要进一步的进展.
相关概念视频
Qualitative Analysis
1.3K
Qualitative analysis is the process of identifying elements, ions, or compounds in an unknown sample. It is the first and most fundamental type of analysis based on the hierarchy of analytical goals. This hierarchy is significant as it provides a structured approach to scientific research, with qualitative analysis serving as the initial step, providing essential information before moving on to quantitative or other forms of analysis.
There are two main approaches to qualitative analysis:...
There are two main approaches to qualitative analysis:...
1.3K
Qualitative Analysis
23.6K
For solutions containing mixtures of different cations, the identity of each cation can be determined by qualitative analysis. This technique involves a series of selective precipitations with different chemical reagents, each reaction producing a characteristic precipitate for a specific group of cations. Metal ions within a group are further separated by varying the pH, heating the mixture to redissolve a precipitate, or adding other reagents to form complex ions.
For instance, group IV...
For instance, group IV...
23.6K
Language Development
848
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
848
Genetic Lingo
113.8K
Overview
113.8K
Typical Model Studies
619
Fluid mechanics model studies often utilize scaled-down systems to predict fluid behavior in full-scale environments, such as river flows, dam spillways, and structures interacting with open surfaces. Maintaining Froude number similarity in river models is crucial, as it replicates surface flow features like wave patterns and velocities.
619
Language and Cognition
711
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
711

