弗兰肯斯坦,主题分析和生成人工智能:质量评估方法和定性研究的考虑
Tanisha Jowsey1, Peta Stapleton2, Shawna Campbell2
1Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Australia.
PloS one
|September 5, 2025
概括
像微软Copilot这样的生成人工智能 (GenAI) 工具在定性研究中尚未可靠进行主题分析. 不同于人类分析, 目前的GenAI版本是伪造的.
科学领域:
- 定性研究方法
- 研究中的人工智能
- 健康科学数据分析
背景情况:
- 越来越多的人工智能 (GenAI) 需要对定性数据分析进行验证.
- 对研究人员来说,评估GenAI对主题分析的可靠性和适用性至关重要.
研究的目的:
- 确定使用GenAI进行主题分析的准确性和效率.
- 提出一种方法来评估GenAI在定性研究中的可靠性.
主要方法:
- 搜索了与健康相关的研究数据库和期刊,并发表了主题分析和数据集.
- 提示微软Copilot (一个封闭的系统GenAI) 对选定的数据集进行主题分析.
- 将GenAI的结果与人类分析进行比较,测量时间,准确性,主题重叠和报价可靠性.
主要成果:
- 人类识别的主题与Copilot的主题之间存在最小的重叠;Copilot经常编造引文 (58%) 并专注于有限的数据部分.
- 人类分析显示了更广泛的代表性和更高的引用准确性 (79%).
- 与人类研究人员不同的是,Copilot没有按主题分发参与者.
结论:
- 目前的GenAI版本,如Copilot,由于不准确和不可靠,不建议进行主题分析.
- 这项研究强调了人类生成和GenAI生成的定性数据分析和报告的有效性.
更多相关视频
相关概念视频
Qualitative Analysis
674
Qualitative analysis is the process of identifying elements, ions, or compounds in an unknown sample. It is the first and most fundamental type of analysis based on the hierarchy of analytical goals. This hierarchy is significant as it provides a structured approach to scientific research, with qualitative analysis serving as the initial step, providing essential information before moving on to quantitative or other forms of analysis.
There are two main approaches to qualitative analysis:...
There are two main approaches to qualitative analysis:...
674
Non-equilibrium in the Cell
4.8K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.8K
Quality Assurance
196
Quality assurance is the overarching term used to describe the activities employed to ensure the proper performance of a system. These activities can be classified into three categories: quality control, quality assessment, and internal corrective measures. Typically, these activities work cyclically: quality control is performed before and during the analysis, while quality assessment occurs during and after the investigation. Internal corrective measures are implemented based on the findings...
196
Stereotype Content Model
14.9K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.9K
Intelligence
8.0K
The term "intelligence" is complex because it refers to both behavior and individuals, and its interpretation varies across cultures. European Americans tend to link intelligence with reasoning and cognitive skills, while in Kenya, it is tied to responsible participation in family and social life. In Uganda, intelligence is seen as the ability to know the right actions and carry them out effectively, while the Iatmul people of Papua New Guinea associate it with the capacity to remember...
8.0K
Lazarus's Cognitive Appraisal Theory
501
Cognitive psychologist Richard Lazarus proposed the cognitive-mediational theory of emotions, which emphasizes how individuals' assessments of stressors significantly affect their experience of stress. According to Lazarus, the stress response is determined by a two-step appraisal process: primary appraisal and secondary appraisal. These cognitive appraisals help individuals evaluate the potential impact of a stressor and determine the adequacy of their coping resources.
Primary Appraisal:...
Primary Appraisal:...
501


