比较人类和生成AI的效率和效率:定性主题分析
Maximo R Prescott1,2, Samantha Yeager1, Lillian Ham1,2
1HIV Neurobehavioral Research Program, University of California, San Diego, San Diego, CA, United States.
JMIR AI
|August 2, 2024
概括
生成型人工智能 (GenAI) 在加快数字健康干预措施的定性分析方面表现有希望,但人类监督对于可靠性和细微主题至关重要. 建议采用混合方法,将GenAI和人类编码器结合起来.
科学领域:
- 数字健康数字健康
- 定性研究是指质量研究.
- 人工智能的人工智能
背景情况:
- 定性方法对于传播数字健康干预至关重要,但需要大量的时间.
- 生成型人工智能 (GenAI) 和大型语言模型 (LLM) 可能加速定性数据分析.
- 对GenAI用于定性分析的有效性和可靠性仍未得到充分探索.
研究的目的:
- 评估GenAI (ChatGPT,Bard) 与人类编码者的主题一致性,编码可靠性和时间效率.
- 用GenAI和人类编码器比较诱导和演的主题分析.
- 评估GenAI在数字健康的定性研究中的潜力.
主要方法:
- 对40个短信短信的定性主题分析来自于数字健康干预.
- 由两个独立的人类编码团队进行的感应和演主题分析.
- 由独立分析师用于主题分析的GenAI工具 (ChatGPT,Bard),与人类编码器结果进行比较.
主要成果:
- 在诱导分析中,GenAI的主题与人类识别的主题的一致性为71%.
- 在演分析中,主题一致性较低 (ChatGPT: 50%,Bard: 58%).
- 人类编码器和GenAI之间的编码器间可靠性从相当到中等 (36-47%);GenAI显著更快 (20分钟对567分钟).
结论:
- 基因智能 (GenAI) 证明了降低定性主题分析资源密集性的潜力.
- 基因人工智能与人类之间的编码可靠性较低,需要混合方法.
- 人类编码者在细微主题上表现出色;未来的研究应该专注于协作的人类-GenAI方法,以提高定性研究效率并解决伦理考虑.
相关概念视频
Non-equilibrium in the Cell
4.4K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.4K
Introduction to Cognitive Psychology
455
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
455
Humanistic Psychology
1.1K
Humanistic psychology emerged in the mid-20th century as a response to the deterministic and pessimistic nature of behaviorism and psychoanalysis. While behaviorism focused on observable behaviors influenced by the environment and psychoanalysis delved into unconscious motivations, both theories suggested that human actions lacked free will. In contrast, humanistic psychology offers a perspective that emphasizes the innate potential for goodness and growth within every individual.
This approach...
This approach...
1.1K
Stereotype Content Model
14.0K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.0K
Intelligence
7.3K
The term "intelligence" is complex because it refers to both behavior and individuals, and its interpretation varies across cultures. European Americans tend to link intelligence with reasoning and cognitive skills, while in Kenya, it is tied to responsible participation in family and social life. In Uganda, intelligence is seen as the ability to know the right actions and carry them out effectively, while the Iatmul people of Papua New Guinea associate it with the capacity to remember...
7.3K
Reason and Intuition
6.4K
The human brain processes information for decision-making using one of two routes: an intuitive system and a rational system (Epstein, 1994; popularized by Kahneman, 2011 as System 1 and System 2, respectively). The intuitive system is quick, impulsive, and operates with minimal effort, relying on emotions or habits to provide cues for what to do next, while the rational system is logical, analytical, deliberate, and methodical. Research in neuropsychology suggests that the...
6.4K


