基于大型语言模型的聊天机器人在口腔和大面部放射学中表现如何?
Hui Jeong1, Sang-Sun Han1,2,3, Youngjae Yu4
1Department of Oral and Maxillofacial Radiology, Yonsei University College of Dentistry, Seoul 03722, Republic of Korea.
Dento maxillo facial radiology
|June 7, 2024
概括
四个基于大型语言模型 (LLM) 的聊天机器人在口腔和面放射学考试中进行了测试. 聊天机器人的表现不满意,表明需要进一步的专业培训和响应验证.
科学领域:
- 牙科 牙科是指牙科的专业.
- 人工智能的人工智能
- 医学教育 医学教育
背景情况:
- 大型语言模型 (LLM) 越来越多地融入到各种专业领域.
- 评估LLM在口腔和面放射学等专业医疗领域的有效性至关重要.
研究的目的:
- 评估四个基于LLM的聊天机器人的表现,与牙科学生进行口腔和大面部放射学检查.
- 根据教育内容领域和问题类型分析聊天机器人的准确性.
主要方法:
- 四个LLM (ChatGPT,ChatGPT Plus,Bard,Bing Chat) 在52个牙科大学考试问题中进行了测试.
- 问题涉及基本知识,成像/设备和图像解释,包括多选题 (MCQ) 和短答题 (SAQ).
- 聊天机器人的准确性与牙科学生的表现进行了比较.
主要成果:
- 学生的整体准确率为81.2%. 聊天机器人的准确度在50.0% (ChatGPT,Bard) 到65.4% (ChatGPT Plus) 之间. 聊天机器人的准确度在50.0% (ChatGPT,Bard) 到65.4% (ChatGPT Plus) 之间.
- 在基本知识方面,ChatGPT Plus的成绩优于学生 (93.8%对比78.7%).
- 所有聊天机器人都在图像解释 (准确率<35.0%) 和MCQ (准确率<60.0%) 中遇到了困难.
结论:
- 目前的口腔和口面放射学LLM成绩不足以适用于专业应用.
- 法律法规需要进一步培训,使用特定领域的可靠数据来提高准确性.
- 在临床或教育使用之前,严格验证LLM产生的反应是必不可少的.
相关概念视频
Nonconscious Mimicry
3.6K
Nonconscious mimicry occurs when individuals alter their mannerisms to match the behaviors and expressions of those nearby, without intention.
3.6K
Stereotype Content Model
13.1K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
13.1K
Facial Feedback Hypothesis
953
Charles Darwin proposed that facial expressions are an evolutionary adaptation for communication. He argued that these expressions are not influenced by culture but are universal across species. For example, a snarling expression with exposed teeth signals a threat in many animals, including humans. Darwin also suggested that displaying an emotion can intensify the feeling. Smiling, for example, could enhance one's sense of happiness. This idea laid the foundation for understanding the role...
953


