在日本外科专家检查中多式大型语言模型的性能
Yuji Miyamoto1,2, Takeshi Nakaura3, Hiro Nakamura4
1Department of Gastroenterological Surgery, Graduate School of Medical Sciences, Kumamoto University, Kumamoto, 1-1-1 Honjo, 860-8556, Japan. miyamotoyuji@kumamoto-u.ac.jp.
BMC medical education
|October 10, 2025
概括
克劳德3.5 索内特在外科考试中表现出色,在基于图像的问题上表现出色. 这凸显了多式大型语言模型 (LLM) 作为外科教育和诊断中的有价值工具的潜力.
科学领域:
- 人工智能在医学中的应用
- 医疗教育 技术 技术 医学教育
- 手术培训和评估的手术培训和评估
背景情况:
- 多模式大语言模型 (LLM) 整合了文本和图像数据,显示了医疗应用的潜力.
- 在标准化医疗检查中评估LLM的表现对于评估其实用性至关重要.
- 2023年日本外科专家考试是评估外科人工智能能力的基准.
研究的目的:
- 评估领先的多式联络LLM在2023年日本外科专家考试中的表现.
- 具体评估基于图像的问题对跨外科子专业的LLM准确性的影响.
- 在高风险的医疗环境中比较GPT-4 Omni,Claude 3.5 Sonnet和Gemini Pro 1.5的功能.
主要方法:
- 三个多模式的LLM (GPT-4 Omni,Claude 3.5 Sonnet,Gemini Pro 1.5) 在2023年考试中的98个问题 (43个基于图像) 上进行了测试.
- 模型的性能在有图像数据和没有图像数据的情况下进行了评估.
- 麦克纳马的测试被用于对精度差异的统计分析.
主要成果:
- 克劳德3.5索内特获得了最高的整体准确度 (84.69%),超过了80%的合格门.
- GPT-4 Omni (79.59%) 和Gemini Pro 1.5 (61.22%) 的得分都低于通过值.
- 克劳德3.5索内特显示了图像包含的统计学上显著的改善 (p=0.041),与其他模型不同.
结论:
- 克劳德3.5索内特表现出卓越的表现,特别是在基于图像的外科问题上,超过了通过分数.
- 这项研究表明,多式联络LLM,特别是克劳德3.5索内特,在外科手术中显示出有前途的诊断和教育辅助工具.
- 视觉推理能力的差异可能解释了LLM在基于图像的医疗任务上的不同表现.
相关概念视频
Improving Translational Accuracy
14.0K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.0K
Improving Translational Accuracy
3.5K
3.5K
Higher Mental Functions of the Brain: Language
3.3K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
3.3K
Language and Cognition
676
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
676
Components of Language
714
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
714
Language
854
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
854


