泌尿病学中的聊天机器人:准确性,校准性和可理解性;DeepSeek正在接管王位吗?
Omer Faruk Asker1, Muhammed Selim Recai1, Yunus Emre Genc2
1School of Medicine, Marmara University, Istanbul, Turkey.
BJU international
|July 31, 2025
概括
这项研究使用泌尿病学考试问题评估了大型语言模型 (LLM). 聊天GPT-4o和DeepSeek-R1显示了高准确度和低误差,表明了医疗应用的潜力.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 自然语言处理自然语言处理.
背景情况:
- 将大型语言模型 (LLM) 集成到医疗保健中需要对其性能进行严格的评估.
- 了解广泛使用的聊天机器人的能力和局限性对于它们在医疗领域的安全和有效应用至关重要.
研究的目的:
- 客观地评估主要聊天机器人的准确性,校准错误,可读性和可理解性,使用泌尿病在职检查问题.
- 为了比较五种不同的大型语言模型 (LLM) 在专业医疗领域的性能.
主要方法:
- 三十五个泌尿检查问题被管理到五个LLMs (ChatGPT-4o,DeepSeek-R1,双子座,Grok-2,克劳德3.5) 使用一个标准化的提示.
- 准确度是用科恩的卡帕测量,可读性是通过既定的指数 (Flesch阅读方便,炮雾等) 来测量. ),并通过利卡特级别上的居民评级来理解.
主要成果:
- 模型显示与答案密钥的基本一致 (弗莱斯的卡帕 = 0.701).
- 聊天GPT-4o,DeepSeek-R1和Grok-2实现了80%的准确性,聊天GPT-4o显示了最低的校准误差 (19.2%).
- 在可读性方面,DeepSeek-R1领先,而Claude 3.5获得了最高的可理解性评分.
结论:
- 聊天机器人在不同的医疗任务中表现出各种各样的性能.
- DeepSeek-R1展示了医疗应用的巨大潜力,即使作为一个新发布的模型.
- 需要进一步优化才能充分利用泌尿病学和其他医学专业领域的聊天机器人功能.
相关概念视频
01:19Urodynamic Studies: Uroflowmetry
590
Uroflowmetry is a non-invasive urodynamic test designed to measure various aspects of urination, including volume, flow rate, and the time to void. This test is crucial for diagnosing and assessing conditions such as bladder outlet obstruction, bladder dysfunction, incomplete bladder emptying, incontinence, and urinary tract blockages caused by benign prostatic hyperplasia (BPH) and urethral strictures.Pre-Test Instructions:Before a uroflowmetry test, patients are typically advised to drink...
590
01:28Urologic Endoscopic Procedure: Cystoscopic Examination
358
Meaning of Cystoscopic Examination:Cystoscopy is an essential diagnostic tool in urology that is used to assess the structure and function of the genitourinary system. It provides a direct view of the urethra, bladder, and, in some cases, the ureteral openings. This procedure helps detect structural abnormalities, infections, cancers, and blockages in the urinary tract. There are two types of cystoscopy:Flexible cystoscopy is commonly performed in outpatient settings due to its less invasive...
358
01:25Urinary Tract Calculi VI: Surgical Management
29
Procedures for Kidney StonesMedical intervention is necessary when kidney stones or renal calculi are too large to pass spontaneously (typically greater than 5 millimeters) when stones are accompanied by symptomatic infection (such as fever or pyelonephritis), when they impair kidney function, or when they cause persistent symptoms like severe pain, nausea, or urinary retention. Additionally, patients with only one kidney or those who cannot be treated with medical management also require...
29
01:22Imaging Studies V: Intravenous Urography and Retrograde Pyelography
161
IntroductionIntravenous Urography (IVU) and Retrograde Pyelography (RP) are important diagnostic imaging techniques used to evaluate the urinary system. These methods help identify structural abnormalities, obstructions, and functional issues in the kidneys, ureters, and bladder. Both procedures use iodine-based contrast media to enhance the visibility of urinary tract structures on X-ray images, though they differ in their methods and indications.1. Intravenous Urography (IVU)Intravenous...
161
01:19Anatomy of the Genitourinary System II: Bladder and Urethra
210
The lower urinary system consists of the urinary bladder and urethra, which are essential in storing and expelling urine from the body. Together with the internal and external sphincters, these structures work together to regulate urination effectively.Anatomy of the BladderThe urinary bladder is a muscular, stretchable organ behind the pubic bone and in front of the rectum. In females, the bladder is positioned anterior to the vagina and inferior to the uterus, while in males, it is located...
210