GynMedEval:一个全面的数据集,用于评估妇科大语言模型的诊断能力
Fangchen Dong1, HanZhi Zhang2, Yi Ren2
1School of Communications and Information Engineering, North University of China, Taiyuan, China.
概括
一个新的数据集,GynMedEval,评估了用于诊断妇科疾病的大型语言模型 (LLMs). 虽然LLM对常见情况有希望,但目前的模型在复杂情况下的准确性不超过90%.
科学领域:
- 医疗信息学 医疗信息学
- 人工智能在医学中的应用
- 妇科瘤学 妇科瘤学
背景情况:
- 大型语言模型 (LLM) 在医学中显示出显著的潜力.
- 目前对LLM的评估缺乏对复杂的妇科病例的全面评估.
研究的目的:
- 介绍GynMedEval,一种用于评估妇科LLM诊断能力的新型数据集.
- 评估最先进的LLM在复杂的妇科疾病诊断方面的表现.
主要方法:
- 从515个现实病例和中国临床病例结果数据库中开发了GynMedEval数据集.
- 包括50多个患者特征和实验室测试数据,每个样本,格式为多选项评估.
- 使用零射击和少数射击学习设置评估多个LLM.
主要成果:
- 在诊断常见的妇科病症方面,LLMs表现出强项.
- 在复杂的诊断场景中,没有一个评估的LLM在复杂的诊断场景中达到90%以上的准确性.
- 性能在不同的LLM和诊断设置中有所不同.
结论:
- GynMedEval解决了在妇科诊断中评估LLM的关键需求.
- 该数据集有助于更深入地分析LLM在医疗保健中的表现.
- 未来的工作可以利用GynMedEval来改进LLM应用程序,以提高诊断准确性和患者护理.
相关概念视频
Improving Translational Accuracy
2.7K
2.7K
Classification of Illness
7.9K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
7.9K
Sensitivity, Specificity, and Predicted Value
661
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
661
Statistical Software for Data Analysis and Clinical Trials
788
Statistical software is pivotal in data analysis and clinical trials by providing tools to analyze data, draw conclusions, and make predictions. These software packages range from simple data management applications to complex analytical platforms, supporting various statistical tests, models, and simulation techniques. Their significance lies in their ability to handle vast amounts of data with precision and efficiency, enabling researchers to validate hypotheses, identify trends, and make...
788


