QUEST-AI:一个使用人工智能用于USMLE风格考试的问题生成,验证和改进的系统
Suhana Bedi1, Scott L Fleming2, Chia-Chun Chiang3
1Department of Biomedical Data Science, Stanford University, Stanford, CA, USA, suhana@stanford.edu.
Pacific Symposium on Biocomputing. Pacific Symposium on Biocomputing
|December 13, 2024
概括
使用大型语言模型 (LLM) 的新系统 QUEST-AI 能够高效地生成有效的美国医学执照考试 (USMLE) 式问题. 这种人工智能工具为医疗教育和考试准备提供了具有成本效益的解决方案.
科学领域:
- 医学教育 医学教育
- 医疗保健中的人工智能
- 评估和评价的评估和评估.
背景情况:
- 创建美国医学执照考试 (USMLE) 问题是耗时且昂贵的.
- 大型语言模型 (LLM) 在回答医学问题方面表现有前途,但它们的生成潜力尚未得到充分探索.
研究的目的:
- 引入QUEST-AI,一个利用LLMs创建,识别和纠正USMLE风格的医学考试问题的新系统.
- 评估LLM生成的医学考试内容的有效性和实用性.
主要方法:
- 开发了QUEST-AI,这是一个基于LLM的系统,用于生成医学问题和纠正错误.
- 创建了一个由50个LLM生成和50个人类生成问题组成的测试集.
- 与医生和医学学生进行了两部分评估,以评估问题的有效性和区分能力.
主要成果:
- 大多数QUEST-AI产生的问题被临床医生小组认为是有效的.
- 在LLM产生的问题和人类产生的问题上的表现之间观察到强烈的相关性.
- 评估人员很难区分人工智能和人类产生的问题.
结论:
- QUEST-AI展示了LLMs在医学教育中的开创性应用,以有效地开发考试内容.
- 该系统为创建USMLE风格评估材料提供了一个潜在的成本效益和可访问的替代方案.
- 通过LLM驱动的问题生成可以显著提高准备医学检查内容的方便性和效率.
更多相关视频
16:23Automated, Quantitative Cognitive/Behavioral Screening of Mice: For Genetics, Pharmacology, Animal Cognition and Undergraduate Instruction
Published on: February 26, 2014
14.3K
08:35Interactive and Visualized Online Experimentation System for Engineering Education and Research
Published on: November 24, 2021
2.4K
相关概念视频
01:17Cochran's Q Test
223
Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square...
223
01:25Critical Thinking II
2.8K
Critical thinking is a cognitive process with several attributes. The attributes of critical thinking include the following:
2.8K
01:19Quality Assurance
115
Quality assurance is the overarching term used to describe the activities employed to ensure the proper performance of a system. These activities can be classified into three categories: quality control, quality assessment, and internal corrective measures. Typically, these activities work cyclically: quality control is performed before and during the analysis, while quality assessment occurs during and after the investigation. Internal corrective measures are implemented based on the findings...
115
01:33Enzyme-Linked Immunosorbent Assay
12.4K
In 1971, Peter Perlman and Eva Engvall developed an Enzyme-linked immunosorbent assay (ELISA or EIA). ELISA differs from western blot in that the assays are conducted in microtiter plates or in vivo rather than on an absorbent membrane.
There are many different types of ELISAs, but they all involve an antibody molecule whose constant region binds an enzyme, leaving the variable region free to bind its specific antigen. Enzyme-substrate reaction allows the antigen to be visualized or...
There are many different types of ELISAs, but they all involve an antibody molecule whose constant region binds an enzyme, leaving the variable region free to bind its specific antigen. Enzyme-substrate reaction allows the antigen to be visualized or...
12.4K
01:30Health Information Technology and Healthcare Information System
799
Health Information Technology (HIT)
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
799