基准测试ChatGPT生成的多选题与牙科教育中教师撰写的项目进行比较
Amber Kiyani1, Fariha Hanif2, Muhammad Muhammad2
1Qatar University, Doha, Qatar. a.kiyani@qu.edu.qa.
Scientific reports
|December 29, 2025
概括
这项研究比较了ChatGPT生成的多选择题 (MCQ) 与指导员为牙科学生撰写的多选择题. 导师项目表现更好,但人工智能生成的问题显示了教育评估的未来潜力.
科学领域:
- 教育技术的教育技术
- 教育中的人工智能
- 医学教育 医学教育
背景情况:
- 大型语言模型 (LLM) 越来越多地用于学生自主学习.
- 评估人工智能产生的教育内容的质量对于有效的整合至关重要.
- 将人工智能工具与传统方法进行比较,可以了解它们的实用性.
研究的目的:
- 将ChatGPT-3.5生成的评估项目的心理测量属性与教师教师创建的评估项进行比较.
- 评估AI产生的多选择题 (MCQ) 在口腔医学背景下对本科牙科学生的表现.
- 分析人工智能生成和教练创建的评估项目的可靠性,难度和歧视.
主要方法:
- 开发了40项口腔医学评估,包括20个MCQ由教练和20个由ChatGPT-3.5.5.5.
- 该评估对四个机构的547名本科牙科学生进行了评估.
- 项目响应理论,特别是3参数物流模型,用于数据分析.
主要成果:
- 与聊天GPT项目 (0.778) 相比,教练项目显示了更高的人员可靠性 (0.845).
- 与ChatGPT项目相比,教练项目显示了更广泛的难度范围 (-1.29至3.22) 和更高的歧视 (0.50至4.64).
- 虽然教练项目平均得分和可变性更高,但ChatGPT项目表现出具有较低猜测参数的有希望的心理测量特性.
结论:
- 教师生成的评估项目目前表现出优越的心理测量特性,包括更好的歧视和难度范围.
- 聊天GPT生成的项目显示出在教育环境中产生MCQ的潜力,需要进一步调查.
- 像ChatGPT这样的LLM可能在未来的教育评估发展中发挥重要作用,可能超越人类的能力.
更多相关视频
13:44Project-Based Learning Guidelines for Health Sciences Students: An Analysis with Data Mining and Qualitative Techniques
Published on: December 9, 2022
4.1K
07:42Quasistatic Mechanical Testing for Computer-Aided Design and Manufacturing Occlusal Veneers Cemented to Milled Dentin Analog Material
Published on: December 20, 2024
683
相关概念视频
Assessment of the Mouth
962
A thorough mouth assessment, including inspection and palpation of the lips, gums, tongue, tonsils, uvula, and pharynx, is crucial in detecting potential health issues. Diseases ranging from oral cancer to systemic conditions like diabetes could be identified early through careful oral examination. This article provides a detailed guide on conducting a comprehensive mouth assessment.
Mouth Inspection
The inspection begins with visually examining the mouth for symmetry, color, and size.
Mouth Inspection
The inspection begins with visually examining the mouth for symmetry, color, and size.
962
Surveys
16.6K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
16.6K
Biostatistics: Overview
687
Biostatistics plays a crucial role in understanding and analyzing data in healthcare and biology. Biostatisticians conduct experiments, gather evidence, and draw meaningful conclusions using statistical methods and techniques. Different variables form the foundation of biostatistical analysis, allowing researchers to understand and interpret data effectively. These variables are classified into different types, each serving a specific purpose in statistical analysis.
Discrete variables are...
Discrete variables are...
687
The Availability Heuristic
6.9K
A heuristic is a general problem-solving framework (Tversky & Kahneman, 1974). You can think of these as mental shortcuts that are used to solve problems. Different types of heuristics are used in different types of situations, and the impulse to use a heuristic occurs when one of five conditions is met (Pratkanis, 1989):
6.9K
Comparing Experimental Results: Student's t-Test
4.7K
The t-test is a statistical method used to compare the sample mean with a population mean or compare two means from two data sets. The test statistic is calculated from the standard deviation, mean, and number of measurements in the data set at a selected confidence interval and then compared to a table of critical values at this confidence level. If the test statistic is smaller than the critical value, the null hypothesis is accepted. In this case, we state that the difference between the...
4.7K
Pre-Procedural Guidelines for Assessing Blood Pressure
788
Accurate blood pressure assessment is crucial for diagnosing and managing various health conditions. To ensure the reliability of these measurements, healthcare professionals must adhere to standardized pre-procedural guidelines. These guidelines enhance patient safety and improve the overall quality of healthcare. The following steps are essential for obtaining accurate and consistent blood pressure readings, from using the appropriate tools to ensuring effective communication with the...
788
