聊天GPT生成耳鼻喉科多选题:质量,心理测量特性和适合评估
Cecilia Lotto1,2,3, Sean C Sheppard1, Wilma Anschuetz4
1Department of Otorhinolaryngology, Head and Neck Surgery Inselspital, Bern University Hospital, University of Bern Bern Switzerland.
OTO open
|September 27, 2024
概括
聊天GPT可以生成耳鼻喉科 (ORL) 教育的多选题,但质量各不相同. 专家审查对于医疗检查的适用性至关重要.
科学领域:
- 医学教育 医学教育
- 医疗保健中的人工智能
- 耳鼻喉科 耳鼻喉科 耳鼻喉科
背景情况:
- 像ChatGPT这样的人工智能工具越来越多地被用于教育应用.
- 开发高质量的评估工具对于耳鼻喉科 (ORL) 等专业领域的医学培训至关重要.
研究的目的:
- 评估ChatGPT 3.5在产生耳鼻喉科 (ORL) 多选题 (MCQ) 的能力.
- 评估人工智能生成的ORLMCQ的质量,难度和内部一致性.
主要方法:
- 聊天GPT 3.5被要求在ORL创建20个基于临床病例的MCQ.
- 由医学学生,住院生和顾问对产生的问题进行了质量标准审查.
- 数据分析包括问题难度,相关性和内部一致性 (Cronbach的alpha).
主要成果:
- 聊天GPT生成了20个ORL MCQ;1个是离题的,3个有错误答案,3个有多个正确答案.
- 问题质量各不相同:焦点和相关性都很好,但图片和分散注意力的质量很低.
- 这种难度适合医学学生,但对住院生和顾问来说太容易了;使用学生数据的15个选择问题,克伦巴赫的α为0.69.
结论:
- 聊天GPT 3.5可以产生语法正确的ORL MCQ,适合医学学生.
- 人工智能生成的MCQ的整体质量是平均的,需要大量的专家审查才能在正式的医学检查中使用.
相关概念视频
Cochran's Q Test
242
Cochran's Q Test is a nonparametric statistical test used to determine if there are potential differences in the outcomes of three or more related groups on a binary (yes/no) or dichotomous outcome. It is essentially an extension of the McNemar Test, which is limited to two related samples - Cochran's Q test can handle three or more related samples, making it more versatile in scenarios where subjects are measured under multiple conditions. The test statistic follows a Chi-Square...
242
Self-Report Tests of Personality
325
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
325
Assessment of the Gastrointestinal System II: Health Perception Pattern
81
Assessing the gastrointestinal (GI) system is a complex process that begins with collecting subjective data. This data, collected through patient interviews, provides crucial insights into the patient's health history, perception patterns, and lifestyle habits, all contributing significantly to GI health.
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
Health Perception Patterns
Health perception patterns offer valuable insights into a patient's lifestyle habits and how they may impact their GI health. These patterns include:
81
Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
Sensitivity, Specificity, and Predicted Value
213
In healthcare diagnostics, laboratory tests play a crucial role in identifying and diagnosing a wide range of medical conditions. However, interpreting test results is not always straightforward. An abnormal test result does not always confirm the presence of a disease, just as a normal result does not guarantee its absence. To assess the reliability of these diagnostic tools, healthcare practitioners rely on two key statistical indicators: sensitivity and specificity.
Sensitivity is the...
Sensitivity is the...
213


