一个生成语言模型工具的评估 写作 考试问题 工具
Christopher J Edwards1, Brian L Erstad1
1Department of Pharmacy Practice and Science, R. Ken Coit College of Pharmacy, University of Arizona, Tucson, AZ, USA.
American journal of pharmaceutical education
|March 13, 2024
概括
生成式语言模型可以为临床实验室解释课程创建多选择题,但大约一半需要修订. 建议对这个教育工具进行进一步的研究.
科学领域:
- 医学教育 医学教育
- 教育中的人工智能
- 制药科学 制药科学
背景情况:
- 为新的制药科学课程开发评估工具至关重要.
- 评估新兴技术,如用于教育应用的生成语言模型是很重要的.
研究的目的:
- 评估生成语言模型在创建常见临床实验室结果解释新选修课程的考试问题方面的能力.
- 通过专家审稿人评估人工智能生成的多选择题的可用性.
主要方法:
- 使用大型语言模型 (ChatGPT 3.5) 创建了100个多选题.
- 两个独立的评价者对问题进行了格式,清晰度,正确性,相关性和难度等方面的评估.
- 每个问题都被分配了一个二分的判断 (可用/不可用).
主要成果:
- 大约50-57%的生成的多选择题被认为可以在没有修改的情况下使用.
- 问题无法使用的最常见原因是存在多个正确答案.
- 评估人员在62%的关于问题可用性的评论上达成一致.
结论:
- 生成式语言模型显示出在专业科学课程中开发考试问题的潜力.
- 需要进一步研究人工智能工具对创建医学教育评估的实用性.
相关概念视频
Detection of Gross Error: The Q Test
6.1K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.1K
Non-equilibrium in the Cell
4.4K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.4K
Genetic Lingo
102.7K
Overview
102.7K
Higher Mental Functions of the Brain: Language
809
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
809
Components of Language
272
Language, whether spoken, signed, or written, consists of specific components: lexicon and grammar. The lexicon is the vocabulary of a language, comprising its words. Grammar is the set of rules used to convey meaning through the lexicon. For example, English grammar adds “-ed” to most verbs to indicate past tense. Words are formed by combining phonemes, which are the basic sound units of a language. Different languages have different sets of phonemes (e.g., “ah” vs.
272
Statically Indeterminate Problem Solving
378
Statically indeterminate problems are those where statics alone can not determine the internal forces or reactions. Consider a structure comprising two cylindrical rods made of steel and brass. These rods are joined at point B and restrained by rigid supports at points A and C. Now, the reactions at points A and C and the deflection at point B are to be determined. This rod structure is classified as statically indeterminate as the structure has more supports than are necessary for maintaining...
378


