教学一致的目标创建可靠的自动临界测试
Brian Ondov1, Dina Demner-Fushman1, Kush Attal2
1National Library of Medicine, Bethesda, MD, USA.
概括
这项研究引入了一种新的方法来创建cloze测试分心器,改进了面具语言模型. 开发的方法为人类理解产生了更可靠和有效的教育评估.
科学领域:
- 自然语言处理自然语言处理.
- 教育技术的教育技术
- 计算语言学 计算语言学
背景情况:
- 蒙面语言模型 (MLM) 适合在封闭测试中生成可信的分心因素.
- 现有的神经系统方法不能充分确保分心器是不同的和不正确的.
- 评估cloze测试的自动化指标缺乏对人类理解的可靠性和有效性.
研究的目的:
- 为分心器生成制定具有教学动机的目标 (可信性,不正确性,独特性).
- 开发一种无监督的,可解释的方法,共同优化分心器集.
- 评估与人类参与者生成的封闭测试的可靠性和有效性.
主要方法:
- 从语言模型中使用条件分布来制定分心目标.
- 开发了一种无监督的方法,根据制定的目标,共同优化分心器集.
- 进行人体评估,以比较与其他方法相对应的cloze测试可靠性和有效性.
主要成果:
- 提出的方法与教师创建的理解测试比最先进的神经方法更强有力的相关性.
- 生成的cloze测试显示出更大的内部一致性.
- 该实现可自由使用,用于创建多选项封闭测试.
结论:
- 开发的方法有效地产生了可信的,明显的和不正确的分心因子,用于cloze测试.
- 这种方法提高了自动化封闭测试创建的可靠性和有效性.
- 该工具为希望创建有效理解评估的教育工作者提供了实用解决方案.
相关概念视频
Complementation Tests
6.1K
A complementation test is a simple cross to identify whether the two mutations are located on the same gene or different genes. It was first performed by Edward Lewis in the 1940s while working on fruit flies. He developed the test to identify the location and arrangement of different mutations on chromosomes.
Organisms heterozygous for different mutations are crossed pairwise in all combinations. If present on different genes, the mutations can complement each other by providing the missing...
Organisms heterozygous for different mutations are crossed pairwise in all combinations. If present on different genes, the mutations can complement each other by providing the missing...
6.1K
Purposive Learning
451
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
451
Cognitive Learning
1.0K
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
1.0K
Reliability and Validity
13.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.7K
Automatic Processing and Automatic Social Behavior
219
Automatic processing refers to the cognitive operations that occur without conscious intent or awareness, playing a fundamental role in shaping social cognition and behavior. These processes enable individuals to navigate complex social environments efficiently by relying on mental shortcuts and pre-existing knowledge structures known as schemas. One of the most influential mechanisms underlying automatic processing is priming, which subtly activates mental representations through exposure to...
219
Social Loafing
38.9K
Another way in which a group presence can affect performance is social loafing—the exertion of less effort by a person working together with a group. Social loafing occurs when our individual performance cannot be evaluated separately from the group. Thus, group performance declines on easy tasks (Karau & Williams, 1993). Essentially individual group members loaf and let other group members pick up the slack. Because each individual’s efforts cannot be evaluated,...
38.9K


