大型语言模型在评估学习成果的准确性和可靠性 跨认知领域的学习成果
Swapna Haresh Teckwani1, Amanda Huee-Ping Wong1, Nathasha Vihangi Luke1
1Department of Physiology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
Advances in physiology education
|November 8, 2024
概括
人工智能 (AI) 大型语言模型 (LLM) 显示了对教育评估的分级潜力,但尚未与人类分级器的准确性相匹配. 为了有效地融入体育生理学课程,需要进一步发展人工智能和教育人员的适应.
科学领域:
- 教育技术的教育技术
- 教育中的人工智能
- 运动生理学评估
背景情况:
- 人工智能 (AI) 的整合,特别是大型语言模型 (LLM),为教育评估提供了新的途径.
- 传统的书面评估评分通常是耗时且主观的.
- 法律学历提供了自动化和标准化评分流程的潜力.
研究的目的:
- 评估LLMs (GPT-3.5,GPT-4o,Gemini) 在体育生理学课程的学生作业评分中的准确性和可靠性.
- 为了比较LLM评分表现与人类评分器在认知领域的表现.
- 评估AI在教育评估中的潜力.
主要方法:
- 人类评分和三个法学士 (GPT-3.5,GPT-4o,双子座) 获得了体育生理学课程的学生作业.
- 任务的评估是根据与布鲁姆的分类学 (理解,分析,评估) 和科学调查能力相一致的标题进行的.
- 使用评分器间可靠性,一致性和相关性分析来比较分级结果.
主要成果:
- 法律学士表现出一些能力,但没有达到人类评级评估标准.
- 与LLM评分一致性相比,人类评分器表现出更高的评分器间可靠性.
- 人类和LLM评级者之间的一致性和相关性主要是中度到差的.
结论:
- 人工智能,特别是LLMs,显示出作为教育评估的补充工具的希望,但需要进一步发展.
- 教师需要调整他们的教学和评估策略,以有效地整合人工智能.
- 持续改进人工智能技术对于实现其在教育环境中的全部潜力至关重要.
相关概念视频
Language and Cognition
325
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
325
Improving Translational Accuracy
9.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.1K
Reliability and Validity
12.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
12.7K
Learning Disabilities
84
Learning disabilities are cognitive disorders caused by neurological impairments that affect cognitive functions like language and reading, without indicating overall intellectual or developmental challenges. These disabilities differ from global intellectual or developmental disabilities as they are limited to distinct cognitive functions. Common learning disabilities include dysgraphia, dyslexia, and dyscalculia, each of which impacts unique aspects of learning.
Dyslexia
Dyslexia is a...
Dyslexia
Dyslexia is a...
84
Associative Learning
298
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
298
Cognitive Learning
220
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
220


