评估大型语言模型以基于标准的分级,从一致性到一致性
Da-Wei Zhang1, Melissa Boey2, Yan Yu Tan2
1Department of Psychology, Jeffrey Cheah School of Medicine and Health Sciences, Monash University Malaysia, Bandar Sunway, 475000, Malaysia. daweizhang.edu@gmail.com.
大型语言模型 (LLM) 可以有效地执行基于标准的分级. 使用特定标准的快速工程提高了LLM分级的准确性,显示领域知识是教育反的关键.
科学领域:
- 人工智能的人工智能
- 教育技术的教育技术
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 在各种应用中显示出前景.
- 自动评分系统对于教育效率至关重要.
- 在基于标准的分级中,LLM的有效性需要进行彻底的评估.
研究的目的:
- 评估LLMs在基于标准的分级方面的能力.
- 调查详细提示工程对分级性能的影响.
- 了解领域特定知识在LLM评分中的作用.
主要方法:
- 量化分析,将LLM绩效与人类基准进行比较.
- 使用已确立的分级标准对LLMs进行评估.
- 使用快速工程技术进行实验,以完善LLM指令.
主要成果:
- 免费的LLM在基于标准的评分中表现出熟练.
- 法学士表现出对评分标准的细微理解.
- 对于准确的分级,特定领域的理解比模型的复杂性更为关键.
结论:
- 法律学士有能力提供基于标准的教育反.
- 快速工程显著影响LLM分级质量.
- LLM提供了一个可扩展的解决方案,用于提供教育反.
更多相关视频
09:09Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
Published on: September 27, 2024
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
相关概念视频
Sieve Analysis and Grading Curves
Improving Translational Accuracy
Types of Aggregate Grading
Well-graded aggregates include a complete range of necessary size fractions that fit together to create a dense matrix with minimal voids, represented by a smooth, continuous gradation curve. This type of grading ensures good...
Accuracy, limits, and approximation
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
Reliability and Validity
Testing a Claim about Standard Deviation
The hypothesis testing for the claim of population standard deviation (or variance) requires the data and samples to be random and unbiased. The population distribution also must be normal. There is no specific requirement on the sample size as the estimation is based on the chi-square distribution.
As a first step, the hypothesis (null and alternative) concerning the claim about...
