使用微调的小型和大型语言模型进行项目难度建模
Ming Li1, Hong Jiao1, Tianyi Zhou1
1University of Maryland, College Park, MD, USA.
Educational and psychological measurement
|July 9, 2025
概括
新的数据增强策略显著改善了使用小语言模型 (SLMs) 的大规模评估中的项目难度建模. 像BERT这样精心调整的SLM的表现优于基准,而大型语言模型 (LLM) 的表现有限.
科学领域:
- 教育测量教育的测量
- 自然语言处理自然语言处理.
- 机器学习 机器学习
背景情况:
- 项目难度建模对于大规模评估至关重要.
- 现有的方法面临着数据不平衡和特征提取的挑战.
- 需要对小型和大型语言模型 (LLM) 对此任务的有效性进行评估.
研究的目的:
- 用先进的语言模型研究和增强项目难度建模.
- 开发和验证新的数据增强策略.
- 为了比较小语言模型 (SLM) 和LLM在预测项目难度方面的表现.
主要方法:
- 实施新型数据增强技术:即时增强和分布平衡.
- 微调SLM (BERT,RoBERTa) 和评估特定领域模型 (BioClinicalBERT,PubMedBERT) 的微调.
- 探索LLM (GPT-4) 功能,包括链式思维提示和逻辑生成;利用基于嵌入的方法 (NV-Embed-v2).
主要成果:
- 增强策略显著改善了业绩,超过了基准,并减轻了数据不平衡.
- 精心调整的SLM实现了比BEA 2024共享任务中的顶级模型更低的根平均平方误差.
- LLM 显示了概括性,但在难度预测方面遇到了困难;用 SLM 进行组合学习提高了准确性.
结论:
- 新的数据增强策略对于项目难度建模非常有效.
- 精心调整的SLM,特别是通过组合方法,为此特定任务提供了比LLM更好的性能.
- 需要进一步的研究来提高LLM的表现,可能是通过增加培训数据或先进的推理技术.
相关概念视频
Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
Language Development
456
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
456
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
103
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
103
Modeling and Similitude
340
Scaled modeling is a fundamental technique in engineering, enabling the study of large and complex systems by creating smaller, manageable replicas that recreate critical characteristics of the original. In hydrology and civil infrastructure, for example, scaled models of dams help analyze water flow, turbulence, and pressure. This method allows for accurate predictions of real-world behavior within a controlled environment, significantly reducing the cost and time involved in full-scale...
340
Survival Tree
166
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
166
Language and Cognition
453
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
453


