通过语言相似性重新概念化评分可靠性
Ji Yoon Jung1, Ummugul Bezirhan1, Matthias von Davier1
1Boston College, Chestnut Hill, MA, USA.
Educational and psychological measurement
|December 22, 2025
概括
本研究介绍了语言集成可靠性审计 (LiRA),这是评估大规模评估中评分可靠性的新方法. LiRA增强了跨多语言数据集的可靠性估计,提供比传统方法更全面的结果.
科学领域:
- 教育测量教育的测量
- 心理测量 心理测量 心理测量
- 计算语言学 计算语言学
背景情况:
- 在大规模评估中,传统的跨国评分可靠性依赖于双重评分,通常使用有限的多语言样本.
- 现有的方法在高效估计各种语言数据集的可靠性方面面临挑战.
研究的目的:
- 引入语言集成可靠性审计 (LiRA),这是一个用于全面评分可靠性估计的新方法.
- 将可靠性分析扩展到大型,多语言评估环境中的整个数据集.
主要方法:
- 通过分析类似响应中的语义对齐,LiRA自动为每个响应生成第二个分数.
- 权重多数投票机制确定共识得分,确保可靠性测量.
- 该方法旨在适用于整个数据集,并容纳多语言响应.
主要成果:
- LiRA提供了对得分可靠性的更全面,更系统的估计.
- 可靠性在项目,国家和语言层面得到有效评估.
- 该方法保留了传统可靠性估计的核心原则.
结论:
- 在国际评估中,LiRA提供了一种先进的,可扩展的方法来评分可靠性.
- 该方法增强了在各种语言背景下对评分质量的系统评估.
- LiRA代表了对多语言教育数据的心理测量分析的重大进步.
更多相关视频
09:09Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody
Published on: September 27, 2024
774
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
8.0K
相关概念视频
Causes of Similarity-Dissimilarity Effect
236
The similarity-dissimilarity effect, a fundamental concept in social psychology, explains how interpersonal similarities and differences influence attraction and social interactions. This effect is supported by three key psychological perspectives: balance theory, social comparison theory, and consensual validation.Balance Theory and Cognitive ConsistencyBalance theory, developed by Fritz Heider, posits that individuals seek cognitive consistency in their relationships. When two people share...
236
Reliability and Validity
13.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.7K
Kendall's Coefficient of Concordance
915
Kendall's Coefficient of Concordance (W), also known as Kendall's W, is a non-parametric statistical measure used to assess the agreement or concordance between multiple raters or judges when they rank a set of items. It is often used when you have ordinal data (ranks) and you want to see if there is consistency or consensus among the raters. It is widely applied in research areas such as psychology, medicine, and social sciences, where multiple judges are asked to rank or rate subjects...
915
Spearman's Rank Correlation Test
1.4K
Spearman's rank correlation test, also known as Spearman's rho, is a nonparametric method for assessing the strength and direction of association between two variables. This test is particularly valuable when the data distribution is unknown or when the assumption of normality does not hold. Named after the English psychologist and statistician Dr. Charles Edward Spearman, it serves as the nonparametric counterpart to Pearson's correlation coefficient.
Spearman's test calculates correlation by...
Spearman's test calculates correlation by...
1.4K
Language and Cognition
688
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
688
Improving Translational Accuracy
3.5K
3.5K
