阅读Me++:用于多域可读性评估的多语言语言模型的基准测试
Tarek Naous1, Michael J Ryan1, Anton Lavrouk1
1College of Computing, Georgia Institute of Technology.
Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing
|July 4, 2025
概括
本研究介绍了ReadMe++,这是一种用于多语言可读性评估的多样化数据集. 在ReadMe++中训练的模型显示了对语言模型的域概括和跨语言转移的改进.
科学领域:
- 自然语言处理自然语言处理.
- 计算语言学 计算语言学
- 机器学习 机器学习
背景情况:
- 现有的多语言可读性评估资源缺乏领域和语言的多样性.
- 这限制了对语言模型性能的跨领域和跨语言分析.
- 开发可靠的方法需要多样化的评估基准.
研究的目的:
- 推出ReadMe++,一个新的多语言,多域数据集用于可读性评估.
- 在多语言可读性方面对大型语言模型 (LLM) 进行基准测试.
- 鼓励对强有力的多语言可读性评估进行研究.
主要方法:
- 创建了ReadMe++:一个数据集,包含来自112个来源的9757个阿拉伯语,英语,法语,印度语和俄语的人类注释句子.
- 在ReadMe++上使用监督,无监督和少数拍摄提示设置对LLM进行基准测试.
- 对域名概括和跨语言转移能力的评估.
主要成果:
- ReadMe++ 能够测试先进的几次射击提示技术.
- 确定了当前最先进的无监督方法的局限性.
- 在ReadMe++上训练的模型表现出优越的域泛化.
- 在受过训练的模型中观察到增强的跨语言传输能力.
结论:
- ReadMe++ 作为多语言可读性评估的有价值的基准.
- 该数据集有助于开发更强大,更可通用的LLM.
- 数据和工具的公开发布旨在推动该领域的发展.
更多相关视频
09:00Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education
Published on: August 16, 2024
918
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.7K
相关概念视频
02:07Improving Translational Accuracy
2.7K
2.7K
01:27Language and Cognition
453
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
453
01:13Multiple Comparison Tests
4.0K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
4.0K
01:22Language Development
456
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
456