评估多语言模型在提取答案和生成问题方面的表现
Antonio Moreno-Cediel1, Jesus-Angel Del-Hoyo-Gabaldon1, Eva Garcia-Lopez2
1Departamento de Ciencias de la Computación, Universidad de Alcalá, 28805, Alcalá de Henares, Spain.
Scientific reports
|July 5, 2024
概括
这项研究通过微调变压器模型 (如mT5基) 以提取答案 (AE) 和生成问题 (QG) 来增强西班牙语的多项选择测试生成. 在西班牙联合数据集上训练的mt5基模型实现了卓越的性能,为未来的研究树立了基准.
科学领域:
- 自然语言处理 (NLP) 是一种自然语言处理.
- 人工智能 (AI) 是一种人工智能.
- 机器学习 (ML) 是指机器学习.
背景情况:
- 多选项测试生成是一个复杂的NLP任务,特别是在非英语语言中,因为先前的研究有限.
- 变压器架构具有先进的答案提取 (AE) 和问题生成 (QG) 任务.
- 现有的研究往往缺乏专注于西班牙语NLP挑战自动化测试创建.
研究的目的:
- 在西班牙语中开发和评估更好的答案提取 (AE) 和问题生成 (QG) 模型.
- 调查针对西班牙NLP任务的答案意识方法的有效性.
- 以西班牙语为AE和QG模型建立一个性能基准,使用各种评估指标.
主要方法:
- 微调三个多语言的变压器模型:mt5基,mt0基和BLOOMZ-560M.
- 使用三个数据集:SQuAD的西班牙语翻译,SQAC数据集,以及它们的结合 (SQuAD + SQAC).
- 使用诸如BLEU1-4,METEOR,ROUGE-L,CIDEr,SARI,GLEU,WER等指标来评估模型的性能.
主要成果:
- 在结合的SQuAD+SQAC数据集上微调的mt5基模型显示了AE和QG任务的最佳性能.
- 仅在SQAC数据集上训练的模型也产生了具有竞争力的结果,表明了数据集的有效性.
- mT5基的表现优于基于BLEU,METEOR和ROUGE-L.等标准评估指标的类似研究工作.
结论:
- 当在西班牙联合数据集上使用答案意识方法训练mt5基模型时,它对AE和QG非常有效.
- 这项研究为未来的西班牙NLP研究提供了有价值的基准,用于自动化测试生成.
- 建议使用较新的模型和数据集进行进一步的探索,以推进该领域的发展.
更多相关视频
相关概念视频
Improving Translational Accuracy
2.6K
2.6K
Translation
14.7K
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of Life
Proteins are...
Translation Produces the Building Blocks of Life
Proteins are...
14.7K
Language and Cognition
340
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
340
Extraction: Advanced Methods
444
Metal ions can be separated from one another by complexation with organic ligands–the chelating agent– to form uncharged chelates. Here, the chelating agent must contain hydrophobic groups and behave as a weak acid, losing a proton to bind with the metal. Since most organic ligands used in this process are insoluble or undergo oxidation in the aqueous phase, the chelating agent is initially added to the organic phase and extracted into the aqueous phase. The metal-ligand complex is...
444
Self-Evaluation: Self-Enhancement and Self-Verification
5.2K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.2K
Complementation Tests
4.9K
A complementation test is a simple cross to identify whether the two mutations are located on the same gene or different genes. It was first performed by Edward Lewis in the 1940s while working on fruit flies. He developed the test to identify the location and arrangement of different mutations on chromosomes.
Organisms heterozygous for different mutations are crossed pairwise in all combinations. If present on different genes, the mutations can complement each other by providing the missing...
Organisms heterozygous for different mutations are crossed pairwise in all combinations. If present on different genes, the mutations can complement each other by providing the missing...
4.9K


