在主题分类中对静态和上下文词嵌入的反向翻译效应
Dávid Držík1, Lívia Kelebercová1
1Department of Informatics, Faculty of Natural Science and Informatics, Constantine the Philosopher University in Nitra, Nitra, Slovakia.
PloS one
|August 29, 2025
概括
逆向翻译显著改善了静态词嵌入的主题分类,但对于像RoBERTa这样的高级上下文模型来说,
科学领域:
- 自然语言处理
- 机器学习
背景情况:
- 主题分类对于组织文本数据至关重要.
- 数据增强技术,如反向翻译,可以提高模型的性能.
- 在不同类型的词嵌入中评估这些技术是必不可少的.
研究的目的:
- 评估反向翻译对主题分类表现的影响.
- 将其对静态词向量 (FastText) 与上下文嵌入 (RoBERTa) 的有效性进行比较.
- 确定低资源语言和各种分类算法的好处.
主要方法:
- 实验使用了物流回归,SVM,随机森林和RNN-LSTM分类器.
- 通过六种语言的逆向翻译来增强数据.
- 使用F1评分评估性能,比较原始和增强数据.
主要成果:
- 反向翻译对静态嵌入的F1得分进行了持续的改进 (对于随机森林,高达2.80%).
- 对于静态嵌入的RNN-LSTM,改进较小,并且往往没有统计学意义.
- 反向翻译对RoBERTa的上下文嵌入的影响很小,没有显著的F1得分增长.
结论:
- 逆向翻译是对主题分类中的静态词嵌入的有价值的数据增强策略,特别是在低资源语言中.
- 像RoBERTa这样的现代上下文模型表现出更低的依赖外部增强的高性能.
- 对于高级上下文感知模型来说,反向翻译的实用性是有限的.
更多相关视频
07:36Eye Tracking During Visually Situated Language Comprehension: Flexibility and Limitations in Uncovering Visual Context Effects
Published on: November 30, 2018
15.8K
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
17.2K
相关概念视频
Improving Translational Accuracy
2.7K
2.7K
Translation
143.3K
Lesson: Translation
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of...
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of...
143.3K
Language and Cognition
438
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
438
Cotranslational Protein Translocation
7.6K
Translocation of proteins across membranes is an ancient process that occurs even in bacteria and archaebacteria. In fact, the components of the translocation machinery are still conserved between prokaryotes and eukaryotes.
Sec61 channel partners for cotranslational translocation
During cotranslational translocation, the Sec61 channel partners with the signal recognition particle (SRP), the signal recognition particle receptor (SR), and the ribosomes to transport the nascent polypeptide chain...
Sec61 channel partners for cotranslational translocation
During cotranslational translocation, the Sec61 channel partners with the signal recognition particle (SRP), the signal recognition particle receptor (SR), and the ribosomes to transport the nascent polypeptide chain...
7.6K
Overview of Transposition and Recombination
16.1K
Transposons make up a significant part of genomes of various organisms. Therefore, it is believed that transposition played a major evolutionary role in speciation by changing genome sizes and modifying gene expression patterns. For example, in bacteria, transposition can lead to conferring antibiotic resistance. Movement of transposable elements within the genetic pool of pathogenic bacteria can aid in transfer of antibiotic-resistant genetic elements. In eukaryotes, transposons can carry out...
16.1K
Long-term Potentiation
2.9K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
Hebbian LTP
LTP can occur when...
Hebbian LTP
LTP can occur when...
2.9K
