文本数据增强和预训练的语言模型,用于增强低资源语言的文本分类
Atabay Ziyaden1,2, Amir Yelenov2,3, Fuad Hajiyev4
1Kazakh-British Technical University, Almaty, Kazakhstan.
PeerJ. Computer science
|April 25, 2024
概括
这项研究通过使用文本增强来增强阿塞拜疆语等低资源语言的自然语言处理 (NLP). 翻译技术显著改善了新闻文本分类模型的性能.
科学领域:
- 自然语言处理 (NLP) 是一种自然语言处理.
- 计算语言学 计算语言学
背景情况:
- 先进的语言模型在很大程度上依赖语言资源.
- 由于有限的标记数据集,阿塞拜疆语等低资源语言面临挑战,这阻碍了有效的模型培训.
研究的目的:
- 为了提高新闻文本分类模型的有效性和概括能力,用于低资源语言.
- 通过使用文本增强来解决阿塞拜疆NLP中标记数据的稀缺问题.
主要方法:
- 利用文本增强技术进行新闻文本分类.
- 使用了使用Facebook的mBart50模型和谷歌翻译API的翻译方法.
- 结合mBart50和谷歌翻译,扩展文字处理能力.
主要成果:
- 在增强数据上训练的模型与在原始数据上训练的模型相比,显示了更好的分类性能.
- 证明了对代表性不足的语言联合数据增强策略的潜力.
- 发表了标记文本分类数据集和阿塞拜疆语预训练的RoBERTa模型.
结论:
- 文本增强,特别是使用组合翻译策略,显著提高了NLP模型对低资源语言的性能.
- 开发的资源 (数据集和RoBERTa模型) 将有助于阿塞拜疆NLP的未来研究和开发.
相关概念视频
Improving Translational Accuracy
10.2K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
10.2K
Classification of Signals
453
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
453
How Data are Classified: Categorical Data
32.6K
A variable, usually notated by capital letters such as X and Y, is a characteristic or measurement that can be determined for each member of a population. Data are the actual values of variables. They may be numbers, or they may be words. Datum is a single value.
Data are classified based on whether they are measurable or not. Categorical data cannot be measured; instead, it can be divided into categories. For example, if Y denotes a person's party affiliation, some examples of Y include...
Data are classified based on whether they are measurable or not. Categorical data cannot be measured; instead, it can be divided into categories. For example, if Y denotes a person's party affiliation, some examples of Y include...
32.6K
Translation
141.9K
Lesson: Translation
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of...
Translation is the process of synthesizing proteins from the genetic information carried by messenger RNA (mRNA). Following transcription, it constitutes the final step in the expression of genes. This process is carried out by ribosomes, complexes of protein and specialized RNA molecules. Ribosomes, transfer RNA (tRNA), and other proteins produce a chain of amino acids—the polypeptide—as the end product of translation.
Translation Produces the Building Blocks of...
141.9K
Classification of Systems-II
141
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
141
Classification of Leukocytes
1.9K
Leukocytes are classified into two groups based on the presence or absence of cytoplasmic granules. Granular leukocytes, which contain granules, belong to the myeloid lineage and are divided into three subtypes: neutrophils, eosinophils, and basophils. These cells are roughly spherical and characterized by the granules in their cytoplasm.
Neutrophils are the most abundant type of granular leukocytes, comprising 50-70% of all leukocytes. They feature small, evenly distributed granules and a...
Neutrophils are the most abundant type of granular leukocytes, comprising 50-70% of all leukocytes. They feature small, evenly distributed granules and a...
1.9K


