斯洛伐克的形态标记器使用字节对编码算法
Dávid Držík1, Frantisek Forgac1
1Department of Informatics, Constantine the Philosopher University in Nitra, Nitra, Slovak Republic.
PeerJ. Computer science
|December 9, 2024
概括
本研究介绍了SlovaK形态标记器 (SKMT),通过保留词根来改善自然语言处理. SKMT 增强了斯洛伐克语言模型,提高了诸如情绪分析等任务的性能.
科学领域:
- 计算语言学 计算语言学
- 自然语言处理自然语言处理.
- 词典写作 词典写作 词典写作
背景情况:
- 传统的代币化方法往往使词根碎片化,从而损害了斯洛伐克语的词典含义.
- 像SlovakBERT和纯字节对编码 (BPE) 等现有的代码化工具在维护根完整性方面存在局限性.
- 形态分析对于斯洛伐克语的准确语言表达至关重要.
研究的目的:
- 介绍SlovaK形态标记器 (SKMT) 以改善斯洛伐克语文的文本标记化.
- 在字节对编码 (BPE) 培训过程中整合斯洛伐克语言形态学.
- 提高斯洛伐克语言自然语言处理 (NLP) 模型的性能和质量.
主要方法:
- 通过使用BPE整合斯洛伐克形态学来开发SKMT,专注于词根保存.
- 从形态词典和数据库中提取词根,用于tokenizer培训.
- 进行了对SlovakBERT和纯 BPE 代币化器的比较评估,使用了体的预处理.
主要成果:
- SKMT实现了99.7%的根完整性,明显优于斯洛伐克BERT (90.5%) 和纯BPE (93.1%).
- 在情绪分类任务上使用SKMT微调的模型显示,F1得分比传统BPE.PE有3.5%的改善.
- 进一步的验证表明,在语义文本相似性 (STS) 任务上表现更好.
结论:
- SKMT为斯洛伐克语提供了一种优越的文本代币化方法.
- 整合形态信息显著提高NLP模型的性能和质量.
- SKMT代表了斯洛伐克NLP研究和应用的关键进展.
更多相关视频
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
16.8K
06:09P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation
Published on: September 8, 2023
499
相关概念视频
Encoding
131
Information enters the brain through encoding, which is the input of information into the memory system. Once sensory information is received from the environment, the brain labels or codes it. The information is then organized with similar information and connected to existing concepts. Encoding occurs through automatic processing and effortful processing.
Automatic processing involves the encoding of details like time, space, frequency, and the meaning of words, usually done without conscious...
Automatic processing involves the encoding of details like time, space, frequency, and the meaning of words, usually done without conscious...
131
Sign Test for Matched Pairs
102
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in...
To conduct the sign test, we first calculate the differences in...
102
DNA Base Pairing
26.9K
26.9K
Isomerism
18.0K
Isomers are molecules with the same molecular formula but different structural arrangements. Isomers can be further classified into constitutional isomers and stereoisomers. Constitutional isomers differ in the connectivity of their constituent atoms. For example, 2-butanol and diethyl ether are constitutional isomers, as they have the same chemical formula, C4H10O, but differ in the connectivity of the carbon and oxygen atoms. Constitutional isomers have different physical and chemical...
18.0K
LTR Retrotransposons
17.3K
LTR retrotransposons are class I transposable elements with long terminal repeats flanking an internal coding region. These elements are less abundant in mammals compared to other class I transposable elements. About 8 percent of human genomic DNA comprises LTR retrotransposons. Some of the common examples of LTR retrotransposons are Ty elements in yeast and Copia elements in Drosophila.
The internal coding region of LTR retrotransposons and their mechanism of transposition closely resembles a...
The internal coding region of LTR retrotransposons and their mechanism of transposition closely resembles a...
17.3K
Non-LTR Retrotransposons
11.4K
As the name suggests, non-LTR retrotransposons lack the long terminal repeats characteristic of the LTR retrotransposons. Additionally, both LTR and non-LTR retrotransposons use distinct mechanisms of mobilization. Non-LTR retrotransposons are further divided into two classes - Long interspersed nuclear elements (LINEs) and short interspersed nuclear elements (SINEs), both of which occur abundantly in most mammals, including humans. Some of the active non-LTR retrotransposons in humans are L1...
11.4K
