用规则和深度学习方法纠正波斯文本中的拼写错误
Sa Kasmaiee1, Si Kasmaiee2, M Homayounpour2
1Department of Computer Engineering, Amirkabir University of Technology, Tehran, Iran. saman.kasmaiee@aut.ac.ir.
Scientific reports
|November 15, 2023
概括
这项研究开发了波斯语拼写检查的两个系统:基于规则的系统和深度学习模型. 深度学习方法实现了高精度,证明了波斯文本的有效自动拼写纠正.
科学领域:
- 自然语言处理自然语言处理.
- 计算语言学 计算语言学
- 人工智能的人工智能
背景情况:
- 自动拼写纠正对于提高数字文本质量至关重要.
- 由于它的脚本和形态学,波斯语语言处理带来了独特的挑战.
- 现有的波斯拼写检查方法需要改进,以获得更好的准确性.
研究的目的:
- 开发和评估两个不同的系统,用于自动修复波斯语拼写错误.
- 将基于规则的方法与深度学习模型的有效性进行比较.
- 用先进的层来增强深层神经网络,以提高性能.
主要方法:
- 一个基于规则的系统,利用一个常见的拼写错误列表 (700条条) 和一个波斯语单词数据库 (55,000个单词) 与112个校正规则.
- 一种深度学习方法,使用编码解码网络与长短期内存 (LSTM) 和FastText字嵌入.
- 通过结合卷积和囊层来增强深度学习模型,在80万个句子上进行训练.
主要成果:
- 基于规则的系统使用了112个规则,并根据2500个句子进行了评估.
- 深度学习模型,包括带有囊和卷积层的增强版本,显示了与基础网络相比的性能.
- 深度学习模型取得了显著的评估分数:87%的准确性,70%的精度,89%的回忆,78%的F-测量和84%的双语评估研究 (BLEU).
结论:
- 基于规则的和深度学习的方法都可以应用于波斯语拼写纠正.
- 深度学习方法,特别是在增强方面,显示出对准确的自动拼写纠正有很大的潜力.
- 这项研究为波斯语自然语言处理的先进技术提供了宝贵的见解.
更多相关视频
06:09P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation
Published on: September 8, 2023
603
06:48Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
Published on: June 25, 2019
9.2K
相关概念视频
Types of Errors: Detection and Minimization
1.6K
Error is the deviation of the obtained result from the true, expected value or the estimated central value. Errors are expressed in absolute or relative terms.
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
Absolute error in a measurement is the numerical difference from the true or central value. Relative error is the ratio between absolute error and the true or central value, expressed as a percentage.
Errors can be classified by source, magnitude, and sign. There are three types of errors: systematic, random, and gross.
Systematic or...
1.6K
Improving Translational Accuracy
2.6K
2.6K
Proofreading
6.3K
Synthesis of new DNA molecules is carried out by the enzyme DNA polymerase, which adds nucleotides on the daughter strand complementary to the template DNA strand. DNA polymerase has a higher affinity to add the correct base and ensures fidelity during DNA replication. Furthermore, it exhibits proofreading activity during replication, using an exonuclease domain that cuts off incorrect nucleotides from the nascent DNA strand.
Errors During Replication are Corrected by the DNA Polymerase...
Errors During Replication are Corrected by the DNA Polymerase...
6.3K
Detection of Gross Error: The Q Test
6.1K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.1K
Non-equilibrium in the Cell
4.4K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.4K
Systematic Error: Methodological and Sampling Errors
1.5K
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
1.5K
