对分类器的性能变化进行研究,同时对中国短文本分类的预处理方法的影响
Dezheng Zhang1,2, Jing Li1,2, Yonghong Xie1,2
1School of Computer and Communication Engineering, University of Science and Technology Beijing, Haidian, Beijing, China.
PloS one
|October 12, 2023
概括
预处理中文文本,包括单词细分和停止词删除,显著提高了文本分类性能. 这项研究表明,系统预处理对中国短文的机器和深度学习模型的积极影响.
科学领域:
- 自然语言处理自然语言处理.
- 计算语言学 计算语言学
- 机器学习 机器学习
背景情况:
- 文本预处理对于中文文本分类至关重要.
- 现有的研究主要集中在英语文本预处理上.
- 有限的研究探讨了预处理对各种中国文本分类算法的影响.
研究的目的:
- 为了实验性地比较中文文本预处理方法.
- 评估它们对15个常用文本分类器的影响.
- 在各种条件下分析性能,如评估指标和分类器类型.
主要方法:
- 使用了三种标准的中文预处理技术:单词细分,停止词删除和符号删除.
- 在两个中国数据集上测试了15个分类器.
- 分析分类结果,使用诸如宏-F1之类的指标,考虑不同的预处理组合和分类器选择.
主要成果:
- 大多数分类器在应用适当的预处理后显示性能有所改善.
- 系统的预处理对中国短文本分类产生了积极的影响.
- 实现了92.13%和91.99%的宏观F1得分,分别超过了0.3%和2%的基线.
结论:
- 系统地应用预处理方法提高了中文短文本分类.
- 单词分割,停止单词删除和符号删除是有效的预处理步骤.
- 预处理对机器学习和深度学习模型都有好处.
相关概念视频
Classification of Signals
485
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
485
Force Classification
1.2K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
1.2K
Survival Tree
89
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
89
Classification of Systems-II
151
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
151
Classification of Leukocytes
2.0K
Leukocytes are classified into two groups based on the presence or absence of cytoplasmic granules. Granular leukocytes, which contain granules, belong to the myeloid lineage and are divided into three subtypes: neutrophils, eosinophils, and basophils. These cells are roughly spherical and characterized by the granules in their cytoplasm.
Neutrophils are the most abundant type of granular leukocytes, comprising 50-70% of all leukocytes. They feature small, evenly distributed granules and a...
Neutrophils are the most abundant type of granular leukocytes, comprising 50-70% of all leukocytes. They feature small, evenly distributed granules and a...
2.0K
Aggregates Classification
329
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
329


