基于位置-上下文增材变压器模型,用于对社交媒体上的文本数据进行分类
M M Abd-Elaziz1, Nora El-Rashidy2, Ahmed Abou Elfetouh3
1Information Systems Department, Faculty of Computers and Information Sciences, Mansoura University, Mansoura, Egypt. moh_abdelaziz7@hotmail.com.
Scientific reports
|March 8, 2025
概括
一个新的位置-上下文添加式变压器模型 (PCA模型) 提高了社交媒体文本分类的准确性. 该PCA模型改进了词嵌入和注意力机制,优于现有模型,并通过更多的训练数据显示了更高的准确性.
科学领域:
- 自然语言处理自然语言处理.
- 机器学习 机器学习
- 社交媒体分析
背景情况:
- 社交媒体文本的扩散需要先进的分类模型.
- 基于变压器的模型对于自然语言处理任务是有效的.
- 现有的模型可能无法完全捕捉社交媒体文本中的位置和上下文信息.
研究的目的:
- 为改进社交媒体文本分类引入一种新的位置-上下文添加式变压器模型 (PCA模型).
- 通过整合单词位置和上下文来增强文本表示.
- 改进变压器架构中的添加注意力机制.
主要方法:
- 开发了两阶段的方法:第一阶段将改进的词嵌入 (位置) 与Bi-LSTM (上下文) 集成.
- 第二阶段通过改进附加注意力机制来增强变压器模型.
- 对六个数据集的PCA模型进行了评估,用于与健康相关的社交媒体文本分类.
主要成果:
- 与最先进的结果相比,PCA模型在五个数据集上获得了更好的F1-Score (0.2-10.2%).
- 在四个数据集中表现优于其他三种基于变压器的模型,F1得分的改善为0.1-2.1%.
- 在培训数据量和绩效准确性之间显示出正相关性.
结论:
- PCA模型为社交媒体文本分类提供了卓越的性能.
- 整合位置和上下文信息对于增强文本表示是至关重要的.
- 较大的培训数据集对基于变压器的文本分类模型的准确性产生了积极的影响.
更多相关视频
08:53Integrating Computerized Linguistic and Social Network Analyses to Capture Addiction Recovery Capital in an Online Community
Published on: May 31, 2019
5.1K
09:20Cloud-Based Phrase Mining and Analysis of User-Defined Phrase-Category Association in Biomedical Publications
Published on: February 23, 2019
8.6K
相关概念视频
Aggregates Classification
298
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
298
Transformers
1.0K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.0K
Classification of Signals
371
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
371
Stereotype Content Model
13.9K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
13.9K
Types Of Transformers
943
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
943
Classification of Systems-I
167
Linearity is a system property characterized by a direct input-output relationship, combining homogeneity and additivity.
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
167
