意义的几何学:从各种基于变压器的模型中评估句子嵌入,用于自然语言推断
1Department of Computer Science, College of Computer, Qassim University, Buraydah, Saudi Arabia.
PeerJ. Computer science
|June 26, 2025
概括
本研究评估了用于自然语言推理 (NLI) 句子嵌入的变压器模型. 最佳的聚合和规范大大提高了SNLI和MedNLI数据集上的NLI性能.
科学领域:
- 自然语言处理自然语言处理.
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 自然语言推理 (NLI) 对于理解句子关系至关重要.
- 像BERT,GPT,RoBERTa和XLNet这样的变压器模型对于NLP任务来说非常强大.
- 有效的句子嵌入是NLI模型性能的关键.
研究的目的:
- 评估基于变压器的模型,用于为NLI生成句子嵌入.
- 调查聚合技术,规范和模型层对NLI绩效的影响.
- 确定改善NLI分类准确性的最佳策略.
主要方法:
- 使用变压器双向编码器表示 (BERT),生成预训练变压器 (GPT),强大优化BERT方法 (RoBERTa) 和XLNet.Net的全面实验.
- 用各种聚合技术 (例如,最大聚合) 和规范 (例如,L2规范) 生成的句子嵌入的评估.
- 对每个变压器模型的多层分析.
主要成果:
- 策略,规范和模型层的组合选择显著影响了NLI的业绩.
- 在特定层的L2规范与L2规范相结合的Max聚合产生了最好的结果.
- 在斯坦福自然语言推理 (SNLI) 数据集上获得了90%的准确性和86%的F1分数.
- 在MedNLI数据集上记录了最高的F1得分84%.
结论:
- 变压器模型嵌入,聚合策略和规范选择对NLI至关重要.
- 聚合,规范和层次的特定组合可以显著提高NLI任务性能.
- 这些发现为优化NLI模型和句子嵌入生成提供了实用见解.
相关概念视频
Transformers
1.2K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.2K
Improving Translational Accuracy
11.9K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.9K
Types Of Transformers
1.1K
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
1.1K
The Ideal Transformer
926
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
926
Source Transformation
9.7K
Source transformation is a fundamental technique employed in circuit analysis, offering a valuable tool for simplifying complex electrical circuits. This technique involves the replacement of either a voltage source in series with a resistor by a current source in parallel with a resistor, or vice versa. The key concept here is that when the original sources are deactivated (turned off), the equivalent resistance at the circuit's end terminals remains the same.
It is essential to note that when...
It is essential to note that when...
9.7K
Natural and Artificial Concepts
270
In psychology, concepts can be divided into two categories: natural and artificial. Natural concepts are formed through direct or indirect experiences. For example, consider the concept of snow. If you live in a place with regular snowfall, such as Essex Junction, Vermont, you know snow through direct experiences. You’ve seen it fall, touched it, shoveled it, and played in it. You recognize its texture, appearance, and even its smell. In contrast, if you live on an island like Saint...
270


