相互注意网络:用于关键短语生成的深度注意神经网络
Wenying Duan1, Hong Rao2, Longzhen Duan1
1School of Mathematics and Computer Sciences, Nanchang University, Nanchang, Jiangxi 330031, China.
Computational intelligence and neuroscience
|October 19, 2023
概括
这项研究介绍了一种新的拒绝架构,MA-net,用于神经关键短语生成 (NKG). 通过减少文档噪音和处理词汇库之外的单词,MA-net提高了关键词提取效率,超过了现有的方法.
科学领域:
- 自然语言处理自然语言处理.
- 信息检索 信息检索
- 机器学习 机器学习
背景情况:
- 神经关键短语生成 (NKG) 自动从文档中提取关键短语.
- 与传统方法不同,NKG可以生成源文本中不存在的新型关键短语.
- 现有的NKG模型经常受到源文件中的噪声的阻碍,影响性能.
研究的目的:
- 引入一种新的无声化架构,即相互注意网络 (MA-net),用于神经关键短语生成.
- 解决现有的NKG模型的局限性,这些模型不考虑源文件的拒绝.
- 提高生成的关键词短语的准确性和质量.
主要方法:
- 开发了一个相互关注网络 (MA-net),结合多头关注来识别标题和摘要的相关性,以拒绝.
- 利用多头注意力进行内容矢量计算,提高关键词生成精度.
- 实现了混合网络,以解决外语 (OOV) 问题,通过使单词生成和从源文档复制.
主要成果:
- 通过利用标题-摘要相关性,MA-net架构有效地否认了源文件.
- 对内容向量的多头注意力使用导致更准确的高质量关键词短语生成.
- 混合网络成功地减轻了OOV的挑战,提高了整体模型的稳定性.
结论:
- 拟议的MA-net通过结合有效的拒绝策略,显著增强神经关键短语生成.
- 该模型在五个基准数据集中展示了与最先进的方法相比更高的性能.
- 这项工作推进了自动关键词提取领域,通过提供强大的和准确的denoising方法.
相关概念视频
Nonconscious Mimicry
4.6K
Nonconscious mimicry occurs when individuals alter their mannerisms to match the behaviors and expressions of those nearby, without intention.
4.6K
The Anchoring-and-Adjustment Heuristic
7.3K
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the...
7.3K
Affinity and Avidity
36.2K
Overview
36.2K
The Availability Heuristic
6.0K
A heuristic is a general problem-solving framework (Tversky & Kahneman, 1974). You can think of these as mental shortcuts that are used to solve problems. Different types of heuristics are used in different types of situations, and the impulse to use a heuristic occurs when one of five conditions is met (Pratkanis, 1989):
6.0K
Predicting Products: SN1 vs. SN2
13.5K
Nucleophilic substitution reactions of alkyl halides can proceed via an SN1 or an SN2 mechanism. While in SN2 reactions, the nucleophile attacks the substrate simultaneously as the leaving group departs, in SN1 reactions, the substrate first dissociates to give the carbocation intermediate. Various factors such as the structure of the substrate, the strength of the nucleophile, and the nature of the solvent promote one mechanism over the other.
With increased substitution on the alkyl halide,...
With increased substitution on the alkyl halide,...
13.5K
Long-term Potentiation
2.8K
Long-term potentiation, or LTP, is one of the ways by which synaptic plasticity—changes in the strength of chemical synapses—can occur in the brain. LTP is the process of synaptic strengthening that occurs over time between pre and postsynaptic neuronal connections. The synaptic strengthening of LTP works in opposition to the synaptic weakening of long-term depression (LTD) and together are the main mechanisms that underlie learning and memory.
Hebbian LTP
LTP can occur when...
Hebbian LTP
LTP can occur when...
2.8K


