在抽象神经总结器中自我重复
Nikita Salkar1, Thomas Trikalinos2, Byron C Wallace1
1Khoury College of Computer Sciences, Northeastern University, USA.
概括
像BART,T5和Pegasus这样的神经总结器表现出自我重复. BART是最容易发生的,尤其是在对抽象或公式数据进行微调时,在摘要中产生不必要的文物.
科学领域:
- 自然语言处理自然语言处理.
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 神经总结模型可以生成重复的内容.
- 了解和减轻自我重复对于提高摘要质量至关重要.
研究的目的:
- 量化和定性地分析神经总结器输出中的自我重复.
- 识别导致跨不同架构和数据集自我重复的因素.
主要方法:
- 用多个输出出现的n-gram (长度为四个或更长) 来测量自我重复.
- 分析了在五个数据集上微调的三个架构 (BART,T5,Pegasus).
- 对生成的摘要进行了回归和定性分析.
主要成果:
- BART,T5和Pegasus架构表现出自我重复的不同倾向,其中BART最容易受到影响.
- 对抽象或公式数据集的微调与自我重复的增加有关.
- 定性分析揭示了诸如无关广告,免责声明和域名特定短语之类的文物.
结论:
- 自重复是神经总结的一个重要问题,受模型架构和训练数据的影响.
- 这些发现为数据清理和开发方法提供了洞察力,以尽量减少重复.
- 进一步的研究可以建立在集体级别的分析上,以提高总结器的可靠性.
更多相关视频
12:49Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition
Published on: July 13, 2019
17.0K
05:48Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception
Published on: August 9, 2024
1.5K
相关概念视频
Chunking and Rehearsal in Sensory Memory
246
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
246
Elaborative Rehearsals
108
Elaborative rehearsal is a crucial cognitive strategy that strengthens information encoding in long-term memory by making meaningful connections between new data and pre-existing knowledge. This approach contrasts with maintenance rehearsal, which involves simple repetition without delving into the significance of the information. While maintenance rehearsal might temporarily keep information active in short-term memory, it is less effective for long-term retention.
The effectiveness of...
The effectiveness of...
108
Neural Circuits
1.3K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.3K
Additional Subnuclear Structures
2.1K
2.1K
Techniques of Therapeutic Communication II: Focusing, Paraphrasing, and Summarizing
8.3K
Focusing involves centering a conversation on a message's critical elements or concepts. Focusing is valuable if the talk is vague or patients begin to repeat themselves. Sometimes, when patients are asked about their symptoms, they may go off-topic and try to tell their entire life story. Respectfully, the nurse should bring the conversation back into focus.
This therapeutic technique can also be used when a patient brings up pertinent information during a health-related conversation. The...
This therapeutic technique can also be used when a patient brings up pertinent information during a health-related conversation. The...
8.3K
Alternative RNA Splicing
3.8K
3.8K
