通过强化学习使用聚合标签进行序列标记
Marcel Geromel1, Philipp Cimiano1
1Center for Cognitive Interaction Technology, Bielefeld University, Bielefeld, Germany.
Frontiers in artificial intelligence
|December 2, 2024
概括
本研究引入了一种新的强化学习方法,用于序列标记任务. 它使用总体注释,就像计数提及一样,来训练模型,减少对详细,昂贵标签的需求.
科学领域:
- 自然语言处理 (NLP) 是一种自然语言处理.
- 机器学习 (ML) 是指机器学习.
- 人工智能 (AI) 是一种人工智能.
背景情况:
- 序列标签对于NLP任务至关重要,例如命名实体识别 (NER),问题解答 (QA) 和信息提取 (IE).
- 对于序列标记的传统监督的ML方法面临的挑战是培训-评估目标不匹配和代币级注释的高成本.
- 现有的方法需要广泛的,细粒度的基准真相数据,这是劳动密集型和昂贵的获取.
研究的目的:
- 为序列标记引入一种新的强化学习 (RL) 方法.
- 通过使用聚合注释 (例如,实体提及数量) 来解决监督方法的局限性,以获得反.
- 为了减少注释成本和序列标记任务的变异.
主要方法:
- 开发了一种用于序列标记的强化学习框架.
- 利用总体注释,特别是对实体提及的计数,以产生培训反.
- 实验了各种聚合反机制和奖励功能,重点是命名实体识别 (NER).
主要成果:
- 证明使用纯粹基于计数的标签可以有效地学习序列标签.
- 即使在序列级别提供反时,也验证了方法的有效性.
- 展示了基于计数的标签的潜力,可以显著降低注释费用并提高一致性.
结论:
- 拟议的基于计数的强化学习方法为传统的监督序列标记提供了可行的替代方案.
- 这种方法与标记级边界确定相比,大大简化了注释过程.
- 这些发现表明,对于更高效和更具成本效益的NLP模型培训,有希望的方向.
相关概念视频
Labeling Emotion
103
Emotional labeling is a cognitive process that involves identifying and naming one's emotions, such as anger, fear, happiness, or sadness. It allows individuals to recognize and express their internal emotional states, a critical aspect of emotional regulation and communication. Labeling emotions requires more than mere recognition; it also involves drawing upon memory and contextual cues to understand the current situation and apply a corresponding emotional label. For instance, feeling...
103
Reinforcement Schedules
132
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
132
Associative Learning
289
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
289
Role of Shaping in Operant Conditioning
266
Shaping is a technique used in operant conditioning to train complex behaviors by rewarding successive approximations toward the target behavior. This method is necessary because organisms are unlikely to perform complex behaviors spontaneously. Instead, shaping breaks down the desired behavior into small, manageable steps.
The steps involved in shaping begin with reinforcing any response that resembles the desired behavior. For example, parents might praise a child for picking up one toy. As...
The steps involved in shaping begin with reinforcing any response that resembles the desired behavior. For example, parents might praise a child for picking up one toy. As...
266
Real-World Application of Classical Conditioning
524
Classical conditioning not only includes the initial pairing of stimuli but also extends to more complex forms, such as higher-order conditioning. Higher-order conditioning involves creating associations beyond the primary conditioned stimulus, resulting in a chain of conditioned responses.
Higher-order, or second-order, conditioning occurs when a neutral stimulus becomes associated with an already established conditioned stimulus through repeated pairings. For instance, if a dog has been...
Higher-order, or second-order, conditioning occurs when a neutral stimulus becomes associated with an already established conditioned stimulus through repeated pairings. For instance, if a dog has been...
524
Reinforcement
179
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
179


