学习博览会的代表性,用于微调预先训练的语言模型
Ke Wang1, Yinghao Zhang1, Hong-Yu Zhang1
1College of Informatics, Huazhong Agricultural University, Wuhan, Hubei, 430070, China.
概括
这项研究介绍了CFPLM,这是一个用于退化预训练语言模型 (PLM) 的新框架. 在不损害性能的情况下,CFPLM使用因果推理来减少AI语言模型中的社会偏见.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 机器学习 机器学习
背景情况:
- 预训练语言模型 (PLM) 在各种NLP任务中表现出色,但继承了人类的偏见.
- 这些偏见,包括社会刻板印象,限制了PLM的安全和道德应用.
- 现有的调解方法往往不能有效地解决偏见的根本原因.
研究的目的:
- 为预先训练的语言模型提出一个新的 debiasing 框架,CFPLM.
- 利用因果推理来识别和干预PLM中诱导偏见的因素.
- 提高PLM的公平性,同时保持他们的语言理解能力.
主要方法:
- 开发了CFPLM (预先训练的语言模型的因果框架) 调试框架.
- 整合了一个复合损失函数,并包含一个公平性罚款条款.
- 集成的对抗性损失和调节,以优化性能.
主要成果:
- 在BERT,RoBERTa和ALBERT等流行的PLM中,CFPLM显著减少了偏差.
- 对标准数据集和指标的评估证实了退化方法的有效性.
- 在GLUE基准指标上的表现显示,在语言理解能力方面没有妥协.
结论:
- 拟议的CFPLM框架有效地使用因果推理来缓解PLM中的偏见.
- 通过CFPLM提高公平性不会对模型的核心语言理解能力产生负面影响.
- CFPLM为开发更道德和可靠的AI语言技术提供了一个有希望的方向.
相关概念视频
01:18Introduction to Learning
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
01:21Cognitive Learning
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
01:22Purposive Learning
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a bonus...
01:12Observational Learning
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning because...
01:22Language Development
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
01:26Modeling in Therapy
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in situations...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in situations...