从一系列的API中持续学习.
IEEE transactions on pattern analysis and machine intelligence
|September 20, 2024
概括
本研究引入了使用API的持续学习 (CL) 的新框架,通过生成伪数据来解决数据稀缺问题. 该方法有效地从API流中提炼知识,在数据效率和无数据场景中减轻灾难性遗忘.
科学领域:
- 机器学习 机器学习
- 人工智能的人工智能
- 持续学习 持续学习
背景情况:
- 持续学习 (CL) 旨在使模型能够学习新的任务,而不忘记以前的任务.
- 传统的CL方法需要大量的数据集,由于隐私和版权问题,这些数据集往往无法访问.
- 通过API提供的机器学习即服务 (MLaaS) 提供了一个替代方案,但也为CL带来了新的挑战.
研究的目的:
- 提出使用API流的新型数据高效的CL (DECL-API) 和无数据的CL (DFCL-API) 设置.
- 开发一个无数据的合作持续蒸学习框架,以应对未知的模型参数和灾难性遗忘等挑战.
主要方法:
- 一个合作蒸框架,其中有两个发电机和一个CL模型被对抗训练.
- 通过查询API来生成伪数据,以将知识蒸到CL模型中.
- 一个网络相似性规范化术语,以防止忘记以前的API.
主要成果:
- 拟议的方法实现了与经典CL相比的性能,在MNIST和SVHN上的DFCL-API设置中提供了完整的数据.
- 在DECL-API设置中,该方法在CIFAR10,CIFAR100和MiniImageNet上分别达到0.97x,0.75x和0.69x的经典CL性能.
结论:
- 拟议的无数据合作持续蒸学习框架有效地解决了使用API的CL数据稀缺问题.
- 该方法在数据高效和无数据的持续学习场景中都表现出强的表现,在具有挑战性的环境中表现优于传统方法.
相关概念视频
Sampling Continuous Time Signal
218
In signal processing, a continuous-time signal can be sampled using an impulse-train sampling technique, followed by the zero-order hold method. Impulse-train sampling involves the use of a periodic impulse train, which consists of a series of delta functions spaced at regular intervals determined by the sampling period. When a continuous-time signal is multiplied by this impulse train, it generates impulses with amplitudes corresponding to the signal's values at the sampling points.
In the...
In the...
218
Introduction to Learning
342
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
342
Observational Learning
149
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
149
Streamlines, Streaklines, and Pathlines
1.1K
A streamline represents the trajectory that is always tangent to the fluid's velocity vector at any given point. The velocity of a fluid particle is always directed along the streamline, ensuring the particle continuously follows the streamline's path. Streamlines are particularly useful for visualizing the overall direction of flow in a fluid system, and they provide an instantaneous representation of the flow's velocity field. In steady flow, where conditions do not change over...
1.1K
Cognitive Learning
229
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
229
Stream Function
1.3K
In two-dimensional incompressible fluid flow, the continuity equation is essential for ensuring mass conservation, meaning that any change in fluid entering or exiting a region is balanced by a corresponding change elsewhere. For incompressible flow, where density remains constant, this requirement simplifies to the condition that the divergence of the velocity field must be zero. Mathematically, this is expressed as,
1.3K


