对于预先训练有素的视觉模型而言,实际的持续忘记
IEEE transactions on pattern analysis and machine intelligence
|January 15, 2026
概括
本研究引入了集团 Sparse LoRA (GS-LoRA) 来选择性地从预先训练的视觉模型中删除信息. GS-LoRA有效地删除不需要的数据,同时保留必要的知识,解决持续忘记的挑战.
科学领域:
- 计算机视觉 计算机视觉
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 越来越多的隐私和安全问题需要从预先训练的视觉模型中删除特定信息.
- 在现实世界中,删除请求是连续的,需要在保留现有知识的同时,持续选择性地删除信息.
- 持续遗忘的问题在有效删除,对剩余知识的影响最小化以及处理稀缺或缺失的培训数据方面提出了挑战.
研究的目的:
- 在预训练的视觉模型中解决持续忘记的挑战.
- 开发一种高效有效的选择性信息删除方法.
- 尽量减少信息删除对模型保留知识的影响.
主要方法:
- 建议集团 Sparse LoRA (GS-LoRA) 为每个忘记任务提供独立的微调模块.
- 采用小组稀疏规范化,自动选择并取消特定的LoRA组.
- 介绍了GS-LoRA++,它包含了原型信息,用于增强类特定的遗忘和保留.
主要成果:
- 在视觉模型中展示了对特定类别的有效忘记.
- 在信息被删除后,对剩余类的表现产生最小的影响.
- 通过面部识别,物体检测和图像分类任务的广泛实验来验证.
结论:
- 在预先训练的视觉模型中,GS-LoRA和GS-LoRA++提供了有效和高效的解决方案,用于在预先训练的视觉模型中持续忘记.
- 这些方法成功地平衡了删除不需要的信息与保存所需的知识.
- 这种方法对现实世界场景具有潜在的数据稀缺性是实用的.
相关概念视频
01:21Forgetting
370
Forgetting is an intrinsic aspect of human memory, characterized by the gradual loss or inaccessibility of information over time. Hermann Ebbinghaus, a pioneering psychologist, extensively studied this phenomenon and formulated the forgetting curve. This curve illustrates that memory loss occurs rapidly immediately after learning and then decelerates over time. Several mechanisms contribute to forgetting, including encoding failure, storage decay, retrieval failure, and interference.
Encoding...
Encoding...
370
01:12Perceptual Constancy
1.3K
Perceptual constancy is the ability to recognize that objects remain consistent and unchanged even when their appearance varies due to changes in sensory input. There are four main types of perceptual constancy: size constancy, shape constancy, color constancy, and brightness constancy.
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
1.3K
01:16Interference and Decay
417
Forgetting is a complex cognitive phenomenon influenced by several factors, among which interference and decay are particularly prominent. These processes explain why individuals often struggle to retrieve specific information from memory, leading to lapses in recall that can be observed in everyday situations.
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
Interference occurs when competing memories hinder the retrieval of particular information. It can be classified into two types: proactive and retroactive interference. Proactive...
417
01:12Observational Learning
838
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
838
01:24Implicit Memories
438
Implicit memories, also known as non-declarative memories, are long-term memories that function outside of conscious awareness. These memories influence behavior and skills without explicit knowledge. This type of memory is evident in tasks like playing tennis, snowboarding, and texting. Implicit memory has three subsystems: procedural memory, conditioning, and priming. This type of memory is essential in various activities, from everyday tasks to specialized skills.
One key aspect of implicit...
One key aspect of implicit...
438
01:18Long-Term Memory
650
Long-term memory is a relatively permanent type of memory, capable of storing vast amounts of information over extended periods. Its storage capacity is generally considered unlimited.
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
Long-term memory can be categorized into two primary types: explicit and implicit memory. Explicit memory, also known as declarative memory, involves the conscious recollection of information that we deliberately try to remember, recall, and articulate. This type of memory encompasses specific facts, events, and...
650