一个统一的面具拼图框架,用于视觉和语言模型
IEEE transactions on pattern analysis and machine intelligence
|October 14, 2025
概括
一个新的面具拼图 (MJP) 框架增强了转换器在联合学习中的安全性. MJP可以防御梯度攻击,并提高计算机视觉和自然语言处理任务的性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 联合学习 (FL) 使用变压器架构来执行计算机视觉 (CV) 和自然语言处理 (NLP) 任务.
- 由于位置嵌入 (PE) 中的敏感信息,变压器容易受到梯度攻击.
- 可以利用PE来重建输入数据,在FL中构成安全风险.
研究的目的:
- 引入一个新的框架,面具拼图 (MJP),以提高变压器在FL的抗梯度攻击的稳定性.
- 在CV和NLP应用中改进变压器模型的性能.
- 开发一种统一的方法,在不同领域确保基于变压器的模型.
主要方法:
- 面具拼图 (MJP) 框架采用随机的代币混合来破坏位置顺序.
- 可学习的未知 (unk) 位置嵌入被用于掩盖混合令牌的PE.
- 这个过程破坏了局部空间信息,迫使模型学习更强大的表示.
主要成果:
- MJP显著提高了变压器模型对抗梯度式攻击的稳定性.
- 该框架在图像分类 (ImageNet-1K) 和文本情绪分析 (Yelp,亚马逊) 中提高了模型性能.
- 实验结果验证了MJP作为视觉和语言任务的有效统一框架.
结论:
- 蒙面拼图 (MJP) 框架有效地减轻了联合学习中的梯度攻击.
- 在多种CV和NLP应用中,MJP提高了变压器的性能.
- 在联邦环境中,MJP为保护变压器模型提供了统一且强大的解决方案.
相关概念视频
Masking and Demasking Agents
3.4K
EDTA titrations may necessitate masking and demasking agents to temporarily protect a particular metal ion in a mixture from the EDTA reaction. These agents facilitate the sequential analysis of the metal ions by forming stable complexes with some—but not all—metal ions during certain steps.
There are many masking agents, such as cyanide, fluoride, triethanolamine, thiourea, and 2,3-bis(sulfanyl)propan-1-ol (formerly 2,3-dimercapto-1-propanol), with the masking agent chosen based on...
There are many masking agents, such as cyanide, fluoride, triethanolamine, thiourea, and 2,3-bis(sulfanyl)propan-1-ol (formerly 2,3-dimercapto-1-propanol), with the masking agent chosen based on...
3.4K
Vision
59.3K
Vision is the result of light being detected and transduced into neural signals by the retina of the eye. This information is then further analyzed and interpreted by the brain. First, light enters the front of the eye and is focused by the cornea and lens onto the retina—a thin sheet of neural tissue lining the back of the eye. Because of refraction through the convex lens of the eye, images are projected onto the retina upside-down and reversed.
59.3K
Collisions in Multiple Dimensions: Problem Solving
5.3K
In multiple dimensions, the conservation of momentum applies in each direction independently. Hence, to solve collisions in multiple dimensions, we should write down the momentum conservation in each direction separately. To help understand collisions in multiple dimensions, consider an example.
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
A small car of mass 1,200 kg traveling east at 60 km/h collides at an intersection with a truck of mass 3,000 kg traveling due north at 40 km/h. The two vehicles are locked together. What is the...
5.3K
Neural Circuits
2.6K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
2.6K
Depth Perception and Spatial Vision
1.8K
Depth perception is the ability to perceive objects three-dimensionally. It relies on two types of cues: binocular and monocular. Binocular cues depend on the combination of images from both eyes and how the eyes work together. Since the eyes are in slightly different positions, each eye captures a slightly different image. This disparity between images, known as binocular disparity, helps the brain interpret depth. When the brain compares these images, it determines the distance to an object.
1.8K
Language and Cognition
711
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
711
