博斯NAS家族:区块智能自主监督的神经架构搜索
概括
这项研究介绍了BossNAS,一种无监督的神经架构搜索 (NAS) 方法. BossNAS通过使用区块智能自主监督学习和以人口为中心的搜索策略,有效地找到高性能模型,包括变压器.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 神经网络的近期进步凸显了对多样化的架构设计的需求.
- 神经架构搜索 (NAS) 方法旨在自动设计高效的神经网络.
- 有效地搜索各种空间,结合卷积神经网络 (CNN) 和变压器仍然是一个挑战.
研究的目的:
- 开发一种新的无监督NAS方法,BossNAS,能够处理多样化的搜索空间.
- 解决大型重量共享空间和有偏见的监督中不准确的架构排名问题.
- 通过整合掩面图像建模来提高NAS性能.
主要方法:
- 引入了BossNAS (区块智能自主监督神经架构搜索),将搜索空间分解为块.
- 开发了Ensemble Bootstrapping,用于无监督,区块智能的训练.
- 建议以人口为中心的搜索来优化候选架构.
- 通过面具图像建模改进了该方法,创建了BossNAS++与面具集团引导和面具人口中心搜索.
主要成果:
- BossNAS和BossNAS++发现了在各种数据集和搜索空间中显示出令人印象深刻的结果的模型.
- 来自BossNAS ++的变压器模型在ImageNet (10.5B MAdds) 上实现了83.2%的准确性,在较低的计算成本下,其性能比DeiT-B高1.4%.
- 实现了高架构评级准确性,Spearman相关性为0.78 (MBConv/ImageNet) 和0.76 (NATS-Bench/CIFAR-100),超过了最先进的NAS方法.
结论:
- 博斯NAS提供了一种有效的无监督方法来搜索各种神经网络架构.
- 区块智能自我监督策略和以人口为中心的搜索减轻了NAS常见的挑战.
- BossNAS++进一步提高了架构选择中的性能和公平性,在基准数据集上表现出强的结果.
相关概念视频
01:37Neural Regulation
34.8K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
34.8K
01:21Distributed Loads: Problem Solving
1.3K
Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...
1.3K
01:30Machines: Problem Solving II
791
Machines are complex structures consisting of movable, pin-connected multi-force members that work together to transmit forces. Consider a lifting tong carrying a 100 kg load. It comprises movable sections DAF and CBG linked together with member AB.
791
01:25Neural Circuits
3.0K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
3.0K
01:22Block Diagram Reduction
727
The process of deriving the transfer function of a control system often involves reducing its block diagram to a single block. This simplification can be achieved through a series of strategic operations, including relocating branch points and comparators. These operations preserve the overall function of the system while allowing for easier manipulation and combination of blocks.
The first step in this process is the identification and relocation of a branch point. A branch point, where a...
The first step in this process is the identification and relocation of a branch point. A branch point, where a...
727
01:24Sequence Networks of Rotating Machines
592
A Y-connected synchronous generator, grounded through a neutral impedance, is designed to produce balanced internal phase voltages with only positive-sequence components. The generator's sequence networks include a source voltage that is exclusively in the positive-sequence network. The sequence components of line-to-ground voltages at the generator terminals illustrate this configuration.
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
592