一个具有终极搜索效率的新管道用于神经架构搜索神经架构
Wenbo Liu1, Xiaoyun Qiao1, Chunyu Zhao1
1School of Information Science and Technology, Southwest Jiaotong University, Chengdu, SiChuan, PR China.
概括
这项研究引入了一种新的神经架构搜索管道,通过使用数据集蒸和增强的算法,显著减少了搜索时间. 高质量的模型可以快速生成,甚至在两分钟内,从而提高计算效率.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 神经架构搜索 (NAS) 是计算密集型的.
- 现有的方法面临着数据集大小和算法选择的挑战.
- 数据集蒸提供了一种有希望的方法来减少NAS的开销.
研究的目的:
- 开发一个新的NAS管道,通过优化数据和算法提高搜索效率.
- 通过将大型数据集凝聚成代理数据集来解决计算开销.
- 通过减轻选择错误来改进现有的DARTS衍生算法.
主要方法:
- 数据集蒸以创建简化代理数据集.
- 合成数据集利用的两个不同的方案:数据丰富性 (方案1) 和关系映射 (方案2).
- 一个增强的搜索算法,结合了注意力,规范化和正常化.
主要成果:
- 方案1模型显示出出色的可扩展性和可转移到复杂的任务.
- 方案2模型维持了对数据减少的源数据集的性能.
- 改进的算法显著减少了搜索时间,在短短两分钟内生成高质量的模型.
- 通过全面的实验验证方案和算法.
结论:
- 拟议的NAS管道有效地减少了计算开销和搜索时间.
- 数据集蒸和增强的算法能够快速生成高质量的神经架构.
- 该管道为现代机器学习任务提供了可扩展和高效的解决方案.
更多相关视频
相关概念视频
Neural Circuits
1.0K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.0K
Parallel Processing
143
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
143


