RNAS-CL:通过跨层知识蒸进行强大的神经架构搜索
Utkarsh Nath1, Yancheng Wang1, Pavan Turaga2
1School of Computing and Augmented Intelligence, Arizona State University, 699 S Mill Ave, Tempe, AZ 85281, USA.
概括
本研究介绍了通过跨层知识蒸 (RNAS-CL) 进行强大的神经架构搜索,以创建更安全的深度学习模型. 通过向强大的教师学习,RNAS-CL增强了神经架构搜索 (NAS) 以产生紧且对抗性强大的网络.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 深度神经网络 (DNN) 容易受到对抗性攻击.
- 神经架构搜索 (NAS) 优化了DNN的预测准确性,但其对抗对手攻击的稳定性未得到充分研究,特别是在知识蒸方面.
- 知识蒸通常只在最后一层专注于匹配输出.
研究的目的:
- 调查NAS是否可以通过从强大的教师模型中继承强度来产生强大的神经架构.
- 提出一种新的NAS算法,RNAS-CL,利用跨层知识蒸来提高对手的稳定性.
主要方法:
- 通过跨层知识蒸 (RNAS-CL) 开发了强大的神经架构搜索.
- RNAS-CL寻找最佳的教师层来监督相应的学生层,而不是传统的方法只专注于最终的输出层.
- 采用跨层知识蒸,将强度从预先训练有素的强大教师网络转移到新的架构中.
主要成果:
- RNAS-CL成功地产生了紧且对抗强大的神经架构.
- 提出的跨层蒸方法有效地从教师模型转移了强度.
- 实验结果验证了RNAS-CL在增强模型安全方面的有效性.
结论:
- RNAS-CL提供了一种用于发现具有增强对抗强度的神经架构的新方法.
- 这些发现表明,通过战略知识蒸,NAS可以被引导产生安全模型.
- 这项研究为为关键应用开发更有弹性的深度学习系统开辟了道路.
相关概念视频
Neuroplasticity
242
Neuroplasticity reflects the brain's remarkable capacity to adapt and evolve, responding dynamically to learning, experiences, or injury by reorganizing its neural circuitry. This reorganization involves creating new neural connections and refining old ones through a series of biological processes that contribute to the brain's lifelong development and adaptability.
242
Neural Circuits
918
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
918


