多尺度知识蒸与基于注意力的融合,以获得强大的人类活动识别
Zhaohui Yuan1, Zhengzhe Yang2, Hao Ning3
1Department of Software Engineering,School of Software, East China Jiaotong University, No. 808 Shuanggang East Street, Nanchang, 330013, Jiangxi, China. yuanzh@whu.edu.cn.
Scientific reports
|May 30, 2024
概括
本研究引入了一个多尺度知识蒸框架,以增强多模式机器学习模型培训,改进跨模式和模型的知识传输,以提高性能.
科学领域:
- 机器学习 机器学习
- 计算机视觉 计算机视觉
- 人工智能的人工智能
背景情况:
- 知识蒸对于训练具有异步数据的多模式模型至关重要.
- 现有的方法在全面的跨模式和跨模式知识转移方面扎.
研究的目的:
- 提出一个新的多尺度知识蒸框架.
- 在多模式学习中增强知识转移效率和模型稳定性.
主要方法:
- 引入了多尺度语义图谱映射 (SGM) 损失函数,用于详细的知识传输.
- 开发了一个融合和调 (FT) 模块,以利用模式内和模式间的相关性.
- 利用基于变压器的骨干来进行高级功能学习.
主要成果:
- 在MMAct数据集上实现了2.31%的性能改进,在UTD-MHAD数据集上实现了0.29%的性能改进,用于多式模式的人类活动识别.
- 废弃性研究证实了每个拟议成分的有效性和必要性.
结论:
- 拟议的多尺度知识蒸框架有效地解决了传统方法的局限性.
- 该框架在多式联运人类活动识别任务中表现出卓越的表现.
相关概念视频
Association Areas of the Cortex
5.3K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
5.3K
Tagging and Fusion Proteins
6.6K
Proteins are involved in several cellular processes and biochemical reactions. Analyzing a specific protein of interest requires it to be isolated from the other proteins in the cell. This is achieved by overexpressing the specific gene in a suitable host to produce large quantities of the target protein. A tag or label is recombined with the gene to produce a fusion protein containing the target protein and the tag. The tags on these fusion proteins can then be used for easy detection and...
6.6K
Multi-input and Multi-variable systems
106
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
106


