多维融合:用于音乐表演艺术的变压器和基于GANs的多模式视听感知机器人
1School of Music and Dance, Shanxi Vocational University of Engineering Science and Technology, Jinzhong, China.
Frontiers in neurorobotics
|October 16, 2023
概括
这项研究介绍了多式机器人音乐表演艺术,通过融合音乐和运动来增强情感表达. 机器人实现了超过94%的流性和95%的准确性,显著改善了对共振艺术体验的性能回忆.
科学领域:
- 机器人技术 机器人技术 机器人技术
- 人工智能的人工智能
- 表演艺术表演艺术
背景情况:
- 社会偏好正在转向艺术中更深层次的情感联系.
- 机器人音乐表演艺术是一个新兴的领域,旨在增强表现力.
研究的目的:
- 探索音乐和运动在机器人表演中的融合,以获得更大的情感影响.
- 研究人工智能模型的使用,以创建同步的视听机器人表演.
主要方法:
- 变压器模型被用来整合音频和视频信号以理解音乐.
- 生成对抗网络 (GAN) 创建了同步的视觉表演.
- 多模式强化学习使机器人运动与音乐元素保持一致.
主要成果:
- 表演顺度得分超过94%,在将动作与音乐对齐时准确率高达95%.
- 与基线方法相比,在性能召回率上观察到33%的提高.
- 对指标的综合改进表明了这种方法在机器人音乐表演艺术中的有效性.
结论:
- 多式机器人音乐表演艺术具有显著的潜力,可以提高情感影响.
- 机器人可以通过组合的视听线索更好地解释和响应音乐,从而实现更精确的表演.
- 这种方法可以将艺术表达转化为机器人,从而实现情感共的表演.
相关概念视频
Transformers
1.1K
A device that transforms voltages from one value to another using induction is called a transformer. A transformer consists of two separate coils, or windings, wrapped around the same soft iron core. However, they are electrically insulated from each other.
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
The iron core has a substantial relative permeability. Therefore, the magnetic field lines generated due to the current in one winding are almost entirely confined within the core, such that the same magnetic flux permeates each turn of both...
1.1K
Transformers in Distribution System
107
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
107
Types Of Transformers
987
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
987
Perception
492
Perception is a fundamental psychological process that enables individuals to organize, interpret, and consciously experience sensory information. This process is crucial for understanding and interacting with the world around us. It includes both bottom-up and top-down processing, each playing a distinct role in how we perceive our environment.
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
Bottom-up processing begins at the sensory level, where receptors detect external environmental stimuli. These could include the tactile sensation of...
492
Perception of Sound Waves
4.5K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
4.5K
Auditory Perception
349
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
349


