一个多式多任务深度学习框架,用于振动反和声音染
Joolekha Bibi Joolee1, Md Azher Uddin2
1Mathematical and Computer Sciences department, Heriot-Watt University Dubai, Dubai, 501745, United Arab Emirates.
Scientific reports
|June 10, 2024
概括
这项研究引入了一种新的深度学习框架,用于在工具表面交互期间生成同步的振动反和声音. 多模式多任务方法显著提高了两种感官反模式的染精度.
科学领域:
- 人与计算机的交互
- 深度学习 (Deep Learning) 是一种深度学习.
- 信号处理 信号处理
背景情况:
- 之前的研究经常将振动反和声音生成单独处理.
- 同步和设计复杂性是多式联络反系统的关键挑战.
研究的目的:
- 引入一种新的多式联机多任务深度学习框架,用于集成的振动和声音染.
- 开发一个端到端的数据驱动系统,用于捕获和生成来自工具表面相互作用的感觉反.
主要方法:
- 开发了一个全面的系统,从纹理中捕获接触加速和声音数据.
- 引入了新的编码器-解码器网络,配有堆叠的变压器和卷积层.
- 应用了基于变压器的数据驱动方法来建模和染振动信号和声音.
主要成果:
- 与最先进的模型相比,拟议的框架在振动和声音数据方面显示出较低的根平均平方 (RMS) 误差.
- 主观评估证实了拟议方法在现有方法上的优越性.
- 实现了多模式感官反的准确同步和染.
结论:
- 这种新的多式多任务深度学习框架有效地模拟并产生同步的振动触觉反和声音.
- 这种方法克服了以前在同时处理多种感官模式方面的局限性.
- 代表了数据驱动方法的重大进步,以实现现实的工具表面交互反.
相关概念视频
Sensory Modalities
1.3K
Sensation typically is the process by which the sensory receptors and sense organs detect stimuli from the internal and external environment and transmit this information to the central nervous system for processing.
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
1.3K
Perceiving Loudness, Pitch, and Location
205
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
205
Multi-input and Multi-variable systems
106
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
106
Perception of Sound Waves
4.4K
The human ear is not equally sensitive to all frequencies in the audible range. It may perceive sound waves with the same pressure but different frequencies as having different loudness. Moreover, the perception of sound waves depends on the health of an individual's ears, which decays with age. The health of one's ears may also be affected by regular exposure to loud noises.
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
The pitch of a sound depends on the frequency and the pressure amplitude of the source. Two sounds of the same...
4.4K
Tactile and Chemical Senses
289
Tactile senses encompass touch, temperature, and pain, each mediated by specific receptors. Touch receptors detect mechanical energy or pressure against the skin. Sensory fibers from these receptors enter the spinal cord and relay information to the brain stem. Here, most fibers cross over to the opposite side of the brain. The touch information then moves to the thalamus, which projects a map of the body's surface onto the somatosensory areas of the parietal lobes in the cerebral cortex.
289
Auditory Perception
331
The auditory system is essential for sound perception, utilizing various critical structures. When sound waves enter the outer ear, they travel through the ear canal and cause the eardrum to vibrate. These vibrations are then transmitted to the middle ear, where three tiny bones – the malleus, incus, and stapes – amplify the sound. This amplification is crucial, as it ensures that the sound vibrations are strong enough to be conveyed to the inner ear. These vibrations then reach the...
331


