Related Experiment Video
Updated: May 17, 2026
05:43
Applying Incongruent Visual-Tactile Stimuli during Object Transfer with Vibro-Tactile Feedback
Published on: May 23, 2019
Transfer of object category knowledge across visual and haptic modalities: experimental and computational studies
Ilker Yildirim1, Robert A Jacobs
1Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY 14627, United States. iyildirim@bcs.rochester.edu
Cognition
|October 30, 2012
Summary
People can transfer object category knowledge between seeing and touching. This study introduces novel 3D objects and a computational model for multisensory shape perception.
Area of Science:
- Cognitive Science
- Neuroscience
- Computer Science
Background:
- Understanding how humans integrate sensory information is crucial for cognitive and neuroscience research.
- Previous studies explored cross-modal object identity transfer, but category knowledge transfer remains less understood.
Purpose of the Study:
- To investigate the transfer of object category knowledge across visual and haptic sensory domains.
- To develop and validate novel, ecologically valid stimuli for cross-modal research.
- To create a computational model for multisensory shape representation and perception.
Main Methods:
- Fabrication of "Fribbles": complex, 3D, multi-part objects with categorical structure for visual-haptic studies.
- Development of the "See and Grasp" dataset, including visual and haptic features of Fribbles.
- Experimental evaluation of cross-modal category knowledge transfer and development of a Bayesian computational model.
Main Results:
- Humans demonstrate significant ability to transfer object category knowledge between visual and haptic perception.
- The "See and Grasp" dataset provides a valuable resource for cross-modal research.
- The computational model successfully accounts for experimental data, highlighting the role of multisensory representations.
Conclusions:
- Object category knowledge is transferable across visual and haptic modalities.
- Multisensory representations and sensory-specific forward models are vital for cross-modal perception.
- The developed stimuli and model advance the study of multisensory integration.
Related Concept Videos
01:15Sensory Modalities
Sensation typically is the process by which the sensory receptors and sense organs detect stimuli from the internal and external environment and transmit this information to the central nervous system for processing.
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
General senses refer to the broad category of sensory information detected by receptors in the body and can be further grouped into somatic and visceral senses. Somatic sensations include touch, pressure, temperature, and pain and are essential for navigating our environment and...
01:11Sensory Perception: Organization of the Somatosensory System
The somatosensory system is the central and peripheral nervous system component that senses and processes touch, pressure, pain, temperature, and body position or proprioception. The process of sensation takes place at three levels:
The receptor level:
The receptor level is the first stage of sensation. It involves the detection of a stimulus by specialized sensory receptors. The stimulus must arrive within the receptor's receptive field. Next, the receptor converts the energy of the stimulus...
The receptor level:
The receptor level is the first stage of sensation. It involves the detection of a stimulus by specialized sensory receptors. The stimulus must arrive within the receptor's receptive field. Next, the receptor converts the energy of the stimulus...
01:33Somatosensation
The somatosensory system relays sensory information from the skin, mucous membranes, limbs, and joints. Somatosensation is more familiarly known as the sense of touch. A typical somatosensory pathway includes three types of long neurons: primary, secondary, and tertiary. Primary neurons have cell bodies located near the spinal cord in groups of neurons called dorsal root ganglia. The sensory neurons of ganglia innervate designated areas of skin called dermatomes.
01:26Introduction to Special Senses
Sensory receptors play an integral part in comprehending our external and internal environments. They receive diverse stimuli, converting them into the nervous system's electrochemical signals. This conversion occurs as the stimulus alters the sensory neuron's cell membrane potential, instigating the generation of an action potential. This action potential is subsequently transmitted to the central nervous system (CNS), which integrates with other sensory data or higher cognitive functions.
01:12Perceptual Constancy
Perceptual constancy is the ability to recognize that objects remain consistent and unchanged even when their appearance varies due to changes in sensory input. There are four main types of perceptual constancy: size constancy, shape constancy, color constancy, and brightness constancy.
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
Size constancy is the recognition that an object remains the same size, even when its image on the retina changes. For instance, a bus is perceived to be large enough to carry people, even if it looks tiny from...
01:14Motor and Sensory Areas of the Cortex
The cerebral cortex, the brain's outermost layer, is pivotal in processing complex cognitive tasks, emotions, and various sensory inputs and executing voluntary motor activities. This intricate structure is divided into three primary functional areas: the motor areas, sensory areas, and association areas.
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex.
Motor Areas
The motor areas located in the frontal lobe are central to controlling voluntary movements. This region is further subdivided into the primary motor cortex and the premotor cortex.