Multi-sensory surgical support system incorporating, tactile, visual and auditory perception modalities

Area of Science:

• Robotics
• Biomedical Engineering
• Human-Computer Interaction

Background:

• Surgical robots currently lack advanced sensory feedback, limiting their precision and safety.
• Integrating multisensory information can significantly improve surgical performance and outcomes.
• Novel tactile technologies are crucial for advancing robotic surgery capabilities.

Purpose of the Study:

• To develop and evaluate a multisensory surgical support platform integrating tactile, visual, and auditory feedback.
• To assess the utility of a piezoelectric transducer (PZT) based tactile sensor for evaluating tissue haptic properties.
• To explore the conversion of tactile information into audio signals for enhanced perception of tissue characteristics.

Main Methods:

• A multisensory platform was designed integrating tactile, visual, and auditory feedback.
• A piezoelectric transducer (PZT) system was employed to measure tissue haptic properties.
• A video camera tracked sensor spatial position, while tactile data was converted into frequency/amplitude modulated audio signals.

Main Results:

• Representative data from biological tissues demonstrated the system's potential.
• The tactile sensor effectively evaluated tissue haptic properties.
• Tactile information successfully translated into perceptible audio signals, representing tissue characteristics.

Conclusions:

• The developed multisensory system shows promise for enhancing surgical robotics and tele-medical applications.
• Integrating tactile, visual, and auditory modalities offers a significant advancement in surgical support platforms.
• Further research is needed to determine if visual, audio, and tactile modalities function independently or synergistically.