Deep learning with 4D spatio-temporal data representations for OCT-based force estimation.

Area of Science:

• Robotics
• Medical Imaging
• Machine Learning

Background:

• Estimating forces between surgical instruments and tissue is crucial for robot-assisted minimally-invasive surgery.
• Vision-based methods and deep learning with optical coherence tomography (OCT) show promise for force estimation.
• Previous methods utilizing 3D volumetric data outperformed 2D depth images.

Purpose of the Study:

• To extend deep learning-based force estimation to 4D spatio-temporal data using streams of 3D OCT volumes.
• To analyze multi-dimensional image data representations for force estimation.
• To investigate the impact of temporal information and future force prediction for enhanced safety.

Main Methods:

• Development and evaluation of novel 4D spatio-temporal deep learning methods.
• Comparison of 4D approaches with lower-dimensional methods using OCT data.
• Analysis of temporal information and prediction of short-term future forces.

Main Results:

• 4D spatio-temporal data processing achieved a mean absolute error of 10.7 mN, outperforming previous data representations.
• Efficient decoupling of spatial and temporal processing in 4D architectures proved advantageous.
• Temporal information was found to be valuable for accurate force estimation.

Conclusions:

• 4D spatio-temporal OCT data significantly enhances deep learning-based force estimation in robotic surgery.
• The study demonstrates the feasibility of predicting future forces, enabling proactive safety measures.
• This research paves the way for more sophisticated and safer surgical robots.