Fusing information from multiple 2D depth cameras for 3D human pose estimation in the operating room.

Area of Science:

• Computer Vision
• Medical Imaging
• Machine Learning

Background:

• Deep convolutional neural networks excel in computer vision tasks, including 3D human pose estimation.
• Clinical environments like operating rooms present unique challenges such as occlusions, clutter, and poor lighting.
• Patient and staff privacy necessitates the use of unidentifiable data in clinical applications.

Purpose of the Study:

• To develop a robust 3D human pose estimation framework specifically for the clinical domain, addressing challenges in operating rooms.
• To enable accurate human pose estimation while ensuring patient and staff anonymity.

Main Methods:

• A 2D-3D information fusion framework utilizing a network of multiple depth cameras.
• Prediction of 2D joint probabilities from single depth images, followed by fusion in a shared voxel space for a coarse 3D pose estimate.
• Refinement of 3D pose by regressing into the latent space of a pre-trained convolutional autoencoder.

Main Results:

• The proposed framework was evaluated against baselines on the MVOR dataset.
• Optimal results were achieved by fusing 2D information from multiple camera views.
• Constraining predictions with learned pose priors further improved performance.

Conclusions:

• A robust 3D human pose estimation framework for operating rooms was developed using a multi-depth camera network.
• The exclusive use of depth images ensures patient and staff anonymity, making the approach highly suitable for clinical applications.