Articulated clinician detection using 3D pictorial structures on RGB-D data

Area of Science:

• Computer Vision
• Medical Imaging
• Robotics

Background:

• Reliable human pose estimation (HPE) is critical for clinical applications like surgical analysis and human-robot cooperation.
• Existing operating room (OR) HPE methods face challenges with color similarity, illumination changes, or invasive sensors.

Purpose of the Study:

• To develop a novel, non-invasive HPE approach for operating rooms using RGB-D data and Pictorial Structures (PS).
• To improve clinician detection and pose estimation accuracy in complex surgical settings.

Main Methods:

• Extended the Pictorial Structures (PS) framework to 3D using RGB-D data.
• Developed robust part detectors utilizing color and depth images, including a novel histogram of depth differences (HDD) descriptor.
• Introduced 3D pairwise constraints and a tractable exact inference method for 3D PS.

Main Results:

• Evaluated the approach on a challenging RGB-D dataset from a live operating room.
• Demonstrated significant improvements in pose estimation and clinician detection compared to existing methods.
• Showcased the effectiveness of 3D PS with RGB-D part detectors in visually challenging environments.

Conclusions:

• The proposed 3D PS approach with RGB-D data offers a robust and easily deployable solution for HPE in operating rooms.
• This method overcomes limitations of previous techniques, enhancing safety and efficiency in clinical settings.