Metric from Human: Zero-shot Monocular Metric Depth Estimation via Test-time Adaptation

Area of Science:

• Computer Vision
• Artificial Intelligence

Background:

• Monocular depth estimation (MDE) is crucial for 3D scene reconstruction from 2D images.
• Current monocular metric depth estimation (MMDE) struggles with unseen scenes due to scene-dependent scale recovery.
• Monocular relative depth estimation (MRDE) is effective but lacks metric scale information.

Purpose of the Study:

• To develop a novel approach for zero-shot monocular metric depth estimation (MMDE) that generalizes to unseen scenes.
• To overcome the scene dependency limitations of existing MMDE models.
• To leverage generative models and human priors for metric scale recovery.

Main Methods:

• Proposed Metric from Human (MfH) approach for annotation-free, test-time adaptation.
• Utilized generative painting models to synthesize human figures within input images.
• Employed an off-the-shelf human mesh recovery (HMR) model to estimate human dimensions.
• Propagated metric scale information from estimated human dimensions to the scene context using MRDE predictions.

Main Results:

• Achieved superior zero-shot performance in monocular metric depth estimation (MMDE).
• Demonstrated strong generalization ability of the MfH approach to novel, unseen scenes.
• Successfully bridged generalizable MRDE to zero-shot MMDE via a generate-and-estimate strategy.

Conclusions:

• Metric from Human (MfH) effectively distills scene-independent metric scale priors using humans as landmarks.
• The proposed method enables robust MMDE without requiring scene-specific training data.
• MfH offers a promising direction for improving the generalization of depth estimation models.