HOIMamba: Bidirectional State-Space Modeling for Monocular 3D Human-Object Interaction Reconstruction.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Monocular 3D human-object interaction (HOI) reconstruction is challenging, requiring simultaneous estimation of human pose, object pose, and contact from a single image.
• Existing token-based methods struggle with disentangling spatial-geometric and semantic cues, limiting their ability to model asymmetric physical interactions.

Purpose of the Study:

• To introduce HOIMamba, a state-space sequence modeling framework for improved monocular 3D HOI reconstruction.
• To address limitations in current methods by enabling structured, bidirectional, and multi-scale interaction state inference.

Main Methods:

• HOIMamba reformulates HOI reconstruction using structured state evolution for evidence propagation, moving beyond symmetric correlation aggregation.
• Introduces a multi-scale state-space module (MSSM) for capturing dependencies from local contact details to global coordination.
• Employs a spatial-channel grouped SSM (SCSSM) block to factorize interaction modeling into spatial and channel pathways, followed by gated fusion.
• Incorporates explicit bidirectional propagation between human and object states to model asymmetric reciprocity.

Main Results:

• HOIMamba demonstrates consistent improvements over state-of-the-art methods on the BEHAVE and InterCap benchmarks.
• Achieved an 8.6% reduction in human Chamfer distance and a 13.5% improvement in contact recall on the BEHAVE dataset compared to Transformer baselines.
• Similar performance gains were observed on the InterCap dataset, validating the framework's effectiveness.

Conclusions:

• HOIMamba's state-space modeling approach, incorporating multi-scale inference, spatial-channel factorization, and bidirectional reasoning, significantly advances 3D HOI reconstruction.
• The framework effectively disentangles geometric and semantic cues, leading to more accurate and physically plausible reconstructions.
• HOIMamba offers a promising direction for future research in complex human-object interaction understanding.