HOIMotion: Forecasting Human Motion During Human-Object Interactions Using Egocentric 3D Object Bounding Boxes.

Area of Science:

• Computer Vision
• Robotics
• Human-Computer Interaction

Background:

• Human motion forecasting is crucial for augmented reality (AR) and robotics.
• Existing methods primarily rely on past body poses, neglecting object interactions.
• Accurate prediction of human movement during object interaction remains a challenge.

Purpose of the Study:

• To introduce HOIMotion, a novel method for human motion forecasting during human-object interactions.
• To leverage both past body poses and egocentric 3D object bounding boxes for improved motion prediction.
• To enhance the realism and precision of forecasted human motion in interactive environments.

Main Methods:

• Feature extraction from body poses and 3D object bounding boxes using encoder-residual graph convolutional networks (GCN) and multi-layer perceptrons.
• Fusion of pose and object features into a novel pose-object graph.
• Future motion forecasting using a residual-decoder GCN.

Main Results:

• HOIMotion significantly outperforms state-of-the-art methods on the Aria digital twin (ADT) and MoGaze datasets.
• Achieved improvements of up to 8.7% on ADT and 7.2% on MoGaze in mean per joint position error.
• A human study confirmed that HOIMotion's forecasted poses are perceived as more precise and realistic.

Conclusions:

• Egocentric 3D object bounding boxes contain significant information for human motion forecasting.
• HOIMotion effectively utilizes object interaction data to improve motion prediction accuracy and realism.
• The proposed method offers a substantial advancement for applications requiring accurate human motion forecasting in interactive settings.