A Joint Global and Local Temporal Modeling for Human Pose Estimation with Event Cameras

Area of Science:

• Computer Vision
• Biologically Inspired Computing

Background:

• Event-based cameras offer high temporal resolution and dynamic range, suitable for challenging human pose estimation.
• Existing methods struggle with information loss from stationary body parts in event data.
• Short-range temporal analysis is insufficient for complete pose reconstruction.

Purpose of the Study:

• To develop a robust human pose estimation method using event cameras.
• To overcome the challenge of missing data from stationary body parts in event streams.
• To enhance pose prediction accuracy by leveraging extended temporal information.

Main Methods:

• Proposed a joint global and local temporal modeling network (JGLTM).
• JGLTM extracts essential cues from a longer temporal range.
• The network refines local features with global temporal context for accurate pose prediction.

Main Results:

• Demonstrated superior performance in event-based human pose estimation.
• Effectively addressed information loss from stationary body parts.
• Achieved accurate pose prediction across diverse scenarios.

Conclusions:

• The JGLTM network significantly improves event-based human pose estimation.
• Integrating longer temporal ranges is crucial for handling stationary body part data.
• The proposed approach offers a promising solution for real-world applications.