SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Current expressive human pose and shape estimation (EHPS) methods are limited by confined datasets and architectural innovations.
• Scaling EHPS is crucial for broader applications but requires extensive data and model capacity.

Purpose of the Study:

• To investigate the impact of data and model scaling on expressive human pose and shape estimation (EHPS).
• To develop generalist foundation models for EHPS that exhibit strong performance and transferability.

Main Methods:

• Systematic investigation of 40 EHPS datasets for data scaling, identifying optimal training schemes and data sources.
• Utilized vision transformers (up to ViT-Huge) and minimalist architectures (SMPLer-X, SMPLest-X) for model scaling.
• Developed a finetuning strategy to adapt generalist foundation models into specialized ones.

Main Results:

• Achieved diminishing returns in EHPS performance at 10 million training instances from diverse sources.
• Foundation models demonstrated strong performance across diverse benchmarks and excellent transferability to unseen environments.
• Finetuned models achieved further performance boosts, delivering state-of-the-art results on seven benchmarks including AGORA, UBody, EgoBody, and SynHand.

Conclusions:

• Scaling data and models significantly enhances EHPS capabilities, leading to generalist foundation models.
• These foundation models offer robust performance and adaptability, outperforming previous state-of-the-art methods.
• The proposed finetuning strategy effectively specializes generalist models for improved performance on specific tasks.