GaussianHand: Real-Time 3D Gaussian Rendering for Hand Avatar Animation

Area of Science:

• Computer Graphics
• Virtual Reality (VR)
• Augmented Reality (AR)

Background:

• Realistic 3D hand avatar rendering is crucial for immersive AR/VR experiences.
• Existing neural radiance field methods require extensive training views and high computational power.
• Limitations in current methods hinder real-time performance and accessibility.

Purpose of the Study:

• To introduce GaussianHand, a novel Gaussian-based real-time 3D rendering approach for hand avatars.
• To enable efficient free-view and free-pose hand avatar animation from sparse view images.
• To overcome the computational demands and extensive view requirements of prior methods.

Main Methods:

• Proposed Hand Gaussian Blend Shapes for effective hand surface geometry modeling and pose-consistent appearance.
• Introduced the Neural Residual Skeleton with Residual Skinning Weights to correct Linear Blend Skinning inaccuracies.
• Utilized sparse view images for training and real-time rendering.

Main Results:

• Achieved significantly more realistic rendering quality using only 5 or 20 training views, compared to 139 views for existing methods.
• Demonstrated real-time rendering performance up to 125 frames per second.
• Outperformed recent methods in both rendering quality and efficiency.

Conclusions:

• GaussianHand offers an efficient and high-quality solution for real-time 3D hand avatar animation.
• The method effectively addresses limitations of previous neural radiance field techniques.
• Enables enhanced user experiences in human-centered AR/VR applications with minimal data requirements.