Unsupervised 3D Pose Transfer With Cross Consistency and Dual Reconstruction

Area of Science:

• Computer Vision
• 3D Graphics
• Machine Learning

Background:

• 3D pose transfer aims to re-pose 3D meshes while maintaining identity.
• Deep learning methods have advanced 3D pose transfer but often require extensive ground truth data.
• Real-world applications face limitations in acquiring supervised data for 3D pose transfer.

Purpose of the Study:

• To develop an unsupervised deep learning approach for 3D pose transfer.
• To enable efficient and accurate pose transfer without relying on ground truth annotations.
• To preserve target mesh identity during the pose transfer process.

Main Methods:

• Introduced X-DualNet, a generator with correspondence learning and pose transfer modules.
• Utilized optimal transport for shape correspondence learning without key point annotations.
• Employed elastic instance normalization (ElaIN) for high-quality mesh generation.
• Implemented a cross-consistency learning scheme and dual reconstruction objective for unsupervised training.
• Incorporated an as-rigid-as-possible deformer for body shape refinement.

Main Results:

• Demonstrated successful unsupervised 3D pose transfer on human and animal datasets.
• Achieved performance comparable to state-of-the-art supervised methods.
• Generated high-quality meshes with preserved identity information.

Conclusions:

• X-DualNet offers an effective unsupervised solution for 3D pose transfer.
• The method overcomes the limitations of ground truth data dependency.
• It shows potential for broad applications in 3D content creation and analysis.