Point-RMAE: Reinforcement Masked Autoencoder for 3D Representation Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Current 3D masked point modeling relies on fixed masking strategies.
• These strategies overlook inherent geometric and structural point differences, leading to suboptimal representation learning.

Purpose of the Study:

• To address limitations in 3D representation learning by introducing adaptive masking.
• To pioneer the application of reinforcement learning in 3D self-supervised representation learning.

Main Methods:

• Proposing Point-RMAE, a Reinforcement Masked Autoencoder for 3D representation learning.
• Utilizing a Masking Strategy Analyzer and Dynamic Masking Generator guided by geometric features.
• Incorporating a Flow Matching Point Cloud Fast Generator for distribution-aware rewards.

Main Results:

• Achieved outstanding performance across diverse downstream tasks including shape classification, medical diagnosis, and object detection.
• Demonstrated superior 3D representation learning capabilities on ten popular 3D and 4D datasets.
• Point-RMAE significantly outperforms existing empirical masking strategies.

Conclusions:

• Reinforcement learning offers a powerful framework for adaptive masking in 3D representation learning.
• Point-RMAE effectively captures complex geometric information, enhancing model robustness and performance.
• The proposed adaptive strategy learning advances the field of 3D self-supervised learning.