Benchmarking the robustness of the correct identification of flexible 3D objects using common machine learning models

Area of Science:

• Molecular science
• Computer vision
• Machine learning

Background:

• True three-dimensional (3D) data are common in molecular science and computer vision.
• Machine learning models often identify objects with intrinsic flexibility in these datasets.
• Molecules exhibit flexibility, leading to shape-based confusions in classification tasks.

Purpose of the Study:

• To assess the classification robustness of common machine learning model/feature combinations for 3D molecular data.
• To establish a baseline for shape abstraction by reducing training and test set overlap.
• To investigate the impact of training data coverage on model performance and overfitting.

Main Methods:

• Introduction of two molecular science datasets for evaluating classification robustness.
• Implementation of blocked training and test sets to minimize data overlap.
• Comparison of various model architectures and featurization techniques, including 2D embeddings of voxelized data.

Main Results:

• Increased training data coverage improved model performance on unseen data and reduced overfitting.
• 2D embeddings of voxelized data demonstrated superior performance among tested featurization methods.
• Model performance was influenced by both featurization choices and task-appropriate model design.

Conclusions:

• Featurization and task-appropriate model design are critical for robust classification of flexible 3D data.
• Learned shape abstraction from database samples can generalize to time-evolving molecular data.
• The study highlights the importance of developing models that can abstract shape from flexible 3D objects.