dpdata: A Scalable Python Toolkit for Atomistic Machine Learning Data Sets

Area of Science:

• Computational Chemistry
• Materials Science
• Machine Learning

Background:

• Effective management of atomistic datasets is crucial for developing and deploying machine learning potentials (MLPs).
• Existing tools can be inefficient for handling diverse data formats and processing needs in MLP workflows.

Purpose of the Study:

• To introduce dpdata, an open-source Python library designed to simplify and optimize the handling of atomistic data for MLPs.
• To provide a flexible and extensible solution for reading, writing, converting, and processing diverse datasets.

Main Methods:

• Developed a plugin-based architecture supporting various file formats from quantum chemistry and molecular dynamics software.
• Implemented key utilities for data preprocessing, including train-test splitting, coordinate perturbation, outlier removal, and unit conversion.
• Utilized efficient NumPy-backed storage and system-level operations for memory and speed optimization.

Main Results:

• dpdata supports a wide array of file formats and allows for custom extensions to new software.
• Achieved significant memory savings and inference speedups compared to configuration-by-configuration tools like ASE.
• Demonstrated practical impact through its use in published studies for data conversion, storage, and processing.

Conclusions:

• dpdata offers a robust and efficient solution for managing atomistic data in machine learning potential development.
• The library's flexibility and performance enhancements facilitate streamlined workflows and accelerate research in computational materials science.