DLProv: a suite of provenance services for deep learning workflow analyses

Area of Science:

• Computer Science
• Artificial Intelligence
• Data Science

Background:

• Deep learning (DL) workflows involve complex, interdependent steps like data preparation, training, and evaluation.
• Ensuring trust, reproducibility, and transparency in DL models is critical for production environments.
• Existing traceability solutions often lack integration across DL workflow stages and use proprietary formats.

Purpose of the Study:

• To introduce DLProv, a suite of provenance services for end-to-end traceability in DL workflows.
• To address limitations of current traceability methods by providing integrated, interoperable solutions.
• To enhance trust, reproducibility, and transparency throughout the DL model lifecycle.

Main Methods:

• Developed DLProv, a framework-agnostic suite of provenance services.
• Implemented SQL-based querying during training and generated PROV standard-compliant provenance graphs.
• Integrated DLProv with DL frameworks like Keras and specialized models like PINNs.
• Evaluated DLProv on standard datasets (MNIST, CIFAR-100) and a handwritten transcription workflow.

Main Results:

• DLProv successfully captured and managed provenance data across diverse DL tasks, ensuring framework independence.
• Provenance graphs facilitated SQL-based queries during model training with minimal performance impact.
• Evaluated overhead was a maximum of 1.4% execution time, outperforming MLflow in comparative analysis.
• Demonstrated adaptability and flexibility across various complexity levels of DL workflows.

Conclusions:

• DLProv provides a robust, framework-agnostic solution for end-to-end traceability in DL workflows.
• The suite enhances trust, reproducibility, and transparency, crucial for deploying DL models.
• DLProv's minimal overhead and interoperability make it suitable for real-world DL applications.