D2VP: A Visual First Platform for Diagnosing Data Distribution Mismatch Toward Reliable Machine Learning in Materials...

Area of Science:

• Chemistry
• Materials Science
• Data Science
• Machine Learning

Background:

• Machine learning (ML) models in chemistry and materials discovery face reliability challenges due to Data Distribution Mismatch (DDM).
• DDM manifests as over-density, sparsity, and extrapolation risk, compromising model predictions.
• Existing reliability assessments often rely on abstract metrics, lacking intuitive scientific insight.

Purpose of the Study:

• To introduce a data-centric, visual-first methodology for assessing ML model reliability.
• To operationalize this methodology through an open-source interactive system called D2VP (Data Diagnosis & Visualization Platform).
• To transform abstract reliability metrics into actionable, evidence-based visual inquiry.

Main Methods:

• Developed D2VP, an interactive system with dual capabilities for diagnosing distribution anomalies and visualizing high-dimensional chemical spaces.
• Employed a human-in-the-loop approach for interactive data exploration and reliability assessment.
• Validated the methodology through case studies addressing over-density, sparsity, and extrapolation risk.

Main Results:

• Redundancy-reduced sampling decreased dataset size by over 60% with minimal accuracy loss.
• Exploration-based sampling, with only 10% targeted data, reduced model mean absolute error by over 80% for sparse datasets.
• The credibility analysis toolkit identified high-risk extrapolation scenarios, including a 1:400 training-to-prediction ratio.

Conclusions:

• The D2VP methodology effectively transforms abstract reliability metrics into actionable scientific insights.
• The platform provides a domain-agnostic framework for building trustworthy predictive models in AI for science.
• Visual inquiry and data-centric diagnosis are crucial for reliable ML applications in scientific discovery.