Enhancing machine learning performance through intelligent data quality assessment: An unsupervised data-centric...

Area of Science:

• Data Science
• Machine Learning
• Analytical Chemistry

Background:

• Poor data quality significantly hinders machine learning (ML) model performance and reliability.
• Increasing data volume and complexity exacerbate data quality issues, demanding extensive preparation.
• Current ML pipelines often involve time-consuming manual data improvement steps.

Purpose of the Study:

• To propose an intelligent data-centric evaluation framework for identifying and improving high-quality data.
• To enhance the performance of ML systems through data quality assessment.
• To develop a flexible framework applicable across various domains.

Main Methods:

• Combines curated quality measurements with unsupervised learning techniques.
• Distinguishes between high- and low-quality data points.
• Validated using real-world datasets in analytical chemistry (anti-sense oligonucleotides).

Main Results:

• The framework successfully identifies characteristics of high-quality data.
• Identified data quality metrics guide efficient laboratory experiment design.
• Implementation led to improved ML system performance in the use case.

Conclusions:

• The data-centric evaluation framework effectively addresses data quality challenges in ML.
• It provides actionable insights for data improvement and experimental design.
• The approach demonstrates significant potential for enhancing ML applications in analytical chemistry and beyond.