Data Valuation with Gradient Similarity.

Area of Science:

• Machine Learning
• Data Science
• Artificial Intelligence

Background:

• High-quality data is fundamental for accurate machine learning (ML) and reliable analytics.
• Mislabeled or noisy data presents a significant challenge across various domains, often necessitating expert knowledge and manual data cleaning.
• Data valuation algorithms quantify individual data sample importance for predictive tasks, aiding in identifying mislabeled data and enhancing ML performance.

Approach:

• Introduces Data Valuation with Gradient Similarity (DVGS), a novel and accessible method for data quality assessment.
• DVGS integrates seamlessly with gradient descent learning algorithms and demonstrates scalability for large datasets.
• The approach aims to reduce reliance on expert knowledge and manual intervention in data preprocessing.

Key Points:

• DVGS effectively identifies mislabeled observations and quantifies data noise.
• The method performs comparably to or surpasses existing data valuation techniques.
• Evaluated across diverse datasets including tabular, image, and RNA expression data, demonstrating cross-domain applicability.

Conclusions:

• DVGS provides a rapid and accurate solution for identifying low-quality data.
• This method has the potential to significantly streamline data cleaning processes.
• By automating data quality assessment, DVGS can enhance the efficiency and effectiveness of ML model development.