DREAMER: a computational framework to evaluate readiness of datasets for machine learning.

Area of Science:

• Computer Science
• Data Science
• Artificial Intelligence

Background:

• Machine learning (ML) is crucial for large-scale data analysis.
• Dataset quality is critical for successful ML model deployment.
• Existing methods for assessing data readiness can be manual and time-consuming.

Purpose of the Study:

• To introduce DREAMER (Data REAdiness for MachinE learning Research), an automated framework for evaluating tabular dataset suitability for ML.
• To provide an open-source tool for the research community to improve data quality.

Main Methods:

• Developed DREAMER, an algorithmic framework using supervised and unsupervised ML techniques.
• Applied the framework to three distinct tabular datasets.
• Utilized established data quality metrics for assessment.

Main Results:

• DREAMER significantly enhanced dataset quality, improving readiness for ML tasks.
• Extraneous features and rows were effectively eliminated, streamlining the datasets.
• The data refinement process led to improved accuracy in both supervised and unsupervised learning.

Conclusions:

• DREAMER offers an automated solution for data readiness, boosting raw dataset integrity for ML pipelines.
• The framework streamlines datasets, enhancing the accuracy and efficiency of ML algorithms.
• Open accessibility on GitHub and Docker promotes community adoption and further development.