Combining data discretization and missing value imputation for incomplete medical datasets.

Area of Science:

• Data Science
• Medical Informatics
• Machine Learning

Background:

• Data discretization simplifies continuous features for better understanding and analysis.
• Incomplete medical datasets with missing values pose challenges for data mining algorithms.
• The order of applying discretization and missing-value imputation can impact analytical performance.

Purpose of the Study:

• To investigate the combined effect of data discretization and missing-value imputation on incomplete medical datasets.
• To determine the optimal order for applying discretization and imputation techniques.
• To identify the best combination of methods for improving classification accuracy in medical data analysis.

Main Methods:

• Utilized two discretizers: Minimum Description Length Principle (MDLP) and ChiMerge.
• Employed three imputation methods: mean/mode, Classification and Regression Tree (CART), and k-nearest neighbor (KNN).
• Evaluated performance using two classifiers: Support Vector Machines (SVM) and C4.5 decision tree on seven medical datasets.

Main Results:

• Applying discretization before missing-value imputation consistently led to better performance compared to the reverse order.
• The combination of ChiMerge discretization, KNN imputation, and SVM classification achieved the highest classification accuracy rate.
• The study demonstrated the effectiveness of strategic preprocessing for handling incomplete medical data.

Conclusions:

• The sequence of preprocessing steps significantly influences the outcome of data analysis on incomplete medical datasets.
• A preprocessing pipeline involving discretization followed by imputation is recommended for such data.
• The specific combination of ChiMerge, KNN, and SVM offers a powerful approach for accurate medical data classification.