A descriptive study of variable discretization and cost-sensitive logistic regression on imbalanced credit data

Area of Science:

• Machine Learning
• Data Science
• Statistical Modeling

Background:

• Imbalanced datasets in classification tasks often lead to model bias favoring the majority class.
• This bias can significantly impact the performance and reliability of predictive models, particularly in domains like credit scoring.

Purpose of the Study:

• To investigate the effectiveness of variable discretization and cost-sensitive logistic regression in mitigating bias in imbalanced classification.
• To demonstrate the generalizability of variable discretization across different data domains beyond credit scoring.

Main Methods:

• Applied variable discretization and cost-sensitive logistic regression to an imbalanced credit scoring dataset.
• Evaluated model performance using metrics such as ROC curves, Area Under the ROC Curve (AUC), Type I and Type II errors, accuracy, and F1 score.
• Tested variable discretization on datasets from other domains to assess its broad applicability.

Main Results:

• Both proper variable discretization and cost-sensitive logistic regression with optimal class weights effectively reduced model bias and/or variance.
• Variable discretization outperformed cost-sensitive logistic regression, offering more robust coefficient estimates for predictors with non-linear relationships.
• Cost-sensitive logistic regression enhanced predictor value while maintaining monotonicity, even for non-optimized predictors.

Conclusions:

• Variable discretization is a robust and effective technique for handling imbalanced data across various domains.
• Cost-sensitive logistic regression offers algorithmic benefits for imbalanced classification, complementing discretization methods.
• The combined or individual application of these methods significantly improves classification model fairness and performance on imbalanced datasets.