An Entropy-Based Measure for Assessing Fuzziness in Logistic Regression

Area of Science:

• Statistics
• Machine Learning
• Data Science

Background:

• Logistic regression is widely used for classification tasks.
• Existing measures of data-model fit for logistic regression have limitations.
• Entropy has been successfully applied in mixture modeling for classification assessment.

Purpose of the Study:

• To introduce and validate an entropy-based measure for assessing data-model fit in logistic regression.
• To quantify the classification quality and group separation capabilities of logistic regression models.
• To demonstrate the unique contribution of entropy as a complementary fit measure.

Main Methods:

• Development of an entropy-based metric for logistic regression.
• Application of the entropy measure to hypothetical data scenarios.
• Validation through an applied example and Monte Carlo simulations.
• Comparison with existing data-model fit measures.

Main Results:

• The proposed entropy measure effectively quantifies classification quality in logistic regression.
• Entropy provides unique information not captured by traditional fit statistics.
• Simulation results confirm the utility and robustness of the entropy measure.
• The measure demonstrates good performance in assessing group separation.

Conclusions:

• Entropy offers a valuable addition to the toolkit for evaluating logistic regression models.
• The entropy-based measure enhances the assessment of how well logistic regression models classify cases.
• Researchers should consider using entropy alongside other measures for comprehensive model evaluation.