Attribute Selection Based on Constraint Gain and Depth Optimal for a Decision Tree.

Area of Science:

• Computer Science
• Machine Learning
• Data Mining

Background:

• Information entropy is crucial for decision tree learning, but its kernel's impact on classification performance requires further investigation.
• Existing heuristic methods for decision tree construction often rely on statistical probabilistic information entropy for uncertainty evaluation.

Purpose of the Study:

• To develop novel decision tree learning algorithms that enhance classification performance by optimizing information entropy evaluation.
• To introduce a constrained gain and depth induction approach for improved decision tree construction.

Main Methods:

• Calculated and analyzed uncertainty distributions of information entropy for single- and multi-value events.
• Proposed an information entropy estimation method using a peak-shift sine function for the entropy kernel.
• Developed the constrained gain decision tree (CGDT) and constraint gained and depth inductive improved decision tree (CGDIDT) algorithms.

Main Results:

• Demonstrated an enhanced property of the single-value event entropy kernel and multi-value event entropy peaks.
• Identified a reciprocal relationship between entropy peak location and the number of possible events.
• The proposed CGDT and CGDIDT algorithms showed significant benefits in decision tree learning and classification performance.

Conclusions:

• The study successfully developed advanced decision tree learning algorithms (CGDT and CGDIDT) based on information entropy optimization.
• The novel approach integrating constrained gain and depth induction provides a robust framework for enhancing decision tree accuracy.