Random forest of perfect trees: concept, performance, applications and perspectives

Area of Science:

• Machine Learning
• Computational Biology
• Statistical Modeling

Background:

• Traditional Random Forests (RF) maximize tree variability by adhering to Breiman's principles.
• This research proposes a novel RF that deviates from these principles by constructing numerous trees with zero classification errors.

Purpose of the Study:

• To develop a new type of random forest (RF) that builds perfect classification trees.
• To introduce a scoring system, the Nguyen Information Criteria (NICs), for evaluating feature predictive qualities.
• To assess the performance of the new RF and NICs against existing methods.

Main Methods:

• Utilized a novel decision tree structure incorporating neurons at each node and a unique "half Christmas tree" architecture.
• Developed ten new statistical information criteria, termed Nguyen Information Criteria (NICs), for feature evaluation.
• Implemented the "perfect tree" construction methods in the R package 'ROP'.

Main Results:

• The first NIC demonstrated faster minimization of the Akaike information criterion compared to the Gini index in logistic regression.
• Features selected using the NICScore showed a marginal advantage over Support Vector Machines-Recursive Feature Elimination (SVM-RFE).
• Artificial neurons in tree nodes enable simultaneous consideration of multiple classifiers, leading to perfect trees without errors.

Conclusions:

• The proposed RF method, utilizing artificial neurons and NICs, offers a powerful alternative for feature selection and predictive modeling.
• The developed NICs provide an effective means to evaluate feature predictive quality.
• The "perfect tree" approach in RFs can achieve high accuracy and overcome limitations of traditional methods.