Super greedy trees

Area of Science:

• Machine Learning
• Statistical Modeling
• Data Mining

Background:

• Decision trees like CART are widely used but can struggle with complex data structures.
• Existing methods may lack flexibility in partitioning data or interpretability.

Purpose of the Study:

• Introduce Super Greedy Trees (SGTs), an extension of CART.
• Develop a flexible and theoretically sound tree-based learning framework.
• Enhance predictive accuracy and interpretability for complex regression tasks.

Main Methods:

• SGTs construct tree splits using lasso-penalized parametric models at each node.
• Adaptive multivariate geometric cuts (linear or curved) are used to reduce empirical risk.
• An ensemble extension, Super Greedy Forests (SGFs), is also developed.

Main Results:

• SGTs and SGFs demonstrate strong performance in simulated and real-world regression studies.
• They outperform CART, oblique trees, random forests, and gradient boosted trees on complex response surfaces.
• SGFs successfully identified sparse signal combinations linked to long-term survival in an ECG case study.

Conclusions:

• The SGT framework provides a flexible and theoretically sound approach to tree-based learning.
• SGTs and SGFs offer richer data partitions than traditional CART while maintaining interpretability.
• This method is particularly effective for complex regression problems and identifying key predictive factors.