Mathematical optimization in classification and regression trees

Area of Science:

• Machine Learning
• Continuous Optimization
• Mixed-Integer Linear Optimization

Background:

• Classification and regression trees are standard Machine Learning tools.
• Existing methods have limitations in flexibility and incorporating advanced properties.

Purpose of the Study:

• To review recent advancements in optimization for tree-based models.
• To explore novel formulations using Continuous and Mixed-Integer Linear Optimization.
• To enhance the flexibility and applicability of tree models.

Main Methods:

• Review of recent contributions in Continuous Optimization.
• Review of recent contributions in Mixed-Integer Linear Optimization.
• Comparison of formulations based on decision variables, constraints, and algorithms.

Main Results:

• Novel optimization formulations offer enhanced flexibility for tree models.
• These formulations facilitate incorporation of cost-sensitivity, explainability, and fairness.
• The methods are effective for handling complex data types, including functional data.

Conclusions:

• Optimization paradigms provide powerful tools for advancing tree-based Machine Learning.
• New formulations significantly improve the capabilities of classification and regression trees.
• This research opens avenues for more sophisticated and interpretable tree models.