Isotonic Modeling with Non-Differentiable Loss Functions with Application to Lasso Regularization.

Area of Science:

• Machine Learning
• Statistical Modeling

Background:

• Isotonic models are valuable for capturing monotonic relationships in data.
• Existing algorithms often require differentiable loss functions, limiting their applicability.
• Non-differentiable loss functions present challenges in optimization and model fitting.

Purpose of the Study:

• To develop an algorithmic approach for fitting isotonic models with convex, non-differentiable loss functions.
• To generalize existing greedy non-regret methods to handle non-differentiable scenarios.
• To ensure the proposed algorithm maintains favorable computational and statistical properties.

Main Methods:

• Generalization of the greedy non-regret approach using sub-gradient extensions.
• Application to isotonic regression and two-class classification with log-likelihood loss.
• Incorporation of a lasso penalty on fitted values for regularization.
• Validation on synthetic and real-world datasets.

Main Results:

• The proposed algorithm effectively solves the isotonic modeling problem for non-differentiable loss functions.
• The method preserves the non-parametric nature of isotonic modeling while controlling complexity.
• Demonstrated efficiency and usefulness on diverse datasets.

Conclusions:

• The developed algorithm provides a robust solution for isotonic modeling with non-differentiable losses.
• This approach enhances flexibility in statistical modeling for regression and classification.
• An open-source implementation is available, facilitating broader adoption and research.