High-dimensional Cost-constrained Regression via Nonconvex Optimization.

Area of Science:

• Statistical modeling
• Optimization techniques
• Machine learning

Background:

• Predictive modeling often faces budget constraints due to high predictor collection costs.
• Developing cost-effective predictive models is crucial for modern data analysis.
• Existing methods may not adequately address high-dimensional data under budget limitations.

Purpose of the Study:

• To develop a novel method for high-dimensional cost-constrained linear regression.
• To minimize expected prediction error within a given budget.
• To address the NP-hard nature of non-convex budget constraints in regression.

Main Methods:

• Proposes a discrete first-order continuous optimization method.
• Estimates regression coefficients by solving a sequence of 0-1 knapsack problems.
• Algorithm generates a series of estimates converging to a stationary point.

Main Results:

• The proposed method converges to a first-order stationary point, potentially a global optimum.
• Demonstrates promising performance on high-dimensional simulated and real-world datasets (e.g., diabetes study).
• Extensions are applicable to general statistical learning and grouped variable problems.

Conclusions:

• The developed method effectively addresses high-dimensional cost-constrained linear regression.
• The approach offers a computationally feasible solution for budget-limited predictive modeling.
• Numerical studies confirm the method's practical utility and performance.