High-dimensional Parameter Transfer With Fused-Regularizer

Area of Science:

• Statistics
• Machine Learning
• Optimization

Background:

• Parameter estimation accuracy is crucial in high-dimensional statistics.
• Leveraging knowledge from related sources (parameter transfer) can enhance estimation.
• Existing methods struggle with heterogeneous sources and data distribution shifts.

Purpose of the Study:

• To develop a robust parameter transfer method for high-dimensional M-estimators from heterogeneous sources.
• To address challenges posed by data distribution shifts.
• To improve estimation accuracy and achieve optimal rates under weaker conditions.

Main Methods:

• A novel one-step estimator incorporating a fused-regularizer and a target-data-oriented constraint.
• Non-asymptotic analysis of the estimation error for the target parameter.
• Extension of the method to a distributed setting with minimal communication.

Main Results:

• The proposed estimator robustly captures parameter knowledge from source data despite distribution shifts.
• Guaranteed performance no worse than target-data-only estimators.
• Achieves minimax-optimal rates under weaker conditions than prior work.
• Distributed version maintains centralized accuracy with one communication round.

Conclusions:

• The developed method enables effective parameter transfer from heterogeneous sources under distribution shifts.
• The estimator offers improved accuracy and efficiency compared to existing techniques.
• The distributed approach provides a practical solution for large-scale applications.