Distributionally robust learning-to-rank under the Wasserstein metric

Area of Science:

• Information Retrieval
• Machine Learning
• Optimization

Background:

• Existing listwise Learning-To-Rank (LTR) models often lack robustness against data contamination, including labeling errors, data shifts, and adversarial attacks.
• Distributionally Robust Optimization (DRO) is a promising approach for enhancing model resilience against various noise types and perturbations.
• There is a need for robust LTR models that can maintain performance despite imperfect or manipulated data.

Purpose of the Study:

• To introduce a novel listwise LTR model, Distributionally Robust Multi-output Regression Ranking (DRMRR), designed for improved robustness.
• To develop a DRMRR model that incorporates local context and cross-document interactions through a multivariate mapping.
• To evaluate the performance and resilience of DRMRR against state-of-the-art LTR models and various noise types.

Main Methods:

• Developed DRMRR, a listwise LTR model with a scoring function as a multivariate mapping from feature vectors to deviation scores.
• Employed a Wasserstein DRO framework to minimize a multi-output loss function under worst-case data distributions within a Wasserstein ball.
• Formulated a computationally solvable version of the DRMRR's min-max optimization problem.

Main Results:

• DRMRR significantly outperformed state-of-the-art LTR models in medical document retrieval and drug response prediction tasks.
• The model demonstrated notable resilience against Gaussian noise, adversarial perturbations, and label poisoning.
• DRMRR maintained stable performance even with increased levels of data noise compared to baseline models.

Conclusions:

• DRMRR offers a robust and effective solution for Learning-To-Rank problems, particularly in scenarios with potential data contamination.
• The Wasserstein DRO framework effectively enhances the model's ability to handle distribution shifts and noisy data.
• DRMRR represents a significant advancement in developing reliable LTR systems for real-world applications.