Learning with privileged and sensitive information: a gradient-boosting approach

Area of Science:

• Machine Learning
• Artificial Intelligence
• Data Science

Background:

• The privileged information setting in machine learning involves using auxiliary features unavailable during deployment to improve model performance.
• Sensitive features, often excluded due to privacy or ethical concerns, can provide valuable information for model training.
• Existing methods may not fully exploit privileged information, especially in the context of tree-based learners.

Purpose of the Study:

• To develop and evaluate methods for learning with sensitive features using privileged information.
• To enhance classifier performance by effectively utilizing privileged information during the training phase.
• To adapt gradient-boosted decision trees for the privileged information setting.

Main Methods:

• Focus on tree-based learners, specifically gradient-boosted decision trees.
• Utilize privileged features as knowledge to guide the learning algorithm.
• Develop theoretical underpinnings for learning with privileged information.
• Empirically validate the effectiveness of the proposed algorithms.

Main Results:

• Demonstrated improved classifier performance by incorporating privileged information.
• Successfully adapted gradient-boosted decision trees for learning with privileged information.
• Algorithms effectively used privileged features to guide learning from fully observed features.
• Validated effectiveness on standard fairness metrics.

Conclusions:

• The proposed methods offer a viable approach to learning with privileged information, particularly for sensitive features.
• Gradient-boosted decision trees can be effectively enhanced using privileged information for better predictive accuracy.
• The study confirms the utility of privileged information in improving machine learning models while adhering to fairness considerations.