Human-in-the-Loop Interpretability Prior.

Area of Science:

• Artificial Intelligence
• Human-Computer Interaction
• Machine Learning Interpretability

Background:

• Achieving both accuracy and interpretability in machine learning models is a significant challenge.
• Previous methods often rely on indirect, quantifiable proxies for interpretability, such as model sparsity or computational complexity.
• These proxies may not fully capture human-perceived interpretability.

Purpose of the Study:

• To develop and evaluate an algorithm that directly optimizes machine learning models for human-perceived interpretability.
• To minimize the number of user studies required to find interpretable and accurate models.
• To investigate how different datasets influence the preferred notions of interpretability.

Main Methods:

• Development of an optimization algorithm that integrates human feedback directly into the model training loop.
• Conducting user studies to gather human judgments on model interpretability.
• Demonstrating the algorithm's effectiveness across multiple diverse datasets.

Main Results:

• The proposed algorithm successfully identifies models that are both predictive and interpretable.
• Human subject results indicated that preferred interpretability proxies vary depending on the specific dataset and task.
• The approach efficiently reduces the number of user studies needed compared to traditional methods.

Conclusions:

• Directly incorporating human feedback into the optimization process is a viable strategy for developing interpretable machine learning models.
• The notion of interpretability is not universal and is context-dependent, varying across different tasks and datasets.
• This human-in-the-loop approach offers a more effective and efficient way to achieve interpretable AI.