Revisiting the fragility of influence functions

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Explaining deep learning model predictions is crucial, but methods to verify explanation accuracy are limited.
• Influence functions, approximating leave-one-out training effects, are a recent approach but show fragility.
• The reasons behind influence function fragility are not fully understood, and regularization offers inconsistent robustness.

Purpose of the Study:

• To investigate the underlying mechanisms of influence function fragility in deep learning.
• To analyze the impact of experimental procedures and validation metrics on influence function reliability.
• To determine if observed fragility stems from validation methods rather than the functions themselves.

Main Methods:

• Verified influence functions under conditions meeting their convexity assumptions.
• Relaxed convexity assumptions to study non-convexity effects using deeper models and complex datasets.
• Analyzed key metrics and validation procedures commonly used for influence functions.

Main Results:

• Influence function fragility was observed even when convexity assumptions were met.
• Non-convexity introduced by deeper models and complex datasets did not fully explain the fragility.
• The validation procedures themselves were identified as a potential cause of the observed fragility.

Conclusions:

• The fragility of influence functions may be an artifact of the validation methodologies used.
• Rethinking influence function validation is necessary for reliable deep learning explanation verification.
• Further research is needed to develop robust methods for assessing the faithfulness of model explanations.