Challenges in explaining deep learning models for data with biological variation

Area of Science:

• Agricultural Science
• Computer Science
• Machine Learning

Background:

• Machine learning models trained on benchmark datasets often fail on complex, real-world biological data.
• Biological data, such as grain images, present unique challenges due to multi-scale variability and entangled signals, especially for disease detection.
• Existing explainability methods struggle with biological data and lack standardized evaluation metrics.

Purpose of the Study:

• To evaluate post-hoc explainability methods for image classification of real-world grain data, focusing on disease and damage detection.
• To address challenges in applying and evaluating explainability in the context of complex biological image data.
• To propose a framework for assessing the robustness and quality of explanations for deep learning models in specific use cases.

Main Methods:

• Focused on image classification of grain data to detect diseases like "pink fusarium" and damages such as "skinned" grains.
• Evaluated various post-hoc explainability methods on grain datasets, assessing robustness, explanation quality, and similarity to expert-annotated ground truth.
• Discussed challenges in explainability, including hyperparameter sensitivity, visualization issues, and the lack of defined ground truth for evaluation.

Main Results:

• Standard explainability methods may not perform well on dissimilar biological images, requiring careful selection and evaluation.
• The study identified key challenges in evaluating explanation methods, particularly the absence of clear ground truth and potential discrepancies between human and model reasoning.
• A pipeline for evaluating explainability methods on specific, challenging datasets like grain images was proposed.

Conclusions:

• Applying machine learning to real-world biological data, like grain disease detection, requires specialized approaches beyond standard benchmarks.
• Robust evaluation of explainability methods is crucial for reliable deep learning applications in sensitive domains.
• The proposed framework aims to guide the selection and validation of effective explainability techniques for practical, high-stakes tasks.