Model interpretability enhances domain generalization in the case of textual complexity modeling

Area of Science:

• Machine Learning
• Natural Language Processing
• Cognitive Science

Background:

• Balancing prediction accuracy, model interpretability, and domain generalization is a key challenge in machine learning.
• Evaluating model performance across different data distributions (out-of-distribution testing) is crucial for real-world applicability.

Purpose of the Study:

• To investigate the trade-offs between model interpretability and domain generalization.
• To assess the performance of interpretable versus opaque models on tasks involving textual complexity and reader processing difficulty.
• To identify strategies for enhancing model robustness and generalization.

Main Methods:

• Tuned 77,640 configurations, selecting 120 interpretable and 166 opaque models.
• Included large language models like ChatGPT and probabilistic models.
• Employed a two-task approach: text classification for textual complexity and generalization to predict processing difficulty.

Main Results:

• Confirmed the accuracy-interpretability trade-off in the initial text classification task.
• Demonstrated that interpretable models significantly outperformed opaque models in domain generalization (task 2).
• Found that multiplicative interactions further improved interpretable models' domain generalization.

Conclusions:

• Advocating for the use of big data in training, augmented by external theories for interpretability.
• Emphasizing the importance of small, well-designed out-of-distribution datasets for validating model generalization and robustness.
• Highlighting interpretable models as key to achieving reliable domain generalization.