Multilingual multi-aspect explainability analyses on machine reading comprehension models

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Machine Learning

Background:

• Pre-trained language models (PLMs) achieve high performance on machine reading comprehension (MRC) tasks.
• The internal workings and explainability of PLMs, particularly their attention mechanisms, remain poorly understood.
• Understanding attention is crucial for advancing explainable AI in NLP.

Purpose of the Study:

• To investigate the relationship between multi-head self-attention mechanisms and the performance of PLM-based MRC systems.
• To reveal potential explainability insights within PLM-based MRC models.
• To conduct multilingual analyses across various PLMs for robust findings.

Main Methods:

• Conducted a series of analytical experiments on PLM-based MRC models.
• Performed multilingual experiments using various PLMs to ensure robustness.
• Utilized comprehensive visualizations and case studies of attention maps.

Main Results:

• Identified passage-to-question and passage understanding attentions as critical components in the question-answering process.
• Demonstrated strong correlations between these specific attentions and final MRC system performance.
• Observed generalizable findings regarding attention map patterns across different models and languages.

Conclusions:

• Multi-head self-attention, specifically passage-to-question and passage understanding, significantly influences PLM-based MRC performance.
• Attention maps offer valuable insights into how PLMs process information for question answering.
• This research contributes to understanding the explainability of PLMs in MRC tasks.