Quantifying confidence shifts in a BERT-based question answering system evaluated on perturbed instances

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Machine Learning

Background:

• Transformer-based neural networks show significant progress in multiple-choice natural language processing (NLP) tasks, including Question Answering (QA).
• Systematic evaluation of these models in ambiguous scenarios, where no correct answer may be present, remains underexplored despite real-world relevance.

Purpose of the Study:

• To experimentally evaluate transformer-based QA models in ambiguous situations.
• To investigate how models perform when presented with prompts lacking correct answer choices.

Main Methods:

• Designed three probes to systematically 'confuse' QA instances by introducing perturbations.
• Conducted experiments using an established transformer-based multiple-choice QA system.
• Evaluated performance on two benchmark datasets.

Main Results:

• Transformer models exhibit confidence levels that diverge from expected behavior in ambiguous QA scenarios.
• Model confidence in incorrect choices suggests a lack of certainty in distinguishing correct from incorrect options.
• High performance on idealized QA tasks does not reliably predict performance on ambiguous instances.

Conclusions:

• Current transformer-based QA models may struggle with ambiguous situations, performing differently than expected.
• The models' inability to reliably distinguish ambiguous from unambiguous contexts highlights limitations.
• Enhanced testing protocols and benchmarking are crucial before deploying these models in critical applications with human impact.