Deceiving question-answering models: A hybrid word-level adversarial approach.

Area of Science:

• Natural Language Processing (NLP)
• Artificial Intelligence (AI)
• Machine Learning (ML)

Background:

• Deep learning powers advanced NLP tasks like question-answering (QA).
• Robustness of QA models against adversarial attacks is a critical, underexplored concern.

Purpose of the Study:

• Introduce QA-Attack, a novel word-level adversarial strategy.
• Evaluate the effectiveness of this strategy in fooling QA models.

Main Methods:

• An attention-based attack exploiting customized attention mechanisms.
• Deletion ranking strategy to identify and target specific words.
• Synonym substitution to create deceptive inputs while preserving grammar.

Main Results:

• QA-Attack successfully deceives baseline QA models across various question types.
• Demonstrates versatility, especially with long textual inputs.
• Outperforms existing adversarial techniques in success rate, semantic changes, BLEU score, fluency, and grammar error rate.

Conclusions:

• QA-Attack is a versatile and effective strategy for evaluating QA model robustness.
• Highlights the vulnerability of current QA models to sophisticated adversarial attacks.