PerAnSel:  A  Novel Deep Neural Network-Based System for Persian Question Answering

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Machine Learning

Background:

• Question Answering (QA) systems are increasingly important for accessing information.
• Most research in QA and Answer Selection (AS) has focused on English, neglecting other languages.
• Persian presents unique challenges for QA due to its free word order, right-to-left script, morphological richness, and low-resource status.

Purpose of the Study:

• To address the lack of resources for Persian AS, this study introduces the first large-scale native Persian AS dataset (PASD).
• To develop and evaluate a novel deep learning system, PerAnSel, specifically designed for Persian question answering.
• To demonstrate the effectiveness of PASD and PerAnSel in advancing Persian QA research.

Main Methods:

• Creation of PASD, a comprehensive dataset for Persian answer selection.
• Development of PerAnSel, a deep neural network integrating sequential and transformer-based methods to handle Persian's free word order.
• Evaluation of PerAnSel on PASD and two other Persian QA datasets (PerCQA, WikiFA) using Mean Reciprocal Rank (MRR).

Main Results:

• PerAnSel achieved state-of-the-art performance on Persian AS tasks.
• The system outperformed existing methods by significant margins: 10.66% on PASD, 8.42% on PerCQA, and 3.08% on WikiFA (MRR).
• The PASD dataset proved effective for training and evaluating advanced QA systems.

Conclusions:

• The introduction of PASD fills a critical gap in Persian NLP resources.
• PerAnSel demonstrates the efficacy of hybrid deep learning approaches for morphologically rich, free-word-order languages.
• This work represents a significant advancement in Persian question answering capabilities.