Closed Domain Semantic Question Answering System as a Use Case of Transformer-Based BERT Models

To extract knowledge from text files and answer users' questions by finding the right information in context, an adaptive learning system, such as a Question Answering System (QAS), is designed for this purpose. This focus encourages further study on direct-answer systems and the use of large-scale tests for Question Answering (QA) tasks. To facilitate this, a Semantic Closed-domain QA (SCD-QA) dataset, which encompasses both factoid and non-factoid questions, is employed in conjunction with pre-trained transformer models. In this study, the ability to make inferences is measured and compared between three pre-trained transformer models, like Google-BERT, DistilBERT, and RoBERTa, on the SQuAD dataset, and a classic keyword-based TF-IDF model with cosine similarity. Results show Google-BERT performs best, with a mean Exact Match (EM) score of 90.0 and a mean latency of 1.27 s. The system also performs well on questions with synonyms, showing a strong understanding of meaning. But it performs inadequately on questions with spelling errors, indicating it is sensitive to misspellings and requires better early processing in use.