CoQUAD: a COVID-19 question answering dataset system, facilitating research, benchmarking, and practice

Area of Science:

• Information Retrieval
• Natural Language Processing
• Biomedical Informatics

Background:

• The exponential growth of COVID-19 research presents a significant challenge for medical experts and researchers seeking to stay current with findings.
• A critical need exists for efficient tools to help researchers and practitioners mine COVID-19 literature and address emerging questions.

Purpose of the Study:

• To introduce CoQUAD, an advanced question-answering system designed for efficient information extraction from COVID-19 literature.
• To provide a system capable of answering questions across the spectrum of COVID-19 research, from early to post-pandemic topics.

Main Methods:

• CoQUAD utilizes a two-component architecture: a Retriever (BM25 algorithm) for document retrieval and a Reader (MPNet Transformer model) for answer extraction.
• The system was trained and evaluated on two datasets: a reference-standard dataset (CORD-19, LitCOVID) and a gold-standard dataset curated by public health experts.

Main Results:

• CoQUAD demonstrates high accuracy in providing effective and relevant answers to natural language COVID-19-related questions.
• The system achieved superior performance compared to state-of-the-art baselines, with an exact match ratio of 77.50% and an F1 score of 77.10%.

Conclusions:

• CoQUAD is an effective question-answering system leveraging natural language processing to mine COVID-19 literature.
• The system assists the research community in accessing the most recent findings and obtaining timely answers to their queries.