Enhancing Question Answering by Injecting Ontological Knowledge through Regularization.

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Knowledge Representation

Background:

• Deep neural networks excel at text-based tasks but struggle with those needing external knowledge.
• Current models lack robust mechanisms for integrating real-world information.
• This limitation hinders performance in complex reasoning and question answering.

Purpose of the Study:

• To introduce Ontology-based Semantic Composition Regularization (OSCR) for injecting task-agnostic knowledge into neural networks.
• To improve the ability of deep learning models to leverage external knowledge during pre-training.
• To enhance performance on natural language processing tasks requiring world knowledge and domain-specific information.

Main Methods:

• Developed OSCR, a novel method for integrating knowledge graphs or ontologies into neural network pre-training.
• Pre-trained BERT models on Wikipedia, with and without the OSCR method.
• Fine-tuned the pre-trained models on diverse question answering tasks, including those involving world knowledge, causal reasoning, and healthcare domain knowledge.

Main Results:

• BERT models pre-trained with OSCR demonstrated significant accuracy improvements on downstream tasks.
• Achieved 33.3% and 18.6% increased accuracy on question answering tasks requiring world knowledge and causal reasoning, respectively.
• Observed a 4% improvement in accuracy for a healthcare domain-specific question answering task.

Conclusions:

• OSCR effectively injects external knowledge into neural networks, enhancing their capabilities.
• The method shows promise for improving deep learning performance on knowledge-intensive NLP tasks.
• OSCR represents a valuable advancement for natural language processing, particularly in areas requiring broad or specialized knowledge.