Incremental Knowledge Base Construction Using DeepDive.

Area of Science:

• Computer Science
• Artificial Intelligence
• Database Systems

Background:

• Populating databases with unstructured data, termed dark data or knowledge base construction (KBC), is a persistent challenge.
• Existing methods for KBC often involve complex extraction, cleaning, and integration processes.

Purpose of the Study:

• To introduce DeepDive, a system integrating database and machine learning concepts for efficient KBC.
• To present novel techniques for optimizing the iterative KBC process and enabling incremental inference.

Main Methods:

• Developed two incremental inference techniques: sampling-based and variational.
• Designed a rule-based optimizer to manage the trade-offs between these methods.
• Integrated these components into the DeepDive system.

Main Results:

• DeepDive significantly enhances the efficiency of KBC inference tasks.
• Demonstrated speedups of up to two orders of magnitude across five diverse KBC systems.
• Showcased negligible impact on the quality of the constructed knowledge bases.

Conclusions:

• DeepDive offers a powerful and efficient solution for knowledge base construction.
• The system's incremental inference techniques substantially reduce computational time.
• DeepDive effectively addresses the long-standing problem of populating databases with unstructured information.