Ad Hoc Information Extraction for Clinical Data Warehouses.

Area of Science:

• Clinical Informatics
• Natural Language Processing
• Health Data Science

Background:

• Clinical Data Warehouses (CDW) leverage Electronic Health Records (EHR) for research and patient recruitment.
• Unstructured clinical data, such as discharge letters, contain valuable information often inaccessible through standard data retrieval.
• Manual information extraction (IE) from clinical text is time-consuming and limits data usability.

Purpose of the Study:

• To develop an ad hoc IE service for querying unstructured clinical text within CDWs.
• To enable users to retrieve specific data points, including frequencies of concepts and numerical values, similar to querying structured data.
• To address challenges in extracting negated concepts and numerical data from clinical notes.

Main Methods:

• Implementation of an extended NegEx algorithm for German clinical texts to detect and scope negations.
• Enhancement of the PaDaWaN CDW with query functions for context-sensitive and regex queries.
• Development of an extraction mode to compute frequencies of Boolean and numerical values at runtime.

Main Results:

• High F1-scores achieved across three subtasks: negation detection (0.97-0.99), Boolean value extraction (approx. 0.99), and numerical value extraction (approx. 0.99).
• Evaluations conducted on chest X-ray reports and discharge letters demonstrated the system's effectiveness.
• The system successfully handles negation detection and extraction of both Boolean and numerical clinical concepts.

Conclusions:

• An ad hoc IE query feature for clinical data warehouses was successfully developed.
• The system offers high recall and precision in extracting Boolean and numerical values from clinical texts.
• The approach combines negation detection with on-the-fly concept extraction, providing a dynamic and adaptable solution for clinical data analysis.