Trie-based rule processing for clinical NLP: A use-case study of n-trie, making the ConText algorithm more efficient...

Area of Science:

• Natural Language Processing (NLP)
• Computational Linguistics
• Bioinformatics

Background:

• Rule-based clinical NLP offers transparency and knowledge integration but faces efficiency challenges in big data.
• Machine learning NLP methods are popular but often require extensive annotation.
• Processing speed is a critical bottleneck for standard rule-based NLP in large datasets.

Purpose of the Study:

• To develop and evaluate an efficient Trie structure, named n-trie, for large-scale, rule-based clinical NLP.
• To address the processing bottlenecks in clinical NLP pipelines, particularly in context detection.
• To improve the efficiency and scalability of rule-based NLP systems.

Main Methods:

• Developed n-trie, a novel rule processing engine utilizing a revised Trie structure for fast execution of lexicon-based NLP rules.
• Implemented the n-trie engine in FastContext for clinical NLP context detection.
• Compared the performance (speed and accuracy) of n-trie against JavaConText, GeneralConText, and a machine learning NegEx implementation (NegScope).

Main Results:

• The n-trie engine demonstrated performance two orders of magnitude faster than comparison implementations.
• N-trie showed less sensitivity to rule set size compared to other methods.
• Accuracy consistently improved with increased rule sets, a desired outcome not observed in other implementations.
• N-trie outperformed the best machine learning negation detector in speed.

Conclusions:

• The n-trie engine provides an efficient and scalable solution for NLP rule processing.
• N-trie shows significant potential for application in various NLP tasks beyond context detection.
• This development addresses a key limitation in the adoption of rule-based NLP for big data clinical applications.