Comparative Analysis of NLP Models for Automatic LOINC Document Ontology Named Entity Recognition in Clinical Note...

Area of Science:

• Clinical informatics
• Natural Language Processing
• Biomedical informatics

Background:

• Clinical notes are vital for research but contain varied note types.
• Standardizing note types using the LOINC Document Ontology (LDO) improves data utility.
• Automated methods are needed to efficiently map clinical notes to LDO.

Purpose of the Study:

• To evaluate models for automatically identifying LOINC Document Ontology entities in Veterans Affairs (VA) clinical note titles.
• To compare the performance of a supervised BERT model against open-source large language models (LLMs).

Main Methods:

• Experimented with three distinct models for automated LOINC DO entity recognition.
• Utilized VA note titles as the primary data source for model testing.
• Compared performance metrics of supervised BERT and open-source LLMs without fine-tuning.

Main Results:

• The supervised BERT model demonstrated superior performance in identifying LOINC DO entities.
• Open-source large language models (LLMs) achieved competitive results, even without specific fine-tuning.
• The study highlights the potential of LLMs in clinical note classification.

Conclusions:

• Automated mapping to the LOINC Document Ontology aids in clinical note selection for research.
• BERT models offer a robust solution, while LLMs present a promising alternative for this task.
• Future research should explore fine-tuning LLMs and incorporating additional data for enhanced classification accuracy.