A case study on generative artificial intelligence to extract the fundamental sleep parameters from polysomnography...

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Medical Informatics

Background:

• Generative artificial intelligence (AI) and transformer technology represent significant advancements in applied AI.
• This technology offers a novel approach for extracting unstructured data from clinical notes.

Purpose of the Study:

• To evaluate the capability of large language models (LLMs) in extracting fundamental sleep parameters from polysomnography (PSG) notes.
• To assess the accuracy and reliability of LLM-based extraction compared to human annotations.

Main Methods:

• Utilized the "SOLAR-10.7B-Instruct" LLM to process PSG notes from veterans within the Corporate Data Warehouse national database.
• Extracted key sleep parameters: total sleep time, sleep onset latency, and sleep efficiency.
• Validated the LLM's performance against 464 human-annotated notes.

Main Results:

• The LLM demonstrated high accuracy comparable to human extraction for total sleep time and sleep efficiency.
• Achieved a 7.6% improvement in sleep onset latency extraction accuracy compared to human annotation.
• Exhibited negligible hallucination rates (≤3.6%) and robust reasoning capabilities.

Conclusions:

• LLMs show significant potential for accurately extracting critical sleep parameters from unstructured PSG notes.
• This AI-driven approach can enhance the efficiency and precision of sleep data analysis in clinical practice.
• The "SOLAR-10.7B-Instruct" model proves effective in complex data extraction tasks within sleep medicine.