Accurately Identifying Cerebroarterial Stenosis from Angiography Reports Using Natural Language Processing Approaches

Area of Science:

• Medical informatics
• Computational linguistics
• Neurology

Background:

• Intracranial artery stenosis is a significant risk factor for stroke.
• Angiography reports contain valuable data for diagnosing cerebrovascular diseases.
• Current methods for identifying stenosis from reports are underutilized.

Purpose of the Study:

• To evaluate natural language processing (NLP) techniques for detecting eleven types of intracranial artery stenosis.
• To compare the performance of rule-based, recurrent neural network (RNN), and XLNet models.

Main Methods:

• Developed and validated three NLP models: rule-based, RNN (LSTM), and XLNet.
• Utilized angiography reports from two medical centers for training, internal validation (9614 reports), and external validation (315 reports).
• Assessed model performance using the area under the receiver operating characteristic curve (AUROC).

Main Results:

• XLNet achieved the highest performance with an AUROC of 0.97-0.99 for eleven targeted arteries.
• The rule-based model yielded an AUROC of 0.92-0.96.
• The RNN model achieved an AUROC of 0.95-0.97.

Conclusions:

• NLP techniques, particularly XLNet, show promise for automated screening of patients at high risk for intracranial artery stenosis.
• Challenges remain in model generalization due to variations in report styles and data distribution.