Modified Bidirectional Encoder Representations From Transformers Extractive Summarization Model for Hospital...

Area of Science:

• Medical Informatics
• Natural Language Processing
• Artificial Intelligence

Background:

• Healthcare professionals face challenges managing numerous patients due to time-consuming medical history reviews.
• Discharge diagnoses contain vital patient information but are often excessively verbose, hindering efficient data handling.
• Deep learning offers a solution but typically requires substantial computational resources, posing a barrier for some systems.

Purpose of the Study:

• To develop an extractive summarization model for patient discharge diagnoses tailored for hospital information systems.
• To create a service that is efficient and operable even with limited computing power.
• To enhance clinical workflow by providing concise diagnostic summaries.

Main Methods:

• A Bidirectional Encoder Representations from Transformers (BERT)-based model, AlphaBERT, was developed using a two-stage training approach.
• The model utilized 258,050 discharge diagnoses from the National Taiwan University Hospital Integrated Medical Database, with doctor-annotated summaries as labels.
• Model size was significantly reduced via character-level tokens, decreasing parameters from over 108 million to under 1 million, and fine-tuned for optimal summarization performance.

Main Results:

• AlphaBERT achieved a high area under the receiver operating characteristic curve of 0.947, comparable to other models but with a drastically reduced size.
• Recall-Oriented Understudy for Gisting Evaluation (ROUGE) L scores indicated competitive summarization performance against BERT, BioBERT, and Long Short-Term Memory (LSTM).
• Doctor feedback via a questionnaire website showed AlphaBERT's summaries were well-received, with critique scores close to doctor-generated labels.

Conclusions:

• Employing character-level tokens in a BERT architecture effectively minimizes model size without substantial performance degradation for diagnoses summarization.
• The developed deep-learning model, AlphaBERT, can significantly improve doctors' patient management capabilities.
• This technology facilitates medical research by enabling efficient analysis of extensive unstructured clinical notes.