Leveraging sequence-to-sequence models for semantic annotation of Dutch pathology reports

Area of Science:

• Medical informatics
• Computational pathology
• Natural Language Processing (NLP)

Background:

• Pathology report annotation is crucial for patient care and research but is manual, time-consuming, and error-prone.
• The Palga Foundation indexes Dutch pathology data using manual annotations from conclusion texts, mapping them to the Palga thesaurus.
• Automating this annotation process can improve efficiency and accuracy.

Purpose of the Study:

• To investigate the use of Text-To-Text Transfer Transformer (T5)-based models for automated annotation of Dutch pathology reports.
• To compare a standard multilingual T5 model (mT5) with a custom T5 model (PaTh5.NL) pre-trained on Palga data.
• To evaluate the impact of constrained decoding (CD) versus default decoding (DD) on annotation performance.

Main Methods:

• Developed a custom T5 model (PaTh5.NL) pre-trained on Palga data.
• Fine-tuned both mT5 and PaTh5.NL models using default decoding (DD) and constrained decoding (CD).
• Assessed performance using Bilingual Evaluation Understudy (BLEU) scores and case-based evaluations for patient retrieval.

Main Results:

• Fine-tuned PaTh5.NL models significantly outperformed mT5 on shorter histology and cytology reports.
• All models showed decreased performance on longer or more complex pathology reports.
• Case-based evaluations indicated that higher BLEU scores did not always translate to better patient retrieval by PaTh5.NL models compared to mT5.

Conclusions:

• Fine-tuned T5 models can enhance Dutch pathology report annotation, especially for specific report types.
• Challenges persist with complex conclusion texts, particularly in histology and autopsy reports.
• Future work should focus on larger datasets and post-processing algorithms to improve annotation generalization.