VetTag: improving automated veterinary diagnosis coding via large-scale language modeling

Area of Science:

• Veterinary Medicine
• Natural Language Processing
• Machine Learning

Background:

• Veterinary records are predominantly unstructured free text, lacking standardized diagnosis coding.
• This data format hinders the use of veterinary records for public health and translational research.
• Previous machine learning efforts were limited to predicting only 42 broad diagnosis categories.

Purpose of the Study:

• To develop a large-scale algorithm for automatically predicting all 4577 standard veterinary diagnosis codes from free-text clinical notes.
• To improve the utility of veterinary records for public health and translational research.
• To explore the application of advanced machine learning techniques in veterinary clinical data.

Main Methods:

• Developed a novel algorithm based on an adapted Transformer architecture.
• Trained the algorithm on a large dataset including over 100,000 expert-labeled and over one million unlabeled veterinary notes.
• Utilized large-scale language modeling through pretraining and an auxiliary objective during supervised learning.
• Employed hierarchical training to address data imbalances for fine-grained diagnoses.

Main Results:

• Successfully developed an algorithm to predict all 4577 standard veterinary diagnosis codes from free text.
• Demonstrated significant performance improvements through large-scale language modeling on unlabeled data.
• Evaluated model performance in challenging cross-hospital settings with substantial domain shift.
• Showcased the effectiveness of hierarchical training for rare or fine-grained diagnoses.

Conclusions:

• The developed algorithm effectively addresses the challenge of systematic coding in veterinary medicine.
• The study highlights the power of unsupervised learning and advanced NLP techniques for clinical data.
• This work facilitates leveraging veterinary records for broader public health and research applications.