Using generative artificial intelligence to standardize unstructured antigen profiles for the alloantibody exchange.

Area of Science:

• Medical Informatics
• Computational Biology
• Bioinformatics

Background:

• Standardized data is crucial for national transfusion history sharing services like the alloantibody exchange.
• Red cell antigen profiles are often unstructured text, hindering data standardization.
• Manual curation of antigen profiles is labor-intensive and limits scalability.

Purpose of the Study:

• To evaluate the efficacy of large language models (LLMs) in standardizing unstructured free-text red cell antigen profiles.
• To compare the performance of locally run open-weight LLMs versus commercially available LLMs accessed via the internet.
• To assess the accuracy, scalability, and cost-effectiveness of LLM-based standardization for transfusion data.

Main Methods:

• Utilized deidentified red cell antigen profiles from an academic center.
• Employed schema-guided decoding and error-aware retries for LLM output.
• Compared performance metrics including exact profile match, antigen-level agreement, error types, and processing costs.
• Statistically analyzed differences using Cochran's Q and pairwise McNemar tests with Holm correction.

Main Results:

• Commercial LLMs significantly outperformed open-source LLMs in standardizing antigen profiles.
• The top-performing commercial LLM achieved 92.5% exact match for antigen profiles and 99.4% antigen-level agreement.
• Open-source LLMs showed improved performance with a higher number of parameters.
• Estimated API costs for commercial LLMs were modest for the evaluated workload.

Conclusions:

• Commercial LLMs, when combined with schema-guided decoding and error-aware retries, can accurately standardize unstructured antigen profiles.
• LLM-based standardization offers significant scalability and reduces manual curation efforts for transfusion history services.
• LLMs represent a foundational technology for enabling efficient and accurate national transfusion history sharing.