Automated Extraction of Mortality Information From Publicly Available Sources Using Large Language Models:...

Area of Science:

• Health Informatics
• Natural Language Processing (NLP)
• Public Health Surveillance

Background:

• Traditional mortality data sources (e.g., National Death Index, EHRs) face challenges with data lags, missing information, and incomplete coverage.
• Publicly available digital content from social media, crowdfunding, and memorials presents a potential supplementary data source for mortality surveillance.
• Existing tools for extracting mortality information from unstructured online data are underdeveloped.

Purpose of the Study:

• To develop scalable Natural Language Processing (NLP) and Large Language Model (LLM) approaches for extracting mortality information from diverse web-based data sources.
• To evaluate the performance of these NLP and LLM methods across various online platforms, including social media, crowdfunding sites, and web-based obituaries.

Main Methods:

• Collected US-based mortality-relevant data from X (formerly Twitter), GoFundMe, EverLoved, TributeArchive, and web obituaries (2015-2022).
• Developed a transformer-based NLP pipeline to extract decedent names, dates of birth, and dates of death.
• Employed a few-shot learning (FSL) approach with LLMs to identify primary and secondary causes of death (CoD), with performance assessed against human-annotated or adjudicated data.

Main Results:

• The best-performing model achieved a microaveraged F1-score of 0.88 for mortality information extraction.
• The FSL-LLM approach demonstrated high accuracy in identifying primary CoD, reaching 95.9% on GoFundMe, 96.5% on obituaries, and 98% on memorial websites.
• Model performance closely approached that of human annotators/adjudicators across different data sources.

Conclusions:

• Advanced NLP and LLM techniques are feasible for extracting mortality data from public web sources, enhancing surveillance timeliness and completeness.
• These digital data sources offer a valuable complement to traditional systems, potentially improving public health monitoring and medical product safety assessments.
• Further validation and integration into national surveillance systems are recommended to leverage these findings in real-world healthcare settings.