Identifying Biomedical Entities for Datasets in Scientific Articles: 4-Step Cache-Augmented Generation Approach Using...

Area of Science:

• Biomedical Informatics
• Computational Biology
• Data Science

Background:

• Accurate extraction of biomedical entities is crucial for effective metadata annotation of research datasets.
• This ensures data findability, accessibility, interoperability, and reusability in collaborative research.

Purpose of the Study:

• To introduce a novel 4-step cache-augmented generation approach for automated biomedical entity identification.
• Leveraging GPT-4o and PubTator 3.0 for dataset metadata annotation.

Main Methods:

• A 4-step process: GPT-4o for candidate entity generation, PubTator 3.0 for validation, schema-based term extraction, and combined evaluation.
• Applied to 23 articles from the Collaborative Research Center OncoEscape, validated via author interviews and random-effects meta-analysis.

Main Results:

• Generated a mean of 19.6 schema-related and 6.7 PubTator-validated biomedical entities per article.
• Achieved an overall annotation precision of 98% (95% CI 94%-100%), with errors mainly in non-basic research articles.
• Including supplemental material did not improve precision (98%, 95% CI 95%-100%).

Conclusions:

• Large language models show significant potential for supporting metadata annotation workflows.
• The findings support the practical feasibility of full-text analysis for routine biomedical metadata generation.