Automating Precision Oncology Literature Curation: A Decision Tree Approach Using Large Language Models

Area of Science:

• Oncology
• Bioinformatics
• Artificial Intelligence

Background:

• Precision oncology knowledge bases require extensive manual literature screening.
• Large Language Models (LLMs) show promise for automating knowledge extraction but haven't been evaluated for this task.
• Automating abstract screening is crucial for efficient knowledge base curation.

Purpose of the Study:

• To develop and evaluate an LLM-based pipeline for automating literature screening in precision oncology.
• To assess the performance of the pipeline in identifying relevant biomarker-therapeutic relationships.

Main Methods:

• A decision tree-based pipeline utilizing LLMs was developed.
• Structured prompts were used to evaluate biomarker-therapeutic relationships in 8,012 abstracts from 2024 oncology journals.
• Performance was benchmarked against expert review using standard metrics.

Main Results:

• The pipeline achieved 92.9% overall accuracy (87.1% sensitivity, 93.7% specificity).
• It shortlisted 1,328 abstracts with a 5.26-fold enrichment in classification precision, identifying 888 relevant ones.
• Identified abstracts covered key molecular targets, immunotherapies, targeted therapies, and cancer types.

Conclusions:

• The LLM pipeline effectively automates literature triage for precision oncology knowledge bases.
• It maintains high specificity and offers a scalable solution for sustainable knowledge base maintenance.
• This approach can significantly reduce manual screening burden.