A human-LLM collaborative annotation approach for screening articles on precision oncology randomized controlled trials
These videos have been matched automatically. Contact us if they are not relevant.
These videos have been matched automatically. Contact us if they are not relevant.
View abstract on PubMed
Summary
This summary is machine-generated.This study introduces a human-large language model (LLM) collaborative annotation method to improve article screening efficiency for systematic reviews. The approach significantly reduces workload while maintaining high reliability, aiding researchers in evidence synthesis.
Area Of Science
- Bibliometrics
- Artificial Intelligence in Research
Background
- Supervised learning accelerates systematic review screening but requires extensive manual annotation.
- Large language models (LLMs) offer rapid screening but face reliability challenges.
- Developing efficient and reliable annotation methods is crucial for systematic reviews.
Purpose Of The Study
- To design an efficient and reliable human-LLM collaborative annotation method for article screening.
- To optimize LLM performance for high recall in identifying relevant articles.
- To validate the efficiency and reliability of the collaborative annotation approach.
Main Methods
- A human-LLM collaborative annotation strategy focusing on verifying LLM-identified positive samples was developed.
- Prompts were iteratively optimized using a manually annotated dataset to achieve near-perfect LLM recall.
- The optimized LLM screened articles, followed by human verification of positive annotations, applied to precision oncology RCT screening.
Main Results
- Iterative prompt optimization achieved near-perfect LLM recall (100% tuning, 85.71% validation).
- The collaborative method achieved an F1 score of 0.9583 and reduced annotation workload by ~80%.
- A BioBERT model trained on collaborative data outperformed one trained on LLM-only data, confirming reliability.
Conclusions
- The human-LLM collaborative annotation method enhances efficiency and reliability in article screening.
- This approach offers significant support for conducting systematic reviews and meta-analyses.
- The findings highlight the potential of AI-assisted methods in scientific literature evaluation.
These videos have been matched automatically. Contact us if they are not relevant.
These videos have been matched automatically. Contact us if they are not relevant.
Related Concept Videos
Combining two or more treatment methods increases the life span of cancer patients while reducing damage to vital organs or tissue from the overuse of a single treatment. Combination therapy also targets different cancer-inducing pathways, thus reducing the chances of developing resistance to treatment.
The combination of the drug acetazolamide and sulforaphane is a good example of combination therapy to treat cancer. The cells in the interior of a large tumor often die due to the hypoxic and...
The targeted cancer therapies, also known as “molecular targeted therapies,” take advantage of the molecular and genetic differences between the cancer cells and the normal cells. It needs a thorough understanding of the cancer cells to develop drugs that can target specific molecular aspects that drive the growth, progression, and spread of cancer cells without affecting the growth and survival of other normal cells in the body.
There are several types of targeted therapies against...