State-of-the-Art Evidence Retriever for Precision Medicine: Algorithm Development and Validation

Area of Science:

• Biomedical Informatics
• Computational Biology
• Oncology

Background:

• Precision medicine (PM) enables personalized therapies based on patient-specific clinical and genetic data.
• The rapid growth of scientific literature makes it challenging for clinicians to find relevant evidence for personalized treatments.
• Accessing comprehensive clinical evidence is crucial for effective evidence-based precision medicine.

Purpose of the Study:

• To develop and evaluate the PM-Search system for retrieving clinical literature relevant to precision medicine therapies.
• To facilitate the identification of evidence supporting or refuting specific cancer therapy recommendations.
• To improve the efficiency and accuracy of evidence retrieval for practicing evidence-based precision medicine.

Main Methods:

• The PM-Search system integrates a large-scale baseline retriever using query expansion and keyword matching (ElasticSearch).
• An evidence reranker employs pretrained language models fine-tuned on expert annotations derived from active learning.
• The system prioritizes document candidates based on the quality of clinical evidence presented.

Main Results:

• PM-Search demonstrated superior performance in retrieving high-quality clinical evidence at the Text Retrieval Conference PM Track 2020.
• Achieved top scores in normalized discounted cumulative gain (0.4780) and exponential normalized discounted cumulative gain (0.4519) at rank 30.
• Significantly outperformed competing systems in identifying relevant and high-quality evidence for precision medicine.

Conclusions:

• PM-Search is a state-of-the-art search engine designed to support evidence-based precision medicine.
• A novel active learning strategy using Bidirectional Encoder Representations from Transformers for Biomedical Text Mining models evidence quality.
• Evidence quality is a critical factor distinct from general relevance, necessitating specialized modeling for effective PM search.