Classifying Clinical Evidence Levels of Cancer Variants in Biomedical Literature Using Machine Learning and Large...

Automating the classification of clinical evidence levels in biomedical literature can support precision oncology by facilitating the acceleration of variant interpretation and informed decision-making. This study compares the performance of two state-of-the-art large language models (LLMs) (GPT-4.1-mini and Gemini-2.5-Flash) and two machine learning (ML) algorithms (decision tree and XGBoost) for classifying publications according to the Clinical Interpretation of Variants in Cancer (CIViC) evidence level system. Zero- and few-shot prompting strategies were tested for LLMs, while Term Frequency-Inverse Document Frequency (TF-IDF) and word embedding representations were evaluated for ML models. XGBoost with TF-IDF achieved the highest performance (micro-F1 = 0.83), outperforming both LLMs and decision trees. All models performed best on mid-range evidence levels (B to D) and struggled with high (A) and inferential (E) levels, reflecting dataset imbalance and linguistic ambiguity. These findings suggest that, at present, abstract-level evidence classification is largely driven by explicit lexical cues, with limited added benefit from standalone LLM-based approaches.