Related Experiment Video
Updated: Jan 12, 2026
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
1.0K
Temporal evolution of large language models (LLMs) in oncology
Zilin Qiu1, Aimin Jiang2, Chang Qi3
1Department of Oncology, Zhujiang Hospital, Southern Medical University, Donghai County People's Hospital (Affiliated Kangda College of Nanjing Medical University), Lianyungang, 222000, China.
Journal of Translational Medicine
|November 4, 2025
Summary
Large language models (LLMs) in oncology show performance decline over time, with Gemini improving while ChatGPT models degrade. This impacts evidence-based LLM deployment in cancer care.
Area of Science:
- Artificial Intelligence in Oncology
- Natural Language Processing in Medicine
- Machine Learning for Healthcare
Background:
- Large language models (LLMs) are increasingly used in healthcare, but their accuracy in specialized fields like oncology can be affected by knowledge decay and concept drift.
- The temporal dynamics influencing LLM performance in oncology question-answering require systematic evaluation.
- This study assesses the evolution of LLM accuracy for oncology-related queries using real-world data.
Purpose of the Study:
- To systematically evaluate the temporal performance trends of different LLMs in answering oncology-related questions.
- To compare the accuracy evolution of ChatGPT-3.5, ChatGPT-4, and Gemini in oncology.
- To provide insights for the evidence-based deployment of LLMs in clinical oncology settings.
Main Methods:
- A systematic literature search was conducted through 2025 using keywords related to LLMs and cancer research.
- A dataset of 614 oncology research questions (subjective and multiple-choice) was curated.
- Accuracy of responses from ChatGPT-3.5, ChatGPT-4, and Gemini was evaluated over time using standardized scoring methods and statistical analysis (random/fixed effects models).
Main Results:
- ChatGPT-3.5 and ChatGPT-4 demonstrated declining performance in both subjective and objective oncology question assessments over time.
- Gemini showed significant performance improvements over the study period for both question types.
- ChatGPT-3.5 experienced a notable performance shift, degrading over time, particularly for subjective oncology questions.
Conclusions:
- Meta-analysis indicates temporal performance degradation in ChatGPT-3.5 and ChatGPT-4 within the oncology domain.
- Gemini exhibited consistent performance improvement over time, contrasting with the degradation observed in other models.
- Findings offer crucial guidance for the judicious and evidence-based integration of LLMs into oncology practice.
More Related Videos
Related Concept Videos
02:07Improving Translational Accuracy
3.5K
3.5K
02:07Improving Translational Accuracy
14.0K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.0K
01:22Language Development
825
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
825
02:07Tumor Progression
7.2K
Tumor progression is a phenomenon where the pre-formed tumor acquires successive mutations to become clinically more aggressive and malignant. In the 1950s, Foulds first described the stepwise progression of cancer cells through successive stages.
Colon cancer is one of the best-documented examples of tumor progression. Early mutation in the APC gene in colon cells causes a small growth on the colon wall called a polyp. With time, this polyp grows into a benign, pre-cancerous tumor. Further...
Colon cancer is one of the best-documented examples of tumor progression. Early mutation in the APC gene in colon cells causes a small growth on the colon wall called a polyp. With time, this polyp grows into a benign, pre-cancerous tumor. Further...
7.2K
01:21Cancer Survival Analysis
639
Cancer survival analysis focuses on quantifying and interpreting the time from a key starting point, such as diagnosis or the initiation of treatment, to a specific endpoint, such as remission or death. This analysis provides critical insights into treatment effectiveness and factors that influence patient outcomes, helping to shape clinical decisions and guide prognostic evaluations. A cornerstone of oncology research, survival analysis tackles the challenges of skewed, non-normally...
639
02:43Mouse Models of Cancer Study
6.4K
Mice have long served as models for studying human biology and pathology because of their phylogenetic and physiological similarity with humans. They are also easy to maintain and breed in the laboratory, and hence, many inbred strains are now available for research. Studies on mice have contributed immeasurably to our understanding of cancer biology.
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
The development of transgenic, knockout, and knock-in mice has led to an exponential increase in their use as model organisms in research,...
6.4K