Enhancing LLM-based medical decision-making by test-time knowledge acquisition.

Area of Science:

• Artificial Intelligence in Medicine
• Clinical Decision Support Systems
• Machine Learning for Healthcare

Background:

• Medical decision-making (MDM) relies on integrating complex knowledge and evidence.
• Current large language models (LLMs) for MDM have limitations due to static training data and poor domain adaptation.
• This compromises diagnostic accuracy and reliability in clinical settings.

Purpose of the Study:

• To develop a framework enabling LLMs to dynamically acquire and refine knowledge during test time.
• To enhance the robustness and precision of LLM-based MDM systems.
• To overcome limitations of static training corpora in LLMs for medical applications.

Main Methods:

• A test-time optimization framework refines a frozen LLM's diagnostic reasoning.
• The model uses test-time knowledge acquisition and integration.
• Self-consistency scores differentiate confident and unconfident cases for targeted knowledge extraction and refinement, updating a dynamic knowledge base without parameter changes.

Main Results:

• The proposed framework consistently improved the performance of a state-of-the-art LLM (DeepSeekv3.2 Exp 671B) on medical decision-making benchmarks.
• On the MMLU-Pro-Health dataset, the method achieved 79.22% accuracy, an improvement of 1.84 percentage points.
• Demonstrated effectiveness in enhancing diagnostic decision-making accuracy and reliability.

Conclusions:

• Introduced a new paradigm for reliable, adaptive, and context-aware medical AI systems through inference-time self-evaluation and experience accumulation.
• Highlighted the importance of continual knowledge evolution for trustworthy AI in clinical decision support.
• Laid the foundation for dynamic and responsive medical reasoning tools.