Related Experiment Video
Updated: Jan 7, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
A scoping review: how evaluation methods shape our understanding of ChatGPT's effectiveness in healthcare
Yuanyuan Liu1, Yu Zhang1, Haoran Mao1
1School of Foreign Studies, China University of Petroleum (East China), No. 66 West Changjiang Road, Huangdao District, Qingdao, Shandong Province 266580, China.
Background:
The rapid growth in research on ChatGPT's healthcare applications has led to diverse evaluation methods and substantially heterogeneous findings, undermining evidence reliability and hindering clinical translation.
Objectives:
This review aims to examine how different evaluation methods shape our understanding of ChatGPT's effectiveness in healthcare.
Methods:
Studies published between 2023 and 2024 that assess the use of ChatGPT in medical or healthcare-related contexts were included. Evidence was obtained from peer-reviewed literature analyzing ChatGPT's applications across clinical, educational, and diagnostic domains. Following the PRISMA guidelines, this systematic review analyzed 131 studies published during 2023-2024 that assess the use of ChatGPT in medical contexts.
Results:
The results indicate that predominant evaluation approaches-controlled trial studies, expert assessment studies, measurement-based evaluation studies, and prompt generation analysis studies-systematically influence conclusions about ChatGPT's performance due to their inherent methodological characteristics, such as subjectivity, objectivity, and differences in ecological validity. Further analysis reveals that ChatGPT's performance is highly context-dependent, shaped by specific application scenarios, model versions, and prompting strategies.
Conclusions:
To address methodological heterogeneity and the lack of standardization, this study recommends multi-method cross-validation strategies and a risk-stratified, standardized evaluation framework. These steps are essential to enhance the scientific rigor and reliability of ChatGPT's assessment in healthcare and to provide a solid foundation for its clinical integration.
More Related Videos
13:44Project-Based Learning Guidelines for Health Sciences Students: An Analysis with Data Mining and Qualitative Techniques
Published on: December 9, 2022
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
Related Concept Videos
Methods of Documentation VI: Case Management Model
For example, a patient with a chronic...
Nursing Process for Patient and Caregiver Teaching III: Evaluation and Documentation
Nurses can use several methods to evaluate patient outcomes. For example, oral questions can assess cognitive learning,...
Patient-centered Care
Methods of Documentation III: PIE
Methods Of Healthcare Delivery System
Managed Care System:
The managed care system is designed to control the cost while maintaining the quality of care. The patient's care from admission to discharge is planned by the primary care provider or the case manager, also known as the gatekeeper. In a managed care system, the number of care providers is...
Therapeutic Communication
Verbal communication depends on language or a prescribed way of using words so that people can share information effectively. The critical aspects of verbal...