Artificial Intelligence in Health Care: A Rallying Cry for Critical Clinical Research and Ethical Thinking

Area of Science:

• Medical Informatics
• Artificial Intelligence in Healthcare
• Generative Artificial Intelligence (GenAI)

Background:

• AI is poised to significantly impact various job sectors, including healthcare, necessitating careful implementation due to potential algorithmic biases and suboptimal patient treatment.
• While AI research in healthcare has historically focused on diagnostic radiology and digital pathology, emerging applications in radiation oncology require evidence of clinical utility beyond efficiency.
• Rapid advancements in Generative AI (GenAI), particularly large language models (LLMs) like GPT-4, present new possibilities and challenges in medical applications.

Purpose of the Study:

• To advocate for independent academic research to establish evidence-based applications of AI in medicine.
• To highlight the critical need for understanding the strengths and limitations of AI tools in healthcare.
• To address the ethical considerations surrounding the integration of GenAI in patient care and the healthcare provider relationship.

Main Methods:

• Review of current AI applications in healthcare, with a focus on radiation oncology.
• Discussion of the potential impact of GenAI and LLMs on medical practice.
• Analysis of challenges including algorithmic bias, data representation, deskilling, and ethical boundaries.

Main Results:

• Existing AI applications in healthcare often lack robust evidence of clinical utility.
• GenAI, including LLMs, offers personalized and contextually relevant outputs but requires rigorous validation.
• Concerns exist regarding the potential for AI to exacerbate health disparities for under-represented minorities and rare cases.

Conclusions:

• Independent, evidence-based research is essential for the safe and effective implementation of AI in medicine, treating AI as a medical intervention.
• A significant educational effort is required for healthcare professionals to understand and critically assess AI tools.
• Ethical guidelines must be established to govern the use of GenAI in patient care, particularly concerning the patient-provider relationship.