The PIEE Cycle: A Structured Framework for Red Teaming Large Language Models in Clinical Decision-Making.

Area of Science:

• Medical Informatics
• Artificial Intelligence Safety
• Clinical Decision Support

Background:

• Large language models (LLMs) offer healthcare benefits but pose risks to patient safety, accuracy, and ethics.
• A standardized method for evaluating LLM safety in clinical decision-making is currently lacking.

Purpose of the Study:

• To introduce the PIEE cycle, a structured red-teaming framework for assessing AI safety in healthcare decision-making.
• To provide a systematic approach for stress-testing LLMs in clinical scenarios.

Main Methods:

• The PIEE cycle involves Planning and Preparation, Information Gathering and Prompt Generation, Execution, and Evaluation.
• Adversarial prompts (jailbreaking, social engineering, distractor attacks) are used to stress-test LLMs.
• Performance is evaluated using metrics like harm detection rates, hallucination rates (TruthfulQA), safety, reliability, bias (BBQ), and ethical scoring.

Main Results:

• The PIEE framework enables simulation of real-world clinical scenarios to test LLM robustness.
• Evaluation metrics provide quantitative and qualitative assessments of LLM performance.
• The framework is adaptable across medical specialties, illustrated with plastic surgery examples.

Conclusions:

• The PIEE cycle offers a practical foundation for evaluating the clinical reliability and ethical integrity of LLMs in medicine.
• While conceptual, the framework is intended for all medical providers to ensure safe AI integration.
• Ongoing validation is necessary to confirm the framework's efficacy.