Prompt injection attacks on vision-language models for surgical decision support

Area of Science:

• Artificial Intelligence in Surgery
• Computer Vision
• Medical Decision Support

Background:

• AI-driven analysis of surgical videos can enhance safety and precision in minimally invasive procedures.
• Vision-language models (VLMs) offer advanced capabilities for interpreting complex video data in surgical contexts.
• VLMs are susceptible to prompt injection attacks, posing risks to clinical applications.

Purpose of the Study:

• To systematically assess the vulnerability of state-of-the-art VLMs to textual and visual prompt injection attacks.
• To evaluate VLM performance on clinically relevant surgical decision support tasks under adversarial conditions.

Main Methods:

• Evaluated four VLMs (Gemini 1.5 Pro, Gemini 2.5 Pro, GPT-o4-mini-high, Qwen 2.5-VL) on eleven surgical decision support tasks.
• Simulated prompt injection attacks using misleading text and visual overlays at varying durations.
• Measured model accuracy by comparing performance under baseline and prompt injection conditions.

Main Results:

• All VLMs showed reduced accuracy due to prompt injections; prolonged visual injections were more detrimental.
• Gemini 2.5 Pro exhibited the highest baseline accuracy (0.82) and demonstrated superior robustness against attacks.
• GPT-o4-mini-high was most vulnerable, with accuracy dropping from 0.67 to 0.24 under full-duration visual injection (P < .001).

Conclusions:

• Current VLMs are susceptible to prompt injection attacks, compromising their reliability for surgical decision support.
• Robust temporal reasoning and specialized safety guardrails are essential for safe real-time deployment of VLMs in surgery.
• Findings underscore the need for rigorous security evaluations before integrating AI into clinical workflows.