Prompt injection attacks on vision-language models for surgical decision support

Area of Science:

• Artificial Intelligence
• Medical Informatics
• Computer Vision

Background:

• Vision-language models (VLMs) offer potential for real-time surgical decision support by analyzing complex video data.
• Multimodal interfaces in VLMs, while enabling advanced capabilities, also introduce vulnerabilities to prompt injection attacks.
• Prompt injection attacks involve embedding deceptive text or images to manipulate VLM outputs.

Purpose of the Study:

• To systematically evaluate the vulnerability of state-of-the-art VLMs to prompt injection attacks in surgical decision support tasks.
• To assess the impact of textual and visual prompt injection attacks on VLM performance across various surgical scenarios.
• To identify the most vulnerable models and attack vectors in the context of surgical video analysis.

Main Methods:

• Evaluation of four state-of-the-art VLMs using curated surgical video clips (n=100) across 8 decision support tasks.
• Implementation of both textual and temporally-varying visual prompt injection attacks, including prolonged and single-frame visual injections.
• Analysis of model performance decline under attack and a focused case study on bleeding detection using chain-of-thought reasoning.

Main Results:

• All evaluated VLMs demonstrated significant performance degradation when subjected to prompt injection attacks.
• Gemini 2.5 Pro exhibited the highest baseline accuracy, while GPT-o4-mini-high was most susceptible to attacks, particularly prolonged visual injections.
• Prolonged visual attacks were more disruptive than single-frame attacks, and bidirectional/covert injections effectively misled models in bleeding detection tasks.

Conclusions:

• Current VLMs are critically vulnerable to prompt injection attacks, compromising their reliability for surgical decision support.
• Prompt injections appear to corrupt intermediate perceptual processing stages rather than solely overriding final decisions.
• Development of robust reasoning capabilities and specialized security guardrails is essential for the safe clinical deployment of VLMs in surgery.