DesInsert: Strategic descriptive term insertion fools text-to-image generation.

Area of Science:

• Artificial Intelligence
• Computer Vision
• Machine Learning

Background:

• Text-to-image generation models (e.g., Stable Diffusion) are increasingly popular but vulnerable to adversarial attacks.
• Existing adversarial attacks often lack imperceptibility, introducing noticeable changes to input prompts.
• Deep neural network vulnerabilities persist in current generative models.

Purpose of the Study:

• To propose an imperceptible adversarial attack method for text-to-image models.
• To develop variants for both white-box and black-box attack scenarios.
• To enhance the stealthiness and efficiency of adversarial attacks on generative models.

Main Methods:

• Introduced Descriptive Term Insert (DesInsert) method with two variants: DesInsert-White and DesInsert-Black.
• DesInsert-White uses a discretized softmax approach for efficient white-box adversarial term discovery.
• DesInsert-Black employs a genetic encoding strategy in word space for black-box attacks.

Main Results:

• DesInsert consistently distorts generated images with minimal perceptible changes across popular text-to-image models.
• Achieved perplexity (PPL) values less than half of existing baselines.
• Demonstrated a 21.98% higher success rate and over 10x faster generation in white-box settings; superior performance in black-box settings with lower query costs.

Conclusions:

• The proposed DesInsert method enables effective and imperceptible adversarial attacks on text-to-image models.
• Highlights significant vulnerabilities in current text-to-image generation systems.
• Underscores the need for developing more robust generative models against such attacks.