PromptGuard a structured framework for injection resilient language models.

Area of Science:

• Artificial Intelligence
• Cybersecurity
• Natural Language Processing

Background:

• Prompt injection attacks pose a significant threat to the reliability and task fidelity of large language models (LLMs).
• Current defense mechanisms often lack adaptability, requiring retraining or having a narrow scope, hindering their practical deployment.
• The need for robust, adaptable, and lightweight defenses against adversarial attacks on LLMs is critical for secure AI applications.

Purpose of the Study:

• To develop and evaluate a modular, multi-layer defense framework to mitigate prompt injection attacks on LLMs.
• To enhance the robustness and reliability of LLMs against adversarial instructions without necessitating model retraining.
• To provide a practical and efficient solution for improving LLM security in real-world scenarios.

Main Methods:

• A four-layer defense framework integrating input gatekeeping, structured prompt formatting, semantic output validation, and adaptive response refinement (ARR).
• Utilized regex and MiniBERT for malicious instruction detection and blocking.
• Employed structured formatting and critic-based validation for consistent task alignment and output verification.

Main Results:

• The proposed framework demonstrated significant improvements in LLM robustness across multiple LLMs.
• Achieved up to a 67% reduction in prompt injection success rates.
• Obtained an F1-score of 0.91 for detection accuracy with a minimal latency increase below 8%.

Conclusions:

• The modular defense framework is an effective, lightweight, and retraining-free approach to enhance LLM safety and reliability.
• The system successfully mitigates prompt injection attacks, ensuring task fidelity and dependable AI behavior.
• This research offers a promising solution for securing LLMs against sophisticated adversarial manipulations in practical applications.