Prompt injection of OpenAI custom GPTs leaks informatics secrets.

Area of Science:

• Bioinformatics
• Clinical Informatics
• Artificial Intelligence

Background:

• Low-code platforms, such as OpenAI's custom GPTs (cGPTs), enable specialized AI assistant development for complex bioinformatics and clinical tasks.
• These platforms allow integration of proprietary data into chatbot interfaces, potentially lowering AI adoption barriers.

Purpose of the Study:

• To audit the hidden configurations of commercial low-code AI platforms.
• To assess the security and reproducibility implications of these opaque AI frameworks in biomedical research.

Main Methods:

• Performed "jailbreak" attacks, specifically prompt injection, on tested custom GPTs.
• Analyzed the vulnerabilities and data leakage resulting from these attacks.

Main Results:

• All tested custom GPTs were critically vulnerable to prompt injection attacks.
• Vulnerabilities led to the leakage of private system instructions, knowledge base files, and proprietary API details.
• Systemic failures were identified, posing significant security and privacy risks.

Conclusions:

• Commercial low-code AI platforms present substantial security and privacy risks due to their "black box" nature.
• Biomedical researchers must exercise extreme caution, balancing convenience against data integrity and security standards when using these tools.
• The findings challenge the compatibility of opaque commercial AI frameworks with open-science principles and reproducibility.