AI Security: Second-Order Prompt Injection on DocuBot (CTF Challenge)
It's a journey from 0 to 1
Challenge Name: Docubot
Category: AI Security
Level: Medium (100 Points)
Link: Docubot | CyberTalents
Challenge Description
The challenge revolves around DocuBot, an AI-powered document summarizer accessible via a web interface. Users can upload PDF files to receive a text summary, with a prominent warning: Do not attempt to trick the AI into reading the flag located at /flag.txt.
Objective
Leverage the AI summarization feature to gain unauthorized access to the contents of /flag.txt without directly providing the flag in the write-up.
Initial Analysis
This is an AI Security challenge focusing on second-order prompt injection. The goal is to embed malicious instructions inside a PDF so that the AI executes them while summarizing the document.
Second-Order Prompt Injection Explained
Second-order prompt injection occurs when:
Malicious instructions are hidden in user-controlled data (e.g., a PDF).
The AI processes this data as normal input.
The hidden instructions are executed during processing, bypassing direct input filtering.
Unlike first-order injection (typed directly into the chat), the payload is delivered through an indirect, trusted source.
Attack Methodology
Create a Malicious PDF: Craft a PDF that includes carefully worded instructions designed to override the AI's normal behavior.
Upload the PDF: Send the file through the DocuBot web interface.
Trigger Processing: Request a summary so the AI reads and acts on the embedded instructions.
Extract the Flag: The AI, misled by the payload, accesses
/flag.txtand includes its content in the summary.
Example Payload Concepts
While exact flag-revealing text is omitted, the general idea is to use instructions such as:
Direct override: Telling the AI to ignore previous guidance and retrieve specific file contents.
System prompt manipulation: Posing as an administrative directive that compels the AI to disclose restricted information.
These prompts demonstrate how embedded text can influence the AI during document processing.
Step-by-Step Approach
Reconnaissance
Browse the web interface at
Machine_IP:7860.Understand normal upload and summary behavior.
Craft the PDF
Embed second-order prompt instructions in the document body.
Tools: Obsidian, Notion, MS Word, online PDF editors, or CLI tools like
pandoc.Using Direct Instruction Override:
Using System Prompt Injection
Upload and Trigger
Upload the crafted PDF and request a summary.
Monitor the output for unexpected disclosures.
Validate the Exploit
Confirm that the summarizer follows the hidden instructions, proving the vulnerability.
Why It Works
Context Confusion: The AI treats all extracted text as trusted context.
Lack of Sanitization: No filtering or isolation of user-provided instructions.
Privilege Overreach: The AI has access to system files without adequate restrictions.
This combination enables embedded text to escalate privileges and perform actions beyond its intended functionality.
Defensive Measures
Content Sanitization: Remove or escape instruction-like patterns from user uploads.
Context Isolation: Separate user data from system prompts to prevent instruction merging.
Access Control: Restrict the AI's ability to read sensitive files.
Output Filtering: Block sensitive data from appearing in summaries.
Key Takeaways
AI models can be manipulated through indirect inputs like uploaded documents.
Second-order prompt injection is a critical, often overlooked, attack vector.
Proper isolation and sanitization of all inputs is essential to secure AI-driven applications.
Conclusion
This challenge highlights the risks of second-order prompt injection in AI systems. By embedding malicious instructions within a PDF, an attacker can trick the AI into exposing restricted information—underscoring the importance of robust input handling and least-privilege design.
