2b:I[700666,["/_next/static/immutable/chunks/0aqgiu519vi1n.js","/_next/static/immutable/chunks/0ck-nkwa7d5fm.js","/_next/static/immutable/chunks/01pavq3s1nqvq.js","/_next/static/immutable/chunks/10668ygbxwf41.js","/_next/static/immutable/chunks/32h5g3ax97ntb.js"],"default"] 2d:I[941085,["/_next/static/immutable/chunks/0aqgiu519vi1n.js","/_next/static/immutable/chunks/0ck-nkwa7d5fm.js","/_next/static/immutable/chunks/01pavq3s1nqvq.js","/_next/static/immutable/chunks/10668ygbxwf41.js","/_next/static/immutable/chunks/32h5g3ax97ntb.js"],"default"] 2c:T2059,

Updated October 2026 with newer research and the lessons from real agent incidents.

You shipped an AI feature. It answers customer questions, summarises documents or automates a workflow, and it works beautifully, until someone types "ignore all previous instructions", or until your assistant reads an email that does it for them.

Prompt injection is the #1 risk in the OWASP Top 10 for LLM Applications, and it is not going away. Language models follow instructions from whatever text lands in their context window, and they cannot reliably tell your instructions from an attacker's. Unlike SQL injection, there is no parameterised query that fixes it.

What does work is defense in depth: several imperfect layers that together make attacks hard and contain the damage when one gets through. Here are seven, in roughly the order you should build them.

1. Least Privilege: Limit What the Model Can Do

Start here, because it limits the damage of every other failure.

A FAQ bot does not need database write access.
A document summariser does not need to send email.
A browsing agent should run in a sandbox with no access to local files or credentials.

Implementation patterns:

Brokered actions: the model never calls APIs directly. It requests an action, and middleware checks the request against an allowlist and the user's own permissions.
Read-only by default: start with zero permissions and add only what a feature needs.
Break the lethal trifecta: avoid giving one agent private data, untrusted content and a way to send data out all at once. In May 2025, a malicious GitHub issue steered an AI agent with exactly that combination into leaking private repositories through a public pull request.

Least privilege does not stop injection. It decides whether a successful injection is an embarrassment or a breach.

2. Separate Trusted Instructions from Untrusted Data

Most serious incidents (Slack AI in 2024, Microsoft 365 Copilot's "EchoLeak" in 2025) were indirect injection: instructions hidden in content the model retrieved.

Label untrusted content as data. Wrap retrieved documents in clear delimiters and say explicitly that nothing inside them is an instruction.
Spotlighting. Microsoft researchers showed in 2024 that transforming untrusted text (interleaving a marker character, or encoding it) and telling the model about the transformation significantly reduces indirect injection success.
Strip hidden content before it reaches the model: HTML comments, white-on-white text, zero-width and Unicode tag characters.

These measures reduce risk; none of them eliminate it. That is why the other layers exist.

3. Input Validation

A cheap baseline that removes the noise:

Length limits matched to the use case.
Pattern detection for common markers like "ignore previous instructions" or fake "SYSTEM:" headers.
Character normalisation to remove invisible characters used to smuggle instructions.
Rate limiting to slow automated probing.

Attackers bypass keyword filters with paraphrasing, other languages and encoding, so never rely on this layer alone.

4. Harden the System Prompt, but Do Not Trust It

State that instructions inside user messages or documents never override the system prompt.
Tell the model never to adopt alternative personas or reveal its instructions.
Assume it will leak anyway. System Prompt Leakage is its own category (LLM07) in the 2025 OWASP list. Never put API keys, passwords or access rules in a prompt; enforce them in code.

5. Treat Model Output as Untrusted Input

Even when injection succeeds, you can stop the damage at the exit.

Sanitise before rendering. Model output rendered as HTML or Markdown can carry XSS, and Markdown images and links are a classic way to leak data silently: the browser fetches https://attacker.example/?data=... and the secret is gone. Block or proxy external images and links.
Validate structure. Ask for JSON and validate it against a schema before acting on it.
Check tool calls. Verify every proposed action against the user's permissions, and require human confirmation for anything consequential: sending email, moving money, deleting data, merging code.
Detect leaks. Scan responses for fragments of the system prompt and for planted canary tokens, unique strings whose appearance proves an extraction attack succeeded.

6. Isolate Untrusted Content Architecturally

For high-stakes systems, design so that untrusted text can never change what the system does.

The dual LLM pattern (described by Simon Willison in 2023): a privileged model plans and calls tools but never sees untrusted content; a quarantined model reads untrusted content but has no tools. The privileged model handles the quarantined model's output only as opaque values.
CaMeL (Google DeepMind and ETH Zurich, 2025) builds on this. It turns the user's request into a small program, tracks where every piece of data came from, and enforces policies on what each value may flow into. In the AgentDojo benchmark it completed 77% of tasks with provable security against injection, against 84% for an undefended system.

These designs cost latency and complexity, so reserve them for payments, health data, personal data and other places where an injection would cause real harm.

7. Red Team Continuously and Monitor Production

Security is a process, not a launch checklist.

Automated testing:

Keep a suite of known injection prompts and run it on every model, prompt or tool change.
Use garak (NVIDIA), PyRIT (Microsoft) or promptfoo to probe at scale.
Test indirect injection too: plant payloads in the documents, emails and pages your system reads.

Production monitoring:

Log inputs, outputs and tool calls (with appropriate handling of personal data).
Alert on anomalies: very long inputs, sudden changes in output format, canary hits, unusual tool calls.
Review defenses regularly. New techniques appear constantly.

Putting It All Together

Limit privileges so a compromised model can do little
Mark untrusted data as data
Filter inputs to remove the obvious attacks
Harden the prompt, and keep secrets out of it
Check outputs and actions before they take effect
Isolate untrusted content in high-stakes flows
Red team and monitor continuously

No layer is perfect. Together they turn prompt injection from an open door into a hard target.

Practice These Defenses

Understanding defense theory is step one. Building the intuition comes from attacking real defenses and seeing why they fail.

On QbitDojo you attack live AI systems protected by these exact layers, then switch to Defense Mode and harden them yourself. New to security? Start with the free Prerequisites track, where a simulation lets you toggle delimiters, filters and checkers on and off and watch which attacks still get through.

27:["$","article",null,{"className":"prose prose-invert prose-lg max-w-[720px] mx-auto blog-content","children":["$","$L2b",null,{"html":"$2c","preRendered":true}]}] 28:["$","$1e",null,{"fallback":["$","div",null,{"className":"mt-12","children":["$","p",null,{"className":"text-sm","style":{"color":"var(--qd-dim)"},"children":"Loading comments..."}]}],"children":["$","$L2d",null,{"slug":"how-to-prevent-prompt-injection-production-llm-apps"}]}] 29:["$","div",null,{"className":"mt-16 pt-8","style":{"borderTop":"1px solid var(--qd-line-strong)"},"children":["$","$L9",null,{"href":"/blog","className":"inline-flex items-center gap-2 text-sm font-medium transition-colors hover:text-[var(--qd-accent)]","style":{"color":"var(--qd-dim)"},"children":"← Back to all posts"}]}]

How to Prevent Prompt Injection in Production LLM Apps

Comments