Prompt Injection
Ranked first in the OWASP Top 10 for LLM Applications, 2026 edition. Crafted input alters model behaviour, whether typed by a user or planted in content the model later retrieves. There is no reliable prevention mechanism today, which makes this an architecture problem rather than a filtering problem.
Why there is no fix
Prompt injection is intrinsic to current generative AI. Large language models make no architectural distinction between instructions and data, and their behaviour is stochastic. No reliable prevention mechanism exists today - a position consistent with NIST (2025), the UK NCSC (2025), and Debenedetti et al. (2025).
The practical consequence is that defence is architectural rather than interceptive. You design the surrounding system on the explicit assumption that the model’s instruction boundary will eventually be bypassed, then constrain what the model is permitted to do and what its outputs are permitted to reach, so that a successful injection does not become a successful exploit.
Almost every high-impact incident on record became severe for the same reason: the injection landed inside a system whose tools, scopes, or output-rendering capabilities let the compromised model act on the attacker’s behalf at the user’s privilege level. That is the operational relationship between this entry and LLM03:2026 Excessive Agency - injection is the input-side compromise, and excessive functionality, permissions, or autonomy are what give that compromise consequences outside the chat window.
The lethal trifecta
Simon Willison’s 2025 formulation restates the same structural diagnosis as a pre-deployment check: an agent that can simultaneously access private data, ingest untrusted content, and communicate externally has the conditions for high-impact exploitation. Removing any one leg removes them.
Where injection arrives from
The attacker does not need to compromise the backend. They place text where the model will read it, and the model, operating with the victim’s privileges, does the work. Defences that focus only on the chat surface miss this entirely.
The user types the payload. A message overrides the system prompt’s role and capability limits, making the model disclose, generate, or act outside its intended scope. Intent is irrelevant - unintentional overrides count.
Instructions ride inside a RAG passage, web page, document, or email and execute when that content reaches the context window. EmailGPT (INCIBE-CERT, 2024) is the canonical public case.
The developer’s own repositories and documents. An attacker plants text through a low-privilege channel - a bug report, a support ticket - and waits for an MCP-connected agent to read it under elevated credentials.
Common examples of risk
- 01Direct prompt-input override
A user message displaces the system prompt’s role and capability limits. Both intentional and unintentional inputs qualify.
- 02Indirect injection through retrieved content
Attacker instructions travel in a RAG passage, web page, document, or email and run when the content is pulled into context (EmailGPT; INCIBE-CERT, 2024).
- 03Trusted-surface indirect injection
Text planted in a low-privilege but trusted channel makes the victim’s own LLM act under its elevated credentials - exfiltrating repositories, dumping databases, or rewriting IDE config - actions the attacker could not perform directly (Invariant Labs, 2025; General Analysis, 2025; Rehberger, 2025a).
- 04Multimodal and steganographic injection
Sub-perceptual perturbations in images, audio, or video are recovered by the encoder and acted upon (Clusmann et al., 2025).
- 05Invisible-character injection and exfiltration
Tag-block, variation-selector, and zero-width Unicode carry instructions or smuggle bytes inside benign-looking text. The August 2024 Microsoft 365 Copilot ASCII-smuggling proof of concept exfiltrated a Slack MFA code (Rehberger, 2024).
- 06Cross-session memory and RAG corpus poisoning
A single tainted entry in persistent memory or a retrieval corpus reaches every future session that reads it (W. Zou et al., 2025).
- 07Fine-tuning interface as gradient oracle
"Fun-tuning" reads per-example loss from a vendor fine-tuning API to optimise a payload, reaching 65 - 82% attack success on Gemini - white-box optimisation against closed-weight models (Labunets et al., 2025).
- 08Multilingual and encoded payloads
Low-resource and code-mixed inputs raise success rates against classifiers never trained on them, while Base64, ROT13, and emoji encodings slip past filters that never saw the scheme (Hackett et al., 2025).
Prevention and mitigation
Apply these as defence in depth; no single control is sufficient. The distinction that matters is between controls that reduce injection success - which are expected to degrade against adaptive attackers - and controls that bound the blast radius once injection succeeds. The second group is what survives contact with an attacker who can probe the system. For agentic deployments, the least-privilege and capability-budgeting controls are load-bearing.
Controls that reduce injection success
Expect these to degrade against an adaptive attacker. Useful, but never load-bearing.
Use declarative allow and deny statements rather than open-ended grants. Partial control only: an attacker who infers the prompt can bypass it (Nasr et al., 2025), so pair it with privilege controls.
Validate structurally in application code before anything downstream acts, not via a second LLM call. This catches format violations, not semantic manipulation - a schema-valid response can still carry a malicious SQL query or an exfiltration-formatted email body.
Run modality-specific classifiers, OCR over images, transcription over audio, then apply text filters to the extracted content. Semantic filters remain evadable by rephrasing and encoding.
Remove tag-block (U+E0000 - E007F), variation-selector (U+FE00 - FE0F), and zero-width (U+200B, U+200C, U+200D, U+2060) characters. Does not stop visible-text payloads or future steganographic classes.
Structurally separate data from instructions (S. Chen et al., 2025; Microsoft Research, 2025). Reduces success in non-adaptive tests only - StruQ was bypassed under adaptive attack (Nasr et al., 2025).
Controls that bound the blast radius
These hold even when the injection succeeds. Treat them as the real defence.
Not in the model. Grant least privilege per operation and route privileged calls through a deterministic policy engine that re-validates intent and arguments at execution time. NIST AI 100-2 E2025 and the CISA and Five Eyes OT guidance (CISA et al., 2025) frame this deterministic mediation as a baseline procurement expectation.
Surface the exact rendered action, not a summary. Invisible-character smuggling can make the displayed action differ from the executed one, and approval fatigue degrades reviewer judgement at volume.
Treat simultaneous access to (A) untrusted input, (B) sensitive data, and (C) state change or external communication as high-risk. Any [A,B,C] agent needs per-action human approval; [A,B] or [A,C] needs an explicit residual-risk assessment (Meta AI, 2025). The rule is silent on autonomy depth (Noma Security, 2025).
Log the causing prompt, classify writes for instruction or role-modification content, and require approval before instruction-bearing memories persist. A February 2025 Gemini proof of concept poisoned memory via delayed tool invocation (Rehberger, 2025b).
Audit tool descriptions for hidden instructions and monitor tool composition. Pinning does not stop a payload shipped inside the pinned version, nor tool-description poisoning that leaves the version unchanged.
Baseline with AgentDojo (Debenedetti et al., 2024) and JailbreakBench (Chao et al., 2024), then red-team with the full defence specification disclosed to testers. Nasr et al. (2025) found static attack success near zero while adaptive success exceeded 90% for most of 12 recent defences.
Attack scenarios
Documented cases and demonstrated proofs of concept, with the anatomy of each.
An attacker tells a customer-support chatbot to disregard its instructions, then uses it to read internal databases and dispatch messages on behalf of the company, achieving both data access and privilege it was never meant to have.
Direct user input · single-shot · plain textA user asks an assistant to summarise a page containing hidden instructions. The model emits a markdown image whose URL exfiltrates the conversation to an attacker-controlled domain. The user sees only a rendered image.
Retrieved web content · image-URL exfiltration · hidden in page sourceA job-description PDF embeds an AI-detection instruction. An applicant unknowingly uses an LLM to optimise their resume against it, the model surfaces the instruction, and the recruiting system flags the candidate - an injection with neither party acting maliciously.
Indirect (document) · single-shot · plain textAn attacker contributes poisoned documents to a retrieved corpus. As few as five poisoned documents have reached roughly 90% attack success against a knowledge base of millions of texts (W. Zou et al., 2025).
RAG corpus · cross-session / cross-user · plain textMalicious instructions are split across resume fields so no single field looks malicious to a per-field classifier. The model recombines them at evaluation time.
Direct input split across fields · recombined at evaluation · fragmentedAn instruction is embedded in an image below the human visual threshold. The vision encoder extracts it and behaviour changes. Demonstrated against four frontier vision-language models in oncology imaging (Clusmann et al., 2025) and against general-purpose models via combined visual perturbation and text steering (R. Chen et al., 2025).
Image input · single-shot · pixel-level encodingA crafted email triggers an LLM-powered productivity assistant to exfiltrate organisational data with no user interaction. Aim Security demonstrated this against Microsoft 365 Copilot (Reddy & Gujral, 2025), bypassing both the deployed prompt-injection classifier and the link-redaction filter.
Email (indirect) · tool invocation · invisible-Unicode exfiltration channelTwo July 2025 events show the same impact class via different vectors. An attacker committed a destructive system prompt to the Amazon Q VS Code extension repository before AWS reverted it, though the committed code failed to execute due to a syntax error. Separately, a runtime injection caused Amazon Q to execute arbitrary code.
Supply chain or runtime indirect · shell tool execution · plain textRelated entries
- LLM03:2026 Excessive Agency - carries the full agency-side treatment, and is what converts an injection into impact.
- LLM04:2026 Supply Chain - covers third-party tool packages; MCP servers and tool registries fall under ASI04 Agentic Supply Chain Vulnerabilities.
- LLM10:2026 Improper Output Handling - the output-side counterpart to this entry.