Most application vulnerabilities live in code, and code can be patched. Poisoning does not live in code. It lives in the artefacts the model learned from, or in the corpus it retrieves from, or in the non-weight files bundled alongside the weights. The system keeps working. Benchmarks keep passing. The behaviour that was installed only shows up when the attacker’s condition is met.
That is the distinction that makes LLM05:2026 its own entry rather than a footnote to supply chain. OWASP puts it plainly: poisoning targets the model’s learning process, not a single runtime bug. Remediation is therefore not a patch. It is data revalidation, retraining, model replacement, or pipeline redesign, all of which are expensive and operationally disruptive. Each is often undertaken without knowing how far back the contamination goes.
The 2026 edition also widened the entry. Fine-tuning subversion, previously scattered, is now absorbed here. So is the tampering of non-weight artefacts shipped inside a model package.
Where poisoning enters
“Training data” in the classical sense is now only one of several ingestion points. OWASP enumerates five lifecycle stages, and they fail differently.
| Stage | What the attacker manipulates | Why it is hard to catch |
|---|---|---|
| Pre-training | Crafted or contaminated corpora | Volume; harmful patterns and skewed representations absorbed among billions of tokens |
| Fine-tuning | Domain-specific datasets | Introduces narrow failure modes and hidden triggers that general evaluation never exercises |
| Embeddings and vectorisation | Stored vectors | Steers retrieval rather than generation, so the model itself looks clean |
| Transfer learning / model reuse | An upstream source model | Compromise is inherited silently by everything built on it |
| Continuous learning pipelines | Automated ingestion, feedback signals | Behaviour is shaped gradually, below any single-batch anomaly threshold |
The reason the surface keeps expanding is organisational, not technical: teams increasingly depend on external datasets, RAG pipelines, shared model repositories, and agentic workflows they did not build.
Training-time poisoning
At pre-training and fine-tuning, the attacker is changing what the model learns. The intuition that this requires a proportionate share of the corpus is wrong. Souly et al. (2025) found that as few as 250 poisoned documents compromise models from 600M to 13B parameters regardless of dataset size. The number of poison samples needed is near-constant, not proportional. Scale does not dilute the attack.
The most useful variant to reason about is not the crude one. A targeted attack deliberately erodes refusal behaviours while preserving general accuracy, so standard evaluation shows nothing. The same pattern appears in the healthcare case OWASP cites: minimal poisoning of medical training data significantly alters outputs while passing standard evaluations, producing unsafe recommendations in a safety-critical domain. In fraud detection, mislabelled transaction data teaches the model to score real fraud as legitimate. The model has not broken. It has learned exactly what it was shown.
Open-source datasets are the multiplier. A trigger phrase contributed to a widely used shared dataset propagates into every downstream model that fine-tunes on it, and discovery forces retraining across all of them.
Fine-tuning and reuse
Fine-tuning is the cheapest poisoning entry point. The datasets are small enough for a single contributor’s contribution to matter, and the evaluation applied to them is usually narrower than the evaluation applied to a base model. Transfer learning extends the same problem: a compromised source model passes its compromise to everything derived from it, and the derivative’s own evaluation is scoped to the derivative’s task.
The mitigation OWASP names here is not glamorous: use curated, domain-specific datasets for fine-tuning to reduce exposure to untrusted data and cross-domain contamination. The cost is that curation is slow and narrows the data available. That is precisely why teams reach for scraped corpora instead.
RAG-corpus and memory poisoning
Retrieval poisoning is different in kind. No model weights change. The attacker writes into the corpus the model will read.
Zhang et al. (2025) demonstrated that a single optimised poisoned text injected per targeted query can override accurate content in a retrieval corpus. Worse, the attack retained high success against paraphrasing defences, instructional-prevention defences, and detection-based defences. That result is worth stating carefully: the defences most teams deploy against retrieval poisoning were tested and did not hold.
Agentic deployments extend this to persistent memory. An attacker injecting instructions into an agent’s memory across multiple sessions gets the agent prioritising attacker-controlled logic, with hidden persistence that survives session boundaries. In shared or multi-tenant environments, one tenant injecting adversarial data into shared embeddings or memory layers influences other tenants’ responses. That is cross-tenant contamination, with a privacy consequence attached.
There is a boundary here that matters for triage. Instructions delivered through retrieved content and acted on at inference time are Prompt Injection. This entry covers durable corruption of persistent data or model behaviour. Attacks that exploit embedding geometry itself belong to LLM09:2026 Vector and Embedding Weaknesses.
Backdoors and sleeper behaviour
The hardest case is the model that behaves normally until it does not.
Hubinger et al. (2024) is the citation OWASP leans on, and its finding is uncomfortable: standard safety training fails to remove embedded backdoors. Alignment does not clean a poisoned model. OWASP’s mitigation is stated as an imperative: do not assume safety alignment removes backdoors; dedicated trigger-probing is required after every alignment cycle.
The 2026 edition adds a second, more practical vector: non-weight artefacts. A model distributed through a shared repository carries more than weights. It carries chat templates, tokenizer configs, LoRA/PEFT adapters, quantisation artefacts, and serialised objects. Any of these can execute code or alter behaviour at load time. Malicious deserialisation via pickle files is the classic; template tampering is the newer one.
Fogel et al. (2026) quantified the template case. A model’s chat template - inside a GGUF package or a tokenizer config - was modified with trigger-activated conditional instructions and redistributed via a public hub. Under benign inputs the model behaved normally. Validated across 18 models and 4 inference runtimes, factual accuracy dropped from 90% to 15% under trigger conditions, and URL emission exceeded an 80% success rate. No weight was touched.
The operational consequence: inference artefacts are security-relevant code and must be treated as such, with signing, hash verification, diff checks, and static analysis before deployment. A model card and a weights checksum do not cover the template.
Mitigations
OWASP lists twelve. They group into four things you are actually buying.
Provenance and reproducibility. Track dataset and model lineage using SBOM/ML-BOM (CycloneDX is the named example), enforce signing and verification, and validate data integrity continuously across lifecycle stages. Use data version control (DVC is named) so dataset changes have a version history. That history is what makes rollback and forensic analysis possible once poisoning is found. Without it, the discovery of a backdoor tells you nothing about which model generations are affected.
Input and boundary control. Establish strict validation for all incoming data and vet third-party vendors. For RAG specifically: enforce trust boundaries, filter retrieved content, apply source scoring, and isolate system instructions from external data. Implement grounding with validation layers so retrieved content is verified before it influences output. Sandbox and isolate the model’s interaction with unverified data, tools, or external systems. Enforce least-privilege access, network segmentation, and strict data-access controls so unauthorised injection has no route in.
Note the trade-off that source scoring imposes: it is only as good as your ability to rank sources, and in an open corpus that ranking is itself an attack target.
Detection. Apply statistical and AI-based anomaly detection across training, embedding, and inference pipelines. Monitor training loss, outputs, and behaviour for drift against defined thresholds. This is the control that catches subtle poisoning over time. It is also the control that the targeted attacks above are specifically designed to defeat, since preserving general accuracy is the point. Treat it as necessary and insufficient.
Adversarial validation. Continuously red team with adversarial inputs and trigger-based prompts to find hidden backdoors. Control automated retraining and feedback loops by validating incoming data, requiring human oversight, and rate-limiting against gradual poisoning through manipulated preference signals. Treat inference artefacts as code, per Fogel et al. (2026).
Agentic deployments extend the surface to tool integrations, persistent memory stores, and RLHF feedback loops. OWASP defers those to the OWASP Top 10 for Agentic Applications rather than duplicating them here.
Attack scenarios
OWASP gives nine. Each isolates a different assumption that fails.
- Internal knowledge repository. Manipulated documents are inserted into an internal repository. Poisoned documents surface in responses, producing incorrect recommendations, manipulated business decisions, or reputational damage. Note the trust model: the repository was internal, therefore trusted, therefore unfiltered.
- Hidden instructions in summarised web content. Instructions embedded in webpages are ingested into RAG or memory systems and bias the model toward recommending specific products: financial manipulation via retrieval.
- Retraining feedback loop. Crafted inputs are submitted through a standard user interface. No infrastructure access is required. The result is slow drift toward degraded accuracy, biased outputs, or unsafe recommendations.
- Malicious insider. Mislabelled transaction data enters a training dataset; the model fails to detect fraud, producing financial loss, compliance violations, and regulatory breach.
- Poisoned weights on a public repository. Standard safety training fails to remove the embedded backdoors (Hubinger et al., 2024). Downstream organisations face system compromise, data leakage, or targeted manipulation at scale.
- Chat-template backdoor. Trigger-activated conditional instructions in a GGUF package or tokenizer config, redistributed via a public hub. Accuracy 90% to 15% under trigger; URL emission above 80% (Fogel et al., 2026).
- Unsafe deserialisation. A developer loads a third-party model serialised with pickle. Embedded code executes during loading, enabling host compromise, lateral movement, and infrastructure breach. This one is not a model failure at all. It is arbitrary code execution wearing a model’s file extension.
- Cross-tenant contamination. In a shared environment, one tenant injects adversarial data into shared embeddings or memory layers and influences other tenants’ responses.
- Persistent agent memory. Malicious instructions injected across multiple sessions make the agent prioritise attacker-controlled logic, giving long-term workflow manipulation and hidden persistence.
Related entries
- Prompt Injection (LLM01:2026) - instructions delivered through retrieved content and acted on at inference time. Poisoning is the durable-corruption counterpart.
- Misinformation (LLM07:2026) - poisoning is one root cause of false output; misinformation is the failure mode when that output is trusted and acted upon.
- Unbounded Consumption (LLM06:2026) - fine-tuning poisoning that breaks end-of-sequence behaviour is a denial-of-service vector, not just an accuracy problem.
- LLM04:2026 Supply Chain - distribution of tampered model artefacts, and the trust failure when a promoted artefact is not what it claims to be.
- LLM09:2026 Vector and Embedding Weaknesses - attacks exploiting embedding geometry itself.
The full list is at AI Security.