Supply Chain

Traditional software supply-chain work is about code flaws and dependencies. In machine learning the risk extends to third-party pre-trained models, datasets, and model artifacts, which can be manipulated through tampering, poisoning, or malicious artifact replacement. The attack surface spans every stage where external components enter the pipeline: training data, base models, adapters, format-conversion tools, and the platforms that serve them. A compromise at any of these points can introduce bias, open a backdoor, or degrade the system silently.

Building LLMs is specialised work that depends on third-party models, datasets, and reusable adapters built with parameter-efficient fine-tuning (PEFT) techniques such as LoRA (Low-Rank Adaptation) and shared on platforms like Hugging Face. LLM04:2026 therefore treats model artifacts, provenance, and conversion/merge workflows as first-class attack surfaces, and notes that on-device LLMs widen the surface further to manufacturing, firmware, and app repackaging.

Two scope boundaries the source draws explicitly:

• There is deliberate overlap with LLM05:2026 Data and Model Poisoning. The split is about which half of the problem is in view: how a tainted artefact reaches you belongs here, what the taint does once it is inside belongs there.
• Supply-chain risks specific to agentic applications, including MCP servers and tool registries, are covered by ASI04 Agentic Supply Chain Vulnerabilities in the OWASP Top 10 for Agentic Applications (OWASP GenAI Security Project, 2026). MITRE ATLAS catalogues the corresponding adversary techniques under AML.T0010 AI Supply Chain Compromise (MITRE, n.d.).

How it arises

Vulnerable or outdated third-party components and models

Outdated or deprecated components (packages, serving frameworks, and the models themselves) can be exploited to compromise an application. This is the LLM-specific extension of A06:2021 Vulnerable and Outdated Components: the same risk, but the blast radius is larger because a vulnerable component may sit in the training, fine-tuning, or inference path rather than only in application code.

The new variant is slopsquatting. LLM coding assistants hallucinate plausible but nonexistent package names at scale (Spracklen et al., 2025), and attackers register those names in advance, so that an unverified AI-suggested dependency resolves to malicious code. The attacker does not need to compromise anything: they only need to predict what the assistant will invent.

Licensing risks

Software and dataset licences in AI projects span a wide range of terms governing usage, redistribution, and commercial deployment. Without active licence tracking, teams accumulate legal exposure that produces no security alert and typically surfaces at the worst possible moment: during due diligence or product launch.

Vulnerable or tampered pre-trained models

Model artifacts are difficult to inspect comprehensively, and static analysis alone cannot establish behavioural safety. A pre-trained model can carry hidden biases or backdoors introduced through poisoned datasets or direct tampering.

Migrating away from unsafe serialisation formats such as Python pickle, which can execute arbitrary code on load, reduces but does not eliminate this risk, for two separate reasons:

• A backdoor can be embedded directly in a model’s computational graph and persist in formats widely considered safe, such as ONNX.
• A crafted model file can exploit memory-corruption bugs in a format’s native parser, as demonstrated by heap overflows in llama.cpp’s GGUF parsing, CVE-2024-23496 (Cisco Talos, 2024).

Weak provenance and unsigned model artifacts

Published models carry no reliable proof of origin. Model Cards describe what a model is and how it was trained, but they are metadata, not cryptographic evidence. A compromised or lookalike supplier account can publish a malicious model under a trusted name and the Model Card will say whatever the publisher writes.

When models, adapters, datasets, fine-tuned checkpoints, and conversion outputs are not signed or hash-pinned, an attacker can replace or silently alter artifacts in transit, in storage, or at the promotion boundary, the point where automated pipelines accept artifacts into trusted environments. This is especially exposed when pipelines resolve artifacts by a mutable reference, such as a latest tag, rather than an immutable digest.

Vulnerable adapters and compromised conversion, merge, and quantisation workflows

LoRA adapters make fine-tuning modular and efficient, but a malicious adapter can compromise the integrity of the pre-trained base model, whether in a collaborative merge environment or on an inference platform that downloads and applies adapters to a deployed model. Model conversion and merge services can introduce malicious changes during transformation between formats, bypassing review controls that ran on the pre-conversion artefact.

Quantisation is a related transformation risk with a nastier property: weights can be crafted so that the full-precision model evaluates benignly while the quantised artefact exhibits attacker-chosen behaviour (Egashira et al., 2025). Full-precision assurances do not transfer to the deployed quantised artefact, so an evaluation suite run before quantisation proves nothing about what ships.

On-device LLM supply-chain vulnerabilities

Shipping LLMs on devices adds compromised manufacturing processes, exploitation of device OS or firmware vulnerabilities, and re-packaged applications carrying tampered models. Device integrity and firmware trust become part of the LLM supply chain.

Unclear T&Cs and data privacy policies

When a model operator’s terms of service or privacy policy are vague about how input data is handled, application data sent through the API may end up in future training runs and surface in other users’ outputs. The same ambiguity can create copyright exposure from material the vendor supplies. This is the supply-chain face of LLM02:2026: a contractual gap that becomes a disclosure.

Mitigations

Grouped by the question each answers.

Who are you taking artifacts from?

Vet who you take data and components from before you take them, terms and privacy policy included, and keep the list short. Re-examine those suppliers on a schedule, and again whenever their security posture or terms change. A supplier assessment performed once at onboarding does not survive an acquisition or a policy rewrite.

Is the component maintained and real?

Apply the A06:2021 mitigations: vulnerability scanning, vulnerability management, and a patching policy that keeps the application on maintained versions of components, APIs, and underlying models. Apply the same controls to development environments with access to sensitive data. Historically these are exempted, and they hold the credentials. Verify that AI-suggested dependencies actually exist and are the intended package before adopting them; this is the direct control for slopsquatting.

Do you know what you have?

Maintain an up-to-date, signed inventory using an SBOM, extended to models, adapters, and datasets with AI BOMs (AIBOMs) and ML SBOMs. The source names the OWASP CycloneDX ML-BOM (OWASP CycloneDX, n.d.) and the OWASP AIBOM project (OWASP GenAI Security Project, 2025) as options to evaluate. Track licences in the same inventory and audit it regularly for compliance and transparency. This is how the licensing risk above becomes visible before it becomes a legal problem.

Is the artefact the one the supplier published?

Only use models from verifiable sources, and compensate for weak provenance with third-party integrity checks, signing, and file hashes. Cryptographic model signing backed by a transparency log (for example the OpenSSF Model Signing project and Sigstore) binds a model artifact to a signer identity (Open Source Security Foundation, 2025). Reproducible builds are not guaranteed for model training, which is the reason signing carries more weight here than it does for compiled software. The Coalition for Secure AI provides a maturity model for signing ML artifacts (OASIS Open, 2025).

The limit is stated plainly in the source: signing proves integrity and origin, not safety. A validly signed model from a compromised or trusted-but-malicious supplier can still be backdoored. Combine signing with immutable artifact references, provenance policy, policy-based release gates such as SLSA (Open Source Security Foundation, n.d.), behavioural evaluation, and continuous validation of upstream model integrity. Anything arriving as code should be signed on the same principle.

Does it behave?

Apply AI red teaming and evaluations when selecting third-party models, focused on the use cases in scope. Continue in production with anomaly detection and adversarial robustness testing in MLOps and LLM pipelines, to detect tampering and poisoning that selection-time evaluation missed.

Where does the artefact change hands?

Strictly monitor and audit collaborative model development environments, and treat model conversion and merge services as high-risk promotion points. These are the places where an artefact that passed review becomes a different artefact.

What ships to devices?

Encrypt models deployed at the edge and verify their integrity on load. Where a vendor publishes an attestation API, use it, since that is the mechanism for catching tampered apps and models, and reject unrecognised firmware and untrusted device states.

Attack scenarios

1. Compromised packages and serving frameworks. A compromised dependency reaches a model development or inference environment, as in the December 2022 PyTorch supply-chain attack, where a malicious torchtriton package on PyPI shadowed the legitimate PyTorch-nightly dependency and exfiltrated data (PyTorch Foundation, 2022). The serving stack is the same surface: the ShadowRay attacks exploited the disputed CVE-2023-48022 (unauthenticated dashboards on production Ray servers) in the wild, later escalating into a self-propagating botnet across exposed clusters (Lumelsky & Elbaz, 2025). In Ollama, CVE-2024-37032 allowed remote code execution through a malicious model manifest pulled from a registry (Wiz, 2024).

2. Tampered model published to a hub. An attacker publishes a tampered model under a trusted-looking name, as demonstrated by the PoisonGPT proof of concept, in which a model with surgically modified parameters spread misinformation while evading detection by standard benchmark evaluation (Huynh & Hardouin, 2023). The same trust gap covers attacker fine-tunes that strip a popular model’s safety features while preserving benign-task performance (Zhan et al., 2023), any pre-trained model deployed without verification, and shared AI-as-a-service platforms - researchers escaped an inference container via a malicious model to reach other customers’ models and data (Tamari & Tzadik, 2024).

3. Compromised supplier LoRA adapter. Someone gets inside a third-party supplier and quietly alters a LoRA adapter that is later merged into a deployed model through a merge workflow. Once merged, the adapter provides a covert entry point.

4. Hijacked model conversion or merge service. An attacker stages an attack through a merge or format-conversion service to compromise a publicly available model and inject malicious behaviour, as shown by HiddenLayer’s research on hijacking the Safetensors conversion bot on Hugging Face (HiddenLayer, 2024).

5. Model namespace reuse. An organisation deploys a model from a public hub referencing it solely by Author/ModelName. The original author deletes or transfers the account, freeing the namespace; an attacker re-registers the same name and publishes a malicious model under the original path. Pipelines and managed model catalogues that resolve by name alone then pull the attacker’s model, leading to remote code execution (Saraf & Balassiano, 2025). Immutable digests, not names, are the fix.

6. Scanner, safe-loader, and safe-format bypass. An organisation gates third-party models behind a malware scanner and a loader option documented as safe against code execution. An attacker defeats both:

Treat scanners and safe-loader flags as defence-in-depth layers rather than guarantees, alongside provenance verification and patched loaders and parsers.

7. Compromised build pipeline for model artifacts. An attacker compromises the CI/CD pipeline used to fine-tune and publish models (through a malicious build-workflow dependency, a stolen artifact-registry credential, or cache poisoning), as in the Ultralytics attack, where GitHub Actions cache injection published trojanised PyPI releases of a flagship AI library (Python Package Index, 2024), including a compromised “fix” release. This is the same build-time substitution seen in classical incidents such as the xz-utils backdoor and the Codecov breach. The failure mode that matters: because the backdoored artefact is built and signed by the organisation’s own release infrastructure, it passes downstream provenance checks, internal attestation, and supply-chain scanners that only flag externally sourced components.

8. Reverse-engineered mobile app. An attacker reverse-engineers a mobile app to replace the on-device model with a tampered version that leads users to scam sites, distributing the re-packaged app through social engineering.

Related entries

• LLM05:2026 Data and Model Poisoning - the mechanics of poisoning; this entry covers how the poisoned artefact arrives.
• Sensitive Information Disclosure (LLM02:2026) - supplier T&Cs and retention behaviour are a disclosure vector, and a leaked adapter is an extraction surface.
• Excessive Agency (LLM03:2026) - a compromised tool package is one of the named triggers for damaging agent actions.
• Prompt Injection (LLM01:2026) - supply-chain compromise of a system prompt or tool description is an injection delivery mechanism.
• ASI04 Agentic Supply Chain Vulnerabilities - MCP servers and tool registries.
• More entries: AI Security.