In 2025, JFrog's security research team discovered 495 malicious AI models on HuggingFace — many with active reverse-shell payloads embedded in model weight files. Separately, researchers catalogued 969 malicious AI agent skills distributed across MCP plugin registries. A parallel study found that 65% of Indian enterprises cannot detect malicious open-source packages before they reach production. This is not a future threat. It is happening now, in your stack.

A supply chain attack targets the components your software depends on — not the software itself. Attackers compromise an upstream package, library, or data source so that every downstream consumer inherits the malicious payload automatically. The AI supply chain is the full dependency graph of an AI-powered application: open-source models downloaded from registries like HuggingFace or PyPI, training and RAG datasets pulled from cloud storage, third-party SaaS AI APIs, and the expanding ecosystem of agent plugins and MCP servers. Every node in that graph is a potential injection point.

What makes this uniquely dangerous is the trust model. Developers who would never blindly run a stranger's Python script will readily install a 600 MB model checkpoint — a binary blob that can execute arbitrary code the moment it is deserialised. The attack surface is massive, largely invisible to traditional security tooling, and growing at a rate that outpaces awareness.

495
Malicious AI models found on HuggingFace with active payloads (JFrog, 2025)
451%
Increase in malicious npm packages targeting AI toolchains in 2025
65%
Of Indian enterprises that cannot detect malicious packages before production

The 5 AI Attack Surfaces You Have Not Secured

Most security programmes have a model for securing application dependencies. None of it was designed for the AI layer. Here are the five surfaces that are almost certainly unmonitored in your organisation right now.

1. Package Dependencies. PyPI, npm, and conda packages that wrap AI frameworks (LangChain, Transformers, LlamaIndex) are heavily typosquatted. Malicious packages exfiltrate API keys, inject prompt-override logic, or install backdoors during the build phase — before your application ever runs. LangChain-related packages on PyPI saw 8 confirmed malicious clones in Q1 2025 alone.

2. Model Weights and Registries. PyTorch's pickle format executes arbitrary Python during deserialisation. HuggingFace's SafeTensors format mitigates this, but hundreds of thousands of legacy models remain in pickle format. Attackers upload convincing impersonations of popular models and embed reverse shells that fire when a data scientist first loads the model. The JFrog team found 495 such models in a single audit pass.

3. RAG and Training Data. Retrieval-Augmented Generation systems ingest documents, web pages, and knowledge bases. Poisoned documents can embed prompt-injection payloads that hijack an AI agent's actions when retrieved. Stanford research found that a single poisoned PDF in a 10,000-document corpus caused a GPT-4-based assistant to execute attacker instructions in 73% of relevant queries. Training data poisoning causes models to produce systematically biased or incorrect outputs on specific adversarial inputs.

4. SaaS-Embedded AI. Every SaaS product your employees use now has an embedded AI copilot. Each one accesses internal data with the user's permissions and processes it via opaque third-party inference infrastructure. A compromised SaaS AI layer can silently exfiltrate customer PII, Salesforce data, or internal knowledge bases without triggering any DLP alerts. 65% of Indian enterprises surveyed by DSCI 2025 had no visibility into what data their SaaS AI tools accessed.

5. Agent Plugins and MCP Servers. AI agent frameworks support plugin ecosystems that grant agents real-world tool access — web browsing, code execution, email, file systems. Researchers found 969 malicious MCP-compatible skills in the wild. A compromised plugin can intercept agent actions, exfiltrate the agent's context window including internal documents it retrieved, or use the agent's permissions to pivot laterally into adjacent systems.

Security teams must now evaluate AI models with the same rigour as source code. A model checkpoint is an executable — not a static asset.

Why Traditional VAPT Misses AI Supply Chain Risks

VAPT as practised today was designed to find misconfigurations, known CVEs, and logical flaws in networked services. It was not designed to answer: "Does this model checkpoint execute code when loaded?" or "Has our RAG corpus been poisoned with adversarial documents?" or "Does this MCP plugin exfiltrate context window data?"

The gap is structural. Traditional VAPT does not test malicious model weights because they are binary, not source. Typosquatted AI packages evade manifest scanning because of obfuscated installs. Prompt injection in RAG documents is an entirely different threat category with no concept in traditional methodology. SaaS AI data exfiltration is scoped outside the internal perimeter. Malicious MCP plugins are not in scope. The attack surface operates in the space between traditional categories — in trust relationships, serialisation formats, and emergent behaviours that have no CVE number because they are not bugs. They are features being exploited.

3 real AI supply chain attack patterns. The Malicious Checkpoint: an attacker uploads a convincing impersonation of a popular model to HuggingFace; when a developer loads it, the pickle deserialiser executes the embedded payload, establishing persistence in the ML training environment. RAG Poisoning: an attacker with write access to a SharePoint folder uploads a PDF containing a hidden prompt injection that reroutes financial guidance when the AI retrieves the document. The Compromised Agent Plugin: a plugin sourced from a public registry logs the full contents of every agent context window — including retrieved internal documents and user queries — and exfiltrates them to an attacker-controlled endpoint via encrypted HTTPS requests that blend into normal traffic.

How to Build an AI Supply Chain Security Programme

This is a solvable problem. But it requires adding a new discipline to your security practice, not just running existing tools at a new target.

Step 1: Inventory your AI artefacts. Every model in use (including those in data science notebooks and shadow IT), every AI-related Python and npm package, every SaaS product with an AI copilot, and every agent plugin or MCP server. This inventory is your SBOM for AI.

Step 2: Implement model scanning before load. Deploy Protect AI's ModelScan or JFrog's ML security scanner in your CI/CD pipeline. Any model checkpoint, whether downloaded from HuggingFace or produced internally, should be scanned for unsafe serialisation formats and suspicious embedded objects before it is loaded in any environment.

Step 3: Move models to SafeTensors format. SafeTensors is HuggingFace's safer serialisation format that does not support pickle execution. Mandate its use for all internally produced models and prefer it when selecting external models.

Step 4: Red-team your RAG corpus. Treat your document knowledge base as an adversarial surface. Run prompt injection probes against your RAG pipeline: submit crafted documents containing instruction-override payloads and verify that your AI does not execute them. This should be part of pre-production security testing for any AI assistant.

Step 5: Audit agent plugin permissions. For every MCP server or agent plugin in use, document what permissions it has, what data it can access, and whether it has outbound network access. Apply least privilege: an agent that helps write code should not have access to customer records.

Step 6: Integrate AI-output SAST into CI. AI-generated code has characteristic vulnerability patterns — insecure defaults, missing input validation, deprecated API usage. Tools like Semgrep and Snyk can be tuned to catch common AI-output flaws before they ship to production.

Step 7: Conduct an AI supply chain red team exercise. Commission an assessment that specifically targets your AI stack with adversarial thinking about model trust, agent permissions, and data pipeline integrity. Run it annually, or after any significant change to your AI architecture.

Indian enterprises are adopting AI faster than their security programmes can adapt. The CERT-In 2025 advisory on supply chain risks mentioned open-source software; the next revision will mention AI artefacts. Getting ahead of this now — before a regulatory mandate and before an incident — is the posture that separates security leaders from security laggards.

Close your security gaps — continuously.