The AI gateway holding every API key you own is attackers' new favorite target.

Somewhere in a cloud region right now, a proxy server is holding the keys to every AI model your company talks to — OpenAI, Anthropic, Google, a dozen internal tools — in one place, because that’s supposed to be the safe way to do it.

For 90 days, researchers at Wiz quietly built fake versions of exactly that kind of server and put them on the open internet to see who’d show up. Real attackers showed up. What they did once they got inside is the most interesting hacking story of the month, and it says something uncomfortable about how fast the defenses are keeping up with the tools.

The gateway that holds all the keys

The tool at the center of this is called LiteLLM, and it’s a genuinely great idea. Instead of every team in a company wiring its own code directly to OpenAI, or Anthropic, or a self-hosted model, everyone talks to one internal gateway. The gateway holds the real API keys, routes each request to whichever model is cheapest or fastest for the job, logs everything, and enforces spending limits. Wiz’s own research puts self-hosted AI software — LiteLLM, Flowise, LangChain, Langflow, ChromaDB, Ollama, OpenWebUI, Node-RED — running inside roughly 90% of cloud environments already. This is not a niche tool. It’s plumbing.

That’s also exactly why it’s such a good target. A gateway like this is a single box that holds credentials for every model provider a company uses, plus — increasingly — access to the Model Context Protocol (MCP) servers that let AI agents actually do things: file tickets, query databases, touch source control. Compromise the gateway and you don’t steal one password. You steal the keys to everything the agents were trusted to reach.

Three ways in, and none of them look like a normal hack

Wiz’s honeypots caught attackers using three distinct techniques, and each one is specifically shaped for AI infrastructure rather than borrowed from a general hacking playbook.

The first exploited a real, now-patched flaw in LiteLLM itself: CVE-2026-59822. The gateway’s Model Context Protocol endpoint was supposed to reject requests without a valid key. Instead, when a bearer token failed validation, the code fell back to treating the caller as an authenticated user with no limits attached — an empty auth object that downstream code read as “logged in, just untracked.” Attackers found they could get in with a bearer token as simple as the single character “x”. Chained with a second bug (command injection in a diagnostic test endpoint), it became full remote code execution. Wiz caught attackers deploying cryptominers dressed up as legitimate MCP tool configurations — the malicious process ran detached in the background while the request that launched it reported back as if nothing had happened.

The second technique, blind prompt injection, didn’t need a software bug at all. Attackers injected instructions into content an AI agent would process, designed to make the agent itself reach out — a DNS callback to an attacker-controlled domain confirming the injection worked, followed by the agent fetching a base64-encoded payload from Pastebin (a favorite because it doesn’t trip most security logging) and executing it. The agent becomes the delivery mechanism for its own compromise.

The third is the one worth sitting with: once inside, attackers skipped the traditional playbook of scanning the filesystem and enumerating users. Instead, they queried the Python process’s own module state directly to pull LiteLLM’s master key straight out of memory, fingerprinted which backend models were connected before deciding what to do next, and staged their malware in directories like .claude/ — specifically because a file sitting in an AI tool’s own configuration folder doesn’t look out of place to a human skimming a server.

One quick word from today’s sponsor

Some teams never seem to stop moving. They're on Attio, the agentic CRM.

It’s your always-on revenue engine: agents and workflows build pipeline, chase every buying signal, and move deals forward alongside your team.

Teams like Parallel, Turbopuffer, and Wordsmith build on Attio. Are you one of them?

Try Attio now

Why this one is worth patching today, not eventually

CISA added CVE-2026-59822 to its Known Exploited Vulnerabilities catalog on September 2, confirming it isn’t a theoretical risk — it’s actively being used, and the deadline for federal agencies to patch fell on September 16. If your organization runs LiteLLM as an internal gateway:

1. Upgrade to 1.84.0 or later. The vulnerable versions treat both a failed key check and a .well-known query-string trick as free passes; both are closed in this release.

2. Rotate every upstream provider key and MCP credential the gateway held, on the assumption it was exposed rather than waiting for proof it was.

3. Search your logs for bearer tokens that don’t match any real key but still returned HTTP 200. That’s the single cleanest sign this exact bypass was used against you.

4. Treat the gateway like a secrets vault, because that’s what it actually is. Network segmentation, egress filtering, and monitoring for unusual outbound DNS all apply here exactly as they would to any system holding root credentials for your cloud.

5. Watch for the AI-native tells, not just the classic ones: unexpected files in AI-tool config directories, processes introspecting their own memory, and outbound requests to paste sites from a server that has no business talking to one.

None of this is an argument against using an AI gateway — the alternative, every team holding its own scattered set of raw API keys with no central logging, is worse. It’s an argument for treating the tool with the seriousness its actual role deserves. A LiteLLM instance isn’t a convenience layer anymore; for a lot of companies it’s the one box that can talk to every model, every dataset, and — through MCP — every internal system an agent has been handed the keys to. Attackers have clearly noticed that before most security teams finished updating their asset inventory to include it.