Intro
If you are running LiteLLM or Dify on a public IP without strict network segmentation, you are currently holding open the doors to your most sensitive assets. The immediate risk is no longer just a curious human scanner; it is an automated, relentless process designed to steal your AI provider keys and use them to run more automated attacks. This is an operational fix. We are not here to debate the ethics of autonomous agents or to theorize about future risks. We are here to look at the specific mechanics of how your infrastructure is being probed and exactly where the gaps are in your current setup.
The threat model has shifted significantly over the last few months. Gone are the days where a single bad actor sat at a terminal crafting custom exploits. Now, the primary vector is agentic software. Tools like ARTEX, which is an open-source AI framework for penetration testing, are being used to decompose complex security goals into executable tasks. A planner agent defines the objective, such as “compromise this LLM gateway,” and then spawns worker agents that execute specific security tools to find entry points. These agents do not hesitate. They do not get tired. They scan, test, and exploit around the clock.
I manage several VPS hosting accounts for clients who use local AI stacks, and I see this pattern in my logs regularly. The difference now is that the traffic is no longer sporadic. It is structured. The attackers are not just looking for open ports; they are looking for the specific configuration files where master keys live. In LiteLLM, for example, the default master key is a known value in many community deployments. If your instance is exposed to the public internet, an agent can identify that default credential in seconds, log in to the admin panel, and export your entire pool of provider API keys. Once those keys are in hand, they are verified against OpenAI, Anthropic, or other providers to ensure they are still active. If they work, they are added to a shared pool for further monetization or operational use.
This is why the current state of affairs for self-hosters is precarious. Most of us set up these tools for convenience during development or for small internal teams, assuming that “internal” means safe. But if the reverse proxy does not explicitly block all traffic except from your office IP range, or if the application firewall is not configured to reject unauthenticated requests to management endpoints, you are effectively publishing your key vault. The Amazon Threat Intelligence team recently captured records from a decoy network called MadPot, which confirmed that these automated campaigns are actively targeting LiteLLM, Dify, One API, and FastGPT. They are not guessing. They have playbooks.
We need to stop treating LiteLLM API key security as a background concern. It is the first thing an automated agent will check. If your gateway is reachable from the public internet, you have already lost the race to lock the door. The rest of this post focuses on how to verify your exposure, how to segment your network so that even if the front door is open, the keys remain out of reach, and how to rotate credentials on the assumption that they have already been seen. This is not about paranoia. It is about the fact that the attackers are already there, waiting.
Background: The Rise of Agentic Attacks
The shift from human-driven intrusion to autonomous agent operations has changed the speed and scale of credential theft. Amazon’s threat intelligence team recently published findings from their MadPot decoy network, which captured logs from automated agents actively hunting for weak spots in internet-exposed AI applications. These records, detailed in Amazon’s report (opens in new tab), reveal a sophisticated campaign where operators use ARTEX, an open-source penetration testing tool, to execute security workflows. The confidence level in these findings is moderate, but the volume of captured data is significant enough to treat as a confirmed trend rather than an isolated incident.
ARTEX operates through a hierarchical structure that mimics a human team. A planner agent takes a high-level goal, such as “locate valid API keys on target domain,” and breaks it down into specific tasks. Worker agents then execute these tasks by running standard security tools and sending their findings back to a large language model. This LLM determines the next logical step, creating a feedback loop that allows the system to adapt to new obstacles in real time. It is not just a script running a list of commands; it is a dynamic process that reasons through access vectors.
The captured logs highlight specific entry points that attackers are prioritizing. The playbook includes default administrator credentials, such as LiteLLM’s default master key, which is still present in many unconfigured installs. It also targets unfinished installations where security hardening was skipped during deployment. For systems like One API and its derivatives, the agents exploit authentication bypasses, specifically the isCliRequest header, which allows unauthenticated access to certain endpoints if not properly filtered. Additionally, the logs reference known flaws in Langflow and RAGFlow that provide direct paths to code execution or data exfiltration.
Attribution for these campaigns points to China-linked groups, based on the use of Simplified Chinese in operator prompts and time zone settings in the logs. This suggests a coordinated effort rather than opportunistic scanning by individuals. The agents also utilized server-side request forgery (SSRF) vectors to reach cloud metadata services, aiming to steal temporary credentials that could grant access to managed AI services. This is a critical detail for anyone running these tools in cloud environments. A single exposed endpoint can become a bridge to your entire infrastructure.
The implication for our networks is clear. These agents do not just find one weak link; they use it to map the broader environment. When they compromise a LiteLLM instance, they do not stop there. They look for other keys, other services, and other paths to expand their foothold. The automated nature of this attack means that the window between discovery and exploitation is measured in seconds, not hours. Human defenders cannot keep up with this pace through manual monitoring alone. We need to assume that if our AI gateways are reachable, they have already been scanned, attempted, and potentially breached. The rise of agentic attacks means that the perimeter is no longer defined by IP addresses, but by the isolation of sensitive data.
What’s Happening Now: Exposed AI Apps and API Key Theft
The attack chain is becoming remarkably standardized. Agentic tools are not just scanning for open ports; they are actively probing specific, known weaknesses in popular AI infrastructure products. The records captured by Amazon Threat Intelligence list a long tail of targets: LLM gateways like One API, New API, Veloera, Sub2API, and LiteLLM, alongside app builders such as Dify, FastGPT, Langflow, RAGFlow, Open WebUI, LobeChat, and Ollama. This breadth indicates that attackers are casting a wide net because any one of these tools often acts as a central hub for multiple cloud provider keys. Compromising a single LiteLLM instance can yield access to Anthropic, OpenAI, and AWS Bedrock credentials simultaneously.
The operational detail that concerns me most is the verification step. These agents do not just grab a key string; they test it. They make a cheap, low-token API call to confirm the key is valid before adding it to their shared pool. This means that if you are running an unauthenticated endpoint, you are not just risking a one-time breach. You are handing over a verified, working credential that the attacker will immediately repurpose for their own automation. In one captured log, the agents used stolen keys to run the very LLMs that powered the search for more keys. It is a self-sustaining loop of credential theft that requires zero human intervention once the initial access is gained.
A specific incident involving Cloudflare illustrates how quickly this escalates beyond the AI app itself. In that case, attackers used a compromised CI/CD token that lacked IP restrictions or rate limiting. Within a single hour, they generated 400 database dump requests. While Cloudflare did not confirm if data was exfiltrated, the volume of requests highlights the speed at which automated tools can drain an environment. This mirrors the risk we face with AI gateways: without strict egress controls and rate limiting, a single leaked key can trigger massive data exfiltration or resource abuse.
The attack surface extends to the metadata service, too. Attackers are using server-side request forgery vectors in these apps to reach cloud metadata endpoints. If you are running LiteLLM or Dify on AWS, GCP, or OCI, an unauthenticated SSRF endpoint can be used to steal temporary instance credentials. Those credentials then allow the attacker to pivot into your VPC, potentially accessing S3 buckets or other internal services that store more sensitive data.
Consider a small business owner running Dify on a VPS. They likely have a single API key for OpenAI and another for Azure stored in an environment file. If that Dify instance is exposed to the public internet without a firewall rule, an automated agent will find the admin panel, read the environment configuration, and extract both keys. The owner might not notice for weeks, until they see a spike in their Azure bill. By then, the keys have been in a shared pool, used by multiple bad actors, and are effectively dead.
The key takeaway here is that “exposed” does not mean “compromised,” but it means “compromised soon.” The scanning is continuous. If your instance is reachable, it is in the queue. The next logical step is to understand how to verify your own exposure and isolate these components before the next agent cycle runs.
What It Means in Practice for Self-Hosters
You do not need to be a network engineer to check if your setup is currently bleeding. The first step is a simple external visibility test. From a machine outside your VPS or cloud account, run a curl command against your LiteLLM or Dify health endpoint. If you see a 200 OK or a 401 Unauthorized without a valid key, you are visible to the scanning bots Amazon Threat Intelligence described. A 403 Forbidden from your firewall or reverse proxy is what you want to see. For LiteLLM specifically, check the litellm/settings.yaml or the environment variables where you store your master_key. If that key is hardcoded in a file that is also mounted into a debug container, you have created a path for exfiltration that an agent can exploit in seconds.
The configuration matters more than the code. In my own hosting stacks, I have seen developers set --no-auth on LiteLLM during development and forget to remove it for production. The default master key, often just a long string of characters if you did not set one, acts as a backdoor. An agent does not need to crack encryption; it just needs to guess the default or read the unauthenticated config endpoint. For Dify, the risk is similar but with a different entry point. The admin panel is often accessible at /admin. If that path is not protected by IP allowlisting, an agent can log in with the default credentials if you never changed them, or exploit known authentication bypasses to access the workspace where provider keys are stored.
Network segmentation is not a “nice to have” here. It is the baseline. If you are running these tools on a VPS, you should not have port 8000 or 8080 open to 0.0.0.0. Place a reverse proxy in front of them, and restrict access to that proxy using firewall rules that only allow your specific office IP or a VPN subnet. This prevents the “scanning” phase entirely. The agents cannot find the door if the door does not exist on the public internet.
Do not rely on the application’s internal security to protect you. These tools are designed for developer convenience, not for hardening against state-sponsored or automated threats. An environment file that contains your AWS access keys is a bomb if it is accessible via a debug route. I have audited CI/CD pipelines where temporary credentials were left in build artifacts, accessible via a public S3 bucket. If you are using GitHub Actions or similar, check your logs for “secrets” that should not be there. An agent scanning GitHub will find those leaks faster than you can rotate the keys.
Finally, understand that even if your web interface is private, the API keys inside are exposed if the underlying services are not isolated. If your LiteLLM container can make outbound requests to the cloud metadata service, an SSRF vulnerability in the app is all an agent needs to steal your instance credentials. That is a pivot into your entire cloud account. Isolation here means limiting the network egress of the container so it can only talk to the specific LLM providers you have configured, and nothing else.
This is where the “practical” advice becomes operational. You cannot out-code an agent. You can only out-network it. The goal is to make your instance invisible to the scanning phase, and impossible to pivot from if the agent does get inside.
Checklist for Immediate Hardening
- Rotate Keys: Assume all provider keys currently in your LiteLLM or Dify instances are compromised. Rotate them in your provider dashboards (OpenAI, Azure, Anthropic, etc.) today.
- Remove Defaults: Change the LiteLLM
master_keyand Dify admin password. Remove any “default” or “test” keys from configuration files. - IP Allowlisting: Configure your firewall (iptables, AWS Security Groups, GCP VPC) to deny access to the application ports from all IPs except your known admin IPs.
- Disable Unauthenticated Endpoints: In LiteLLM, ensure
general_settingsdoes not allow unauthenticated access to/model_group/infoor similar endpoints. - Audit Egress: Configure network policies to block the application containers from reaching cloud metadata services (169.254.169.254 on AWS/GCP/OCI).
The next section covers how to verify this setup is actually holding against the specific attack patterns we have seen.
What to Expect Next
The next phase of this campaign will likely look less like a new attack vector and more like an escalation of the current one. The agents have already proven they can steal keys and add them to a shared pool. The logical next step is using that pool to access premium AI models for further automation. Imagine an agent that now has access to a high-tier Anthropic or OpenAI key. It can run complex reasoning tasks at low cost, which means it can execute more sophisticated social engineering attacks against your other employees or parse your private code repositories faster. The infrastructure for the theft is already built. The only variable left is the budget for the inference.
I expect defenders to start blocking common agent user-agents within the next few weeks. If you have looked at the Cloudflare logs, you will see patterns like artex-pgrecon-* or artex-do-*. These are distinctive. Security teams will add these strings to their WAFs. However, this will only buy time. ARTEX is open-source. A motivated operator can change the user-agent header in five minutes. Relying on user-agent filtering is a false sense of security. It stops the noise, but it does not stop the signal.
We should also watch for a broader trend beyond just LLM gateways. Microsoft has already noted that attackers are using agentic AI tools for GitHub secret hunting, specifically looking for exposed credentials that include API keys for AI inference. This indicates that the goal is no longer just “hack the cloud.” The goal is “own the AI capability.” Attackers are prioritizing managed AI services over general cloud access because inference keys are the crown jewels. They represent a recurring revenue source that can be resold or used to power their own agent swarms.
In my view, the real danger is the persistence of the stolen keys. A generic AWS key is often ephemeral or scoped tightly. A LiteLLM master key or a Dify admin token can sit in a configuration file for years. Once an agent extracts it, that key is valid indefinitely until you manually rotate it. This means the attack surface does not shrink over time. It grows. Every new integration you add, every new model provider you plug in, adds another potential entry point for an agent to siphon credentials.
I do not know if this campaign will evolve into a botnet that actively attacks the AI providers themselves, but I suspect the focus will remain on harvesting keys from intermediaries. The intermediaries, like LiteLLM and Dify, are the easiest targets because they are often misconfigured by non-specialists. The attackers do not need to break into OpenAI. They just need to break into the small business that proxies traffic to OpenAI. Expect the scanning to become more aggressive on port 4000 and standard web ports, targeting specifically these open-source gateways. The window to fix your configuration before the next wave of automated scans is closing fast.
Concrete Next Step: Harden Your LLM Gateway Security
You cannot wait for a scanner to find your configuration before you act. The first step is to assume every provider key currently stored in your LiteLLM or Dify instance is compromised. Rotate them all immediately. Do not reuse old keys. If you have a master key in your LiteLLM configuration that has been sitting there since your initial setup in 2024, delete it. Replace it with a new, random string. Then verify that your upstream providers, such as Anthropic or OpenAI, have invalidated the old tokens. A key that works in one gateway but fails in another is still a liability if the gateway log contains it in plain text.
Next, strip out default credentials. Many self-hosters install Dify or LiteLLM with the default admin user and password, or they leave the LiteLLM master key blank if they forget to set one. Check your environment files. In LiteLLM, look at litellm/config.yaml or your .env file. Search for MASTER_KEY. If it is empty or set to default_key, you are wide open. The same applies to Dify’s .env file, where SECRET_KEY and database passwords often live. If you are using Docker, these files are visible in the container’s filesystem. If you mounted them incorrectly, they might be readable by any user on the host.
You need to lock down the management interface. Running these tools behind a reverse proxy with IP allowlists is non-negotiable. Your inference API might need to be public, or semi-public if you are building a SaaS wrapper, but your admin dashboard and configuration endpoints should never be accessible from 0.0.0.0/0. Use Nginx or Caddy to restrict access to /admin and /config routes. Only your own static IP range should have access. If you use a cloud provider, enable VPC endpoints or private link for these management services. Do not rely solely on the application’s internal login. Network segmentation is the only layer that stops an agent from even reaching the login form.
Finally, audit your build pipeline. The Cloudflare incident mentioned static CI/CD tokens that allowed agents to dump databases. Check your GitHub Actions, GitLab CI, or Jenkinsfiles. Look for secrets that are hardcoded in scripts rather than stored in the secrets manager. If you have ever pushed a .env file to a public repository, even accidentally, assume the keys inside are known. Use git-secrets or trufflehog to scan your repository history. A key leaked in commit history in 2023 is still valid today if you never rotated it. Attackers scan GitHub continuously. They are not looking for complex zero-days. They are looking for the mistake you made two years ago and forgot to fix.
Do this now. Rotate keys. Lock down the admin panel. Scan your repos. It takes an afternoon. The alternative is having your inference bills paid for by an attacker who is using your credentials to train their own models.