Intro
An attacker used an SGLang RCE exploit to compromise inference servers at NEAR AI, and the incident exposes a gap that every AI infrastructure operator needs to understand. The breach happened on a Sunday, on one of the company’s inference servers, and it did not stop at the perimeter. The attacker moved from that compromised server into internal legacy systems, pulling account details for roughly 100 users and data from 18 AI agents. What makes this case worth reading is not the scale of the theft, but what it reveals about how modern hosting stacks get bypassed, as reported by The Hacker News (opens in new tab).
Think of it like a smart lock on your front door that only works if the software inside it is honest. Your firewall, your VPN, your container isolation, they all sit outside the problem. The SGLang RCE exploit worked because the vulnerability lived inside the application layer, in the inference engine itself, and that engine was sitting on a server that accepted requests from the outside world. Standard server hardening checks for open ports, weak passwords, and unpatched system packages. It rarely asks whether the Python library processing your user input has a hole an attacker can drive code straight through.
NEAR AI was running some of its workloads inside a trusted execution environment, a TEE, which is supposed to keep data private even from the company’s own engineers. The company itself said the TEE does not protect against a vulnerability in the software running inside it. That is the uncomfortable truth here. If your inference engine has a remote code execution flaw, putting it in a secure enclave is like locking your diary in a safe and then leaving the key under the mat. The safe works. The diary is still readable by anyone who can pick the lock.
I run my own VPS boxes and manage real customer sites, and this kind of attack makes me wary of any third-party library that handles raw sockets and input parsing without a very careful review. The AI stack is held together with Python packages, and each one is a surface area. When one of them becomes an entry point, everything downstream inherits the problem.
I expect that automated scanners will start hunting for exposed SGLang instances on public cloud providers within the next few months, now that this attack path is documented in the wild.
The practical takeaway is straightforward: if you run inference engines, check your versions today, not next week.
Background on the SGLang Vulnerability
To understand how the NEAR AI breach happened, we need to look at what SGLang actually does in a production environment. As an open-source inference engine, SGLang is designed to handle the heavy lifting of serving large language models quickly. It manages KV caching, token generation, and the massive amounts of parallel request handling required to keep an AI application responsive. Because modern language models demand high throughput, these inference engines sit right at the boundary between the public internet and the internal infrastructure. They accept raw HTTP requests containing user prompts, parse them, pass them to the model weights loaded in memory, and stream the generated text back out to the client.
The problem arises when an SGLang RCE exploit is discovered in the wild, as detailed in this report on The Hacker News (opens in new tab). A remote code execution flaw in software that sits directly on the edge is about as dangerous as it gets for a systems engineer. It means an attacker does not need valid credentials, an API key, or a stolen session cookie to get inside. They can simply craft a malicious payload, send it directly to the listening port of the inference server, and trick the engine into executing arbitrary commands on the underlying operating system.
Traditional infrastructure setups completely failed to catch this initial payload before execution occurred because the traffic looked entirely normal at the network layer. A Web Application Firewall or an intrusion detection system inspecting standard HTTP headers sees a standard JSON payload destined for an API endpoint. It does not know that the internal Python runtime parsing that specific JSON structure will inadvertently evaluate unsafe input.
Think of it like running a front desk in a high-security building where the receptionist accepts sealed envelopes from anyone off the street and immediately reads them aloud, without checking if the paper contains a hypnotic command. The security guards at the front door are doing their jobs, checking IDs, and making sure nobody jumps the turnstile. But the receptionist is vulnerable to a specific type of social engineering embedded right inside the permitted mail.
When SGLang processes those incoming requests, it relies on complex internal logic and external Python dependencies to handle serialization, tokenization, and request routing. If an attacker finds a flaw in how those inputs are handled, they bypass every perimeter defense you have built. You can configure firewalls and restrict outbound traffic until you are blue in the face, but if the process handling the incoming user traffic can be coerced into spawning a shell, the perimeter has already been leapfrogged.
What Happened During the NEAR AI Data Breach
Once the attacker established initial execution on that single NEAR AI inference server via the SGLang RCE exploit, the perimeter was effectively gone. Moving from a compromised inference container to internal network resources is rarely a leap across a chasm. It is usually just walking down a carpeted hallway. According to reports published by The Hacker News (opens in new tab), the intruder used that initial foothold to reach deeper into the company environment, landing squarely inside a deprecated Private Chat product that should have been isolated or decommissioned long before.
The attacker downloaded account details for roughly one hundred users from that legacy database. Those records included names, email addresses, and in specific instances, plan and balance information. Fortunately, stored chat logs and uploaded files tied to Private Chat remained untouched. That was a narrow escape, but the lateral movement did not stop at the chat database.
The attacker also pivoted to an older deployment of NEAR AI Agents that happened to sit outside the security boundaries of the company trusted execution environment. Eighteen of those older agents fell victim to the intrusion. From those unshielded systems, the attacker extracted internal logs, operational settings, and complete chat histories. Worse still, the exposed logs contained active NEAR AI Cloud API keys belonging to two distinct customers.
NEAR AI engineers moved quickly once the telemetry screamed. They revoked both exposed API keys, terminated all active sessions across the platform, and wiped out existing administrator credentials. They also force-rotated remaining agent tokens, took the vulnerable Private Chat application offline entirely, and deployed an emergency patch for the SGLang vulnerability across all active inference servers.
The incident report showed a scramble to recover control after the fact. While the immediate bleeding stopped, the attack exposed the dangerous friction points that appear when modern, rapidly moving AI infrastructure intersects with older, forgotten systems left sitting on the same internal network.
How Do Trusted Execution Environments Fail Against an SGLang RCE Exploit?
There is a common misunderstanding about confidential computing. People often assume that hardware-level isolation makes applications bulletproof. When NEAR AI processed requests in confidential mode, they relied on a trusted execution environment designed to keep data private even from the underlying host provider. Hardware enclaves secure memory regions and cryptographic keys against external snooping. Yet, they possess a glaring architectural blind spot. They protect data from the outside, but they offer zero defense against a software vulnerability living directly inside the enclave itself.
The SGLang RCE exploit bypassed standard perimeter defenses because it lived at the application layer. SGLang ran inside the trusted execution environment to process incoming inference queries. When an attacker executed arbitrary code through that software flaw, they executed it inside the security boundary, not outside it. The enclave performed exactly as designed. It encrypted the memory and isolated the process from the host operating system. Unfortunately, it trusted the execution of SGLang unconditionally.
Imagine locking your most sensitive financial records inside a titanium vault. You leave the combination written on a sticky note attached to the front door, and a burglar simply walks in and opens it. The titanium walls did not fail. The lock did not break. The system failed because the software processing the incoming requests contained a backdoor.
This brings up a hard operational truth that hosting providers and systems engineers must accept. Container isolation, virtual machine boundaries, and secure enclaves cannot patch faulty inference engine code. If an application takes raw, untrusted user input and hands it to an unvetted Python library without proper sanitization, the security perimeter becomes entirely irrelevant. The attacker does not need to break the wall if they can trick the gatekeeper into opening it from the inside.
Running my own VPS boxes has made me deeply paranoid about trusting third-party software stacks that handle raw sockets and complex input parsing. I spend hours vetting every package that touches the public internet. When teams deploy massive, fast-moving AI inference engines like SGLang, they often prioritize throughput and latency over paranoid dependency management. They want tokens per second. Security audits frequently lag behind feature releases in the open-source artificial intelligence ecosystem.
Hardware enclaves are useful tools for compliance and privacy, but they are not a substitute for secure coding practices. When an application layer vulnerability allows remote code execution, the security boundary collapses instantly. The lesson from this breach is simple. A secure container running insecure software is just a very expensive trap.
What It Means in Practice for AI Infrastructure Operators
Exposing complex inference engines directly to untrusted user input without strict sandboxing is an operational disaster waiting to happen. Most web applications sit behind mature reverse proxies and Web Application Firewalls that parse incoming JSON payloads or form data before it ever touches core business logic. Inference servers operate differently. They accept massive, unstructured text prompts, system instructions, and complex parameters designed to manipulate probabilistic outputs. When an engine like SGLang parses these inputs while simultaneously handling memory allocation, tensor routing, and token generation, the attack surface expands dramatically.
Python-heavy artificial intelligence stacks create massive blind spots for systems engineers who are used to managing traditional LAMP stacks or static web deployments. A typical AI inference environment relies on hundreds of deeply nested Python packages, custom C++ bindings, and rapid-fire GitHub checkouts to squeeze every ounce of performance out of enterprise graphics processing units. Dependency management in this ecosystem is chaotic. When a vulnerability drops in an underlying library or the core inference framework, patching it is rarely as simple as running an automated package manager update. You might break model compatibility, crash CUDA drivers, or disrupt production pipelines that clients depend on for real-time applications.
Managing my own VPS boxes has taught me to look at every new dependency with extreme suspicion. When I provision a server for a client running custom backend scripts, every socket, open port, and incoming payload is treated as hostile until proven otherwise. AI infrastructure operators often skip these basic hygiene steps because the software stack moves too fast. They pull massive container images straight from public registries, run them with elevated permissions to optimize hardware access, and assume that modern cloud infrastructure will absorb any security mistakes.
The technical debt in modern machine learning infrastructure is piling up faster than engineering teams can secure it. Based on how quickly automated exploitation tools follow public vulnerability disclosures, I expect to see widespread automated scanning targeting unpatched inference servers across public cloud providers within the next few weeks. Attackers know that teams prioritize uptime and model performance over immediate patching, creating a wide window of opportunity for anyone hunting for exposed management endpoints.
To check your exposure right now, log into your inference servers and run pip list or inspect your active container images to verify whether your SGLang installations or related inference packages require immediate updates.
What to Expect Next for SGLang and Cloud Security
The absence of a CVE assignment and detailed patch information from the initial disclosure is more than an administrative gap; it leaves operators flying blind. When NEAR AI reported the flaw to the SGLang project but neither side published a specific identifier or affected versions, security teams had no official reference point to verify their own exposure. As The Hacker News detailed in its coverage, the company has not said whether the project has released a fix, leaving everyone downstream guessing about their actual risk posture. This kind of slow, opaque response is common in open-source AI projects where maintainers are often volunteers balancing day jobs, and it creates a vacuum that attackers exploit faster than vendors can fill.
Based on how quickly automated exploitation tools follow public vulnerability disclosures, I expect to see widespread automated scanning targeting unpatched inference servers across public cloud providers within the next few weeks. Attackers know that teams prioritize uptime and model performance over immediate patching, creating a wide window of opportunity for anyone hunting for exposed management endpoints. A small e-commerce site owner might not even realize their hosting provider spun up an inference server with default settings, listening on a public IP. It is like leaving a back door open after a delivery driver comes to swap out a package; you trust the process, but the vulnerability persists until someone physically checks the lock.
Downstream services like Brave’s Leo assistant, which routes chats through NEAR AI’s infrastructure, will likely face heightened scrutiny over their backend hosting choices. When a browser company bets its reputation on a third-party inference stack, any breach in that chain becomes a public relations problem for everyone involved. I would not be surprised if we see more transparency requirements from regulators and enterprise clients about exactly which inference engines run where, and under what isolation guarantees. The SGLang RCE exploit incident should serve as a wake-up call that the security model for AI hosting needs to evolve beyond just wrapping models in containers. Operators should treat inference engines with the same suspicion they reserve for web servers handling user input, and start auditing their dependency trees before the next automated wave hits.
Closing Actionable Step
You need to audit your current inference stack right now. If you run AI workloads on your own infrastructure, or if you manage servers that handle model requests, open your terminal and check what you actually deployed. Run pip list or inspect your container images to confirm your SGLang versions are fully up to date. Too many system administrators rely on automated build pipelines that pull the latest packages from PyPI without pinning versions or checking security advisories. That approach works fine until an unauthenticated remote code execution flaw drops into a core library you forgot was even on the machine.
When I log into my own VPS boxes, I check the package managers and container registries with a healthy amount of paranoia because I know how quickly a neglected dependency can compromise an entire node. Do not assume that your cloud provider or your container base image handles security patching for application-level Python packages. They secure the hypervisor and the underlying operating system kernel, but they rarely audit the specific inference engine libraries you install on top.
Start by checking your Python environment. Look for sglang in your package list and compare the version against the official repository releases. If you find an outdated deployment, stop the service, pull the patched release, and restart your containers immediately. Check your firewall rules while you are at it. Make sure your inference servers do not expose management ports or raw API endpoints directly to the public internet without an authenticating proxy or a strict VPN tunnel in front of them. Layer your defenses so that even if an application vulnerability exists, an attacker cannot easily pivot to internal legacy systems or sensitive log files. Security in hosting is rarely about a single brilliant fix, and it is almost always about doing the boring operational maintenance before someone else forces your hand.