AI Inference Data Leak: Why 155 Is the Wrong Number

Cloudflare disclosed that 155 responses from its Workers AI platform came back carrying fragments of other customers' prompts and partial generations. The figure is small, it is precise, and it is the least useful number in the entire disclosure — because the defect that produced it raises no error and writes no log line. 155 is the number of leaks Cloudflare could infer from the shape of its request logs, not the number that occurred. For this class of failure, no hosted-inference provider can tell you after the fact whether your prompts ended up inside somebody else's answer.

That is the sentence worth carrying into your next vendor conversation, and it has very little to do with Cloudflare's competence.

What is verified here, and what is the vendor's word

This post is built from two kinds of claim, and blurring them would make it worth less.

Checked independently, against GitHub's API and the engine maintainers' own security post: the dates, the public bug report that preceded the launch by seven weeks, the release history, and the statement that the corruption leaves no trace. Those are public record, and we read them ourselves on 27 October 2026.

Reported by Cloudflare about Cloudflare, with no outside party able to confirm or bound it: the count of 155; that every affected request carried a prompt longer than 681,280 tokens; that no other Workers AI model was affected; that there is "no evidence of deliberate exploitation"; and the incident-response timeline. Cloudflare is the only party holding those logs. And by the mechanism described in Cloudflare's own post, those logs cannot show the leak directly — so 155 is a floor derived from inference about request shape, not a measurement. Reading it as a ceiling is the error this post is about.

Saying so is not a dig at the disclosure. Cloudflare's post is unusually candid, including about its own testing. It is a statement about what evidence exists.

The failure signature: every light was green

The model was @cf/zai-org/glm-5.3, served with prefill and decode split across separate nodes — the prefill node computes the prompt's key-value cache, ships it over the network, and the decode node generates from it. A request whose prompt was too long for the prefill worker got rejected by the scheduler. The rejection was stored in the wrong field, so both admission queues checked req.finished(), saw False, and waved it through as live.

The prefill node then swapped the prompt for a one-token placeholder and transferred a single page of cache. The decode node had already reserved eight. Attention read all eight anyway. Seven of them still held whatever the previous occupants had left there: other customers' prompts and partial generations.

What the client received was an HTTP 200 that should have been a 400, a prompt_tokens count matching the real prompt, latency better than expected, and a coherent, well-formed answer.

Cloudflare's own diagram. The rejected request clears both admission queues, one real page is written, and attention reads all eight. The client gets HTTP 200 and a fluent answer assembled from seven pages of someone else's data. Source: Cloudflare.

This is the part the coverage restated without working out. In an ordinary CRUD system, a read of uninitialised memory produces a crash, a null, a stack trace — something an assertion can catch. In a language model, a corrupted read produces fluent text, because the decoder will happily continue from whatever bytes are sitting in the attention window. The failure does not announce itself. It renders.

Why the tests passed

Cloudflare states the gap in its own words, and it is the most transferable sentence in the post:

This was one critical gap in our testing: we only validated HTTP response codes and engine performance at long contexts, without validating model output and coherence.

The admission that makes this everyone's problem rather than Cloudflare's. The team knew the cache pool was smaller than the advertised context window and shipped, because the tests came back green — and the tests asserted on status codes. Source: Cloudflare.

Read that as a design lesson, not a confession. Status-code-and-smoke-test acceptance testing is entirely adequate for a REST API. Against an inference engine it is structurally blind, because the system's failure mode is a 200 with good prose. The blindness is a property of the architecture, not of anyone's diligence — which is precisely why your tests have it too.

The bug was public seven weeks before launch

Here is the part nobody else checked. The exact defect was filed on GitHub as sgl-project/sglang#30233 on 6 July 2026 — 52 days before Cloudflare launched GLM 5.3 on Workers AI on 27 August. It is not a vague complaint: it names set_finish_with_abort, the one-token truncation and the decode side reading uninitialised cache slots, in six steps, with a working patch and a reproduction recipe. Its environment block lists the same model family, the same transfer backend and the same million-token context length as Cloudflare's deployment.

Issue #30233, 6 July 2026. The root cause is complete and correct. The symptom is filed as garbage output — the same uninitialised cache pages Cloudflare would later have to report as other customers' data. Source: github.com/sgl-project/sglang.

The title calls it "garbage output". The words leak, tenant and security do not appear anywhere in it. Step four of the reproduction reads: "decode returns garbage content unrelated to the input."

Same uninitialised memory. Two readers. One filed a quality defect; the other had to declare a data breach. The severity was decided by who read the symptom, not by the bug.

And the issue was not closed by a fix. An inactivity bot closed it as completed and tagged it inactive on 4 September — four days before the security report from Knostic researcher Gadi Evron reached Cloudflare via HackerOne on 8 September.

No human closed it. A staleness bot marked the public report of this bug completed on 4 September; the security report landed on 8 September. Source: github.com/sgl-project/sglang.

The two numbers that outrank 155

The engine maintainers published a coordinated disclosure alongside Cloudflare's. For the related aborted-transfer race — same signature, different trigger — it states that within disaggregated deployments, default configurations were affected from the v0.4.x line in early 2025 until v0.5.21 on 2 October 2026. More than eighteen months, on by default, across everyone running that topology.

Then the sentence that should reset how you read every breach notice you get from an AI vendor:

A deployer has no reliable way to tell [whether exposure already occurred], because the corruption leaves no error or log line.

The two sentences that outrank the headline number: default configurations affected since early 2025, and no reliable way for any operator to determine after the fact whether data was exposed. Source: SGLang / LMSYS.

The fix that closes the default-on path measured at −0.38% throughput, with a confidence interval spanning zero. So the isolation property was not traded away for speed anyone could measure. It was simply off by default for a year and a half, because nobody had framed it as a security property yet.

Which means "we found no evidence of exposure" is a statement about the absence of instrumentation, not the absence of a breach. Both can be true at once. Only one of them is what a buyer hears.

Three things to do on Monday

Not opinions — decisions, in the order they cost you money.

1. Ask your inference provider, in writing, which engine and which version they run, whether they use prefill/decode disaggregation, and whether transfer checksums are enabled. You signed a data-processing agreement promising your customers an isolation property. If your provider cannot name the version, they cannot tell you whether they were inside that window — and the obligation stays yours either way.
2. Make acceptance tests assert on output content, not response codes, before anything AI-shaped ships. A golden-output check, a coherence assertion, a canary prompt whose answer you know — anything that would notice a fluent wrong answer. A 200 is not a pass.
3. Budget for instrumentation rather than for assurance. "No evidence of exposure" should read to you as a line item, not a clean bill of health. If a leak in your stack would leave no trace, that is the thing to fix first, ahead of the leak you have not had yet.

The same discipline we apply to containing AI-written code applies to the AI serving it, and we wrote about an agent that quietly muted its own security alerts for the same reason. In AI systems the default shape of a failure is a confident, well-formed answer. Correctness has to be asserted on content, or it is not asserted at all. That is most of what our production work consists of.

Key takeaways

• Cloudflare's 155 affected responses is a floor inferred from request logs, not a measured total. No external party can confirm or bound it.
• The bug produced HTTP 200, matching token counts, lower-than-expected latency and coherent prose. Every signal a competent test asserts on was green.
• The same class of defect sat in the inference engine's default disaggregated configuration from early 2025 until 2 October 2026.
• The corruption leaves no error and no log line, so no operator — Cloudflare included — can retroactively prove that a given customer's prompts did not leak.
• The fix costs −0.38% throughput, inside the measurement noise. This was never a performance trade-off.

FAQ

Was my data leaked? If you used GLM 5.3 on Workers AI between 27 August and 9 September 2026 with prompts over roughly 681,000 tokens, Cloudflare can tell you whether you were in its inferred set. Outside that, and for any other provider running disaggregated serving, the honest answer is that nobody can establish it either way — which is the point of this post.

Is this a Cloudflare problem or an SGLang problem? Neither, usefully. The admission bug is in the engine; the decision to ship against a cache pool smaller than the advertised context window was Cloudflare's; the testing pattern that missed it is the industry's. Cloudflare is one of the few providers that published enough detail to be criticised at all.

Does this affect single-node inference? The specific aborted-transfer race requires prefill/decode disaggregation with a cache-transfer backend. A single-instance deployment does not have that path. The general lesson — a corrupted read in an LLM surfaces as fluent text, not as an error — is not architecture-specific.

What should be in my vendor questionnaire now? Engine and version, whether disaggregated serving is in use, whether transfer integrity checks are enabled, and what instrumentation would detect cross-request contamination if it happened. The last one is the question almost nobody asks and the only one that produces evidence.