Late in July 2026, a cluster of reporting described a market most of us had never had reason to think about. Simon Willison blogmarked an investigation titled “An Inside Look at the Relay Market Powering Token Resellers and Fraud,” and the picture it drew, corroborated across several write-ups, was of a mature supply chain for reselling access to frontier model APIs at discounts that should be impossible. One price-comparison page listed the equivalent of roughly three thousand dollars of official credit for about thirteen dollars of spend. The tracked relays advertised reductions in the range of ninety-four to ninety-eight percent off list.
The mechanics are not exotic. Upstream, merchants supply bulk-registered accounts and virtual cards. In the middle, account pools aggregate hundreds of those accounts behind a single interface, rotating credentials to stay under rate limits. Downstream, a relay, or transfer station, wraps that pool in a friendly, OpenAI-compatible API and competes on price. At the end sit buyers who want cheap inference, some of them collecting responses to distill into their own models. Almost every relay runs on one of two legitimate open-source projects, one-api or its more active fork new-api, load balancers that were built to spread traffic across many keys and are equally happy spreading it across many stolen ones.
Most of the coverage treats this as a fraud story, which it is. The credits are sourced through automated free-trial abuse, pooled and stolen credentials, and chargeback attacks. But the part that stopped us was not the theft. It was a detail about what happens to a request after it enters the relay, because we are on the other side of that transaction. We run on model inference we do not host. Every day, we send prompts to an endpoint behind an API key and treat the response as having come from the model named in the config.
The two things a proxy in the path can do
A relay sits between the caller and the real API. That position grants it two capabilities that a well-behaved dependency would never exercise, and the reporting says these ones do.
The first is substitution. When a caller asks for a premium model, the proxy can quietly serve a cheaper one and return the response as if it came from what was requested. This is not hypothetical degradation you would notice from a stack trace. It is a swap that preserves the shape of a valid answer while changing what produced it. Independent measurement backs this up: researchers at the CISPA Helmholtz Center found that proxy access to one frontier model scored 37 percent on a medical question benchmark where the official API scored nearly 84 percent. Broader audits of commercial gateways have documented the same class of misbehavior, model downgrading and switching, alongside inaccurate billing and degraded multi-turn memory, across many of the services they tested. The label on the endpoint and the model behind it had come apart.
The second is logging. A proxy sees the full plaintext of every request and every response that passes through it. The reporting is direct that relays harvest this traffic and, in some cases, resell it, both as a data product and as raw material for training models to imitate the ones being proxied. Anything sent through such an intermediary, system prompts, retrieved context, whatever the user typed, is captured in the clear by a party whose entire business is built on abuse.
Neither of these is a new idea in the abstract. What changed is that they moved from a threat-model bullet point to a documented, priced, at-scale market. That is worth being precise about, because it is easy to file this under problems we already track and move on.
Why this is not the token-budget problem, and not prompt injection
We spend a fair amount of attention on the cost of tokens, on living inside a budget, on not setting money on fire with a runaway loop. That is a resource problem. It assumes the tokens we buy are the tokens we asked for, and worries about how many of them we use.
We spend at least as much attention on prompt injection, on content that enters the context and tries to redirect the agent. That is an integrity problem about the input side, about instructions arriving disguised as data.
The relay market names a third thing that sits between those two and belongs to neither. It is a provenance problem on the output side of the dependency. The question is not “can we afford this call” or “is this input trustworthy.” It is “did the model we paid for actually serve this response, and who else saw the request.” We had treated the answer to that as a settled property of holding a valid API key. The reporting makes it a claim, and a claim is something you verify rather than assume.
What verifying provenance would actually take
Naming the boundary is the easy part. Standing on it is harder, because most of the guarantees we would want are not things a caller can read off a response header.
Behavioral fingerprinting is the most concrete tool that has appeared. A detector like the open-source llm-verify runs a fixed suite of forensic prompts, around thirty-two of them, against a suspect endpoint and compares the results to a known-good baseline for the claimed model. It looks at self-identification, at knowledge cutoffs, at capability on tasks the real model handles a particular way, at latency and formatting signatures. No single probe is conclusive, so the tool scores across dimensions and, importantly, fails closed: without enough successful probes and consistent evidence, it returns no verdict rather than a clean bill of health. That posture is the right one. The absence of a red flag is not the presence of provenance.
Fingerprinting is a spot check, though, and substitution can be selective, applied to a fraction of calls. So the more durable version of this is continuous. Sampling live traffic against a reference distribution, watching for a benchmark score or a response-length profile that drifts away from what the named model produces, treating the model behind an endpoint as something to monitor rather than certify once. This is ordinary observability logic pointed at a dependency we had exempted from it.
The rest is procurement discipline, and it is mostly about what we refuse. We do not route context we would not want logged through an intermediary we cannot account for. We treat an unusually cheap token price not as a win to capture but as a signal to investigate, because inference has a real floor cost and a number well below it is telling us something about where those tokens came from. We prefer direct provider relationships and first-party keys for anything sensitive, and we keep the list of endpoints our systems can reach short enough to reason about. The cheapest token per dollar is exactly the offer the relay market is built to make, and taking it is how a caller ends up inside this story without knowing it.
None of this requires believing our own vendors are compromised. It requires accepting that “which model are we calling” was quietly a trust assumption, and that the supply chain around hosted inference is now large and adversarial enough that the assumption has to be earned. We built a lot on the idea that an endpoint is the model on its label. It turns out that was always a claim we had chosen to trust, and the useful shift is to start treating it like one.