Open Responses spec Joins Agentic AI Foundation

Open Responses spec joins Agentic AI Foundation to boost LLM API portability.

Two people sitting at a wooden table outside a cafe with laptops open.

Intro

The headline says the Open Responses spec has a new home. According to The New Stack’s report (opens in new tab), OpenAI donated the spec to the Agentic AI Foundation. That is the whole claim, and it is a governance claim, not a technical one.

I care less about the foundation name than about the wire format. The Open Responses spec is the thing your client library serializes into and your gateway parses back out. When someone asks me whether they can swap an upstream provider without touching application code, the answer depends on whether two servers agree on what a messages array looks like, how tool calls arrive mid-stream, and what shows up in the usage object at the end. A foundation holding the trademark does not by itself change any of those bytes.

The source material I am working from is thin. What I have is The New Stack’s piece on the donation, plus a lot of newsletter boilerplate around it, and the article itself does not hand over a version number, a list of breaking changes, or the ratification process the foundation will use. Dates and maintainer names need checking against the foundation’s own announcement before anyone treats them as settled. I am flagging that here because a spec transfer announcement is exactly the kind of news where people screenshot the headline and skip the repo.

There is a reason to pay attention anyway. Every self-hosted LLM gateway configuration I have written in the last two years has been a pile of per-provider exceptions wrapped around one OpenAI-shaped core. If a neutral spec with real conformance tests exists, that pile gets smaller. If it exists as a document with no test suite, nothing changes and the OpenAI shape stays the de facto standard regardless of who nominally owns it.

The rest of this post separates what the headline actually claims from what would have to ship for it to matter to someone running their own boxes.

Background: what an LLM API standard actually is

When a vendor advertises “OpenAI API compatible,” they are claiming something narrower than the phrase suggests. In practice it means a POST /v1/chat/completions endpoint that accepts a JSON body with model, messages, and optionally tools, tool_choice, temperature, and stream. The reply is either a single JSON object with choices[0].message, or a stream of SSE frames each prefixed data: and terminated by a literal data: [DONE]. Tool calls in a stream arrive as fragments inside delta.tool_calls[0].function.arguments, and the client is responsible for gluing those string pieces back into valid JSON before parsing. That assembler loop is where most compatibility breaks quietly.

Everyone copies this shape because copying is free. Client SDKs already exist in every language, so a new inference server that speaks the same dialect gets working clients on day one. Gateways like LiteLLM already parse it, which means a provider that matches the shape appears in existing configs without a new parser. Requesting a rewrite is a much harder sell than matching a JSON schema.

There have been attempts at neutral, vendor-independent descriptions of model APIs before, and they mostly lost to whatever the biggest client library already shipped. A spec that nobody’s SDK imports is a PDF, not a standard. The pattern repeats enough that I have stopped expecting a committee document to displace a working wire format; the format wins first and the document catches up. Worth separating here: MCP standardizes how a model reaches tools and data sources, not the request and response envelope of a chat call. People conflate the two constantly.

The catch with this particular piece of news is that The New Stack’s report on the OpenAI donation (opens in new tab) does not give me a version number, a diff against the current OpenAI shape, or the process by which changes get approved. What it describes is a transfer of custody. A foundation receiving a spec repo typically inherits a trademark, the document itself, maybe reference implementations and conformance tests, and the obligation to run a review process. Which of those items actually moved here is not stated in the material I have. Anyone quoting the headline as proof of a ratified, testable standard is ahead of the source.

What’s happening now with the Open Responses spec

The headline is short: OpenAI handed the spec to the Agentic AI Foundation. That is a custody change, not a release. The wire format shipping inside OpenAI’s SDKs today does not change because a repository moved between two GitHub orgs, and no existing client stops working.

A transfer like this usually carries a bundle. Trademark on the name, the spec repository with its commit history, reference implementations, conformance tests if any exist, and sometimes a charter describing who votes and how fast. I have watched open source projects change foundations before, and the repo move is the easy half. The trademark and the test suite decide whether the new home can actually enforce anything when a provider ships something adjacent to the spec and calls it compatible. The New Stack’s report on the donation (opens in new tab) does not itemize which pieces moved, so I am treating the full list as plausible rather than confirmed.

Three things are missing from the material I have: a version number, a statement about breaking changes against the current OpenAI shape, and the ratification process. Those matter more than the foundation’s letterhead to anyone writing gateway config. A spec sitting at v0.x with a loose review process behaves very differently from a numbered release with a documented compatibility promise, and I cannot tell from the headline which one this is.

On the Agentic AI Foundation itself, what I can say is limited to what is stated: it is the recipient of this donation. I am not going to fill in a member list or a governance model that the source does not describe. If the foundation publishes a charter, read it. If it publishes only a blog post, note that too.

Think of it like a landlord handing over a shop lease. The signage can change overnight. Whether the locks get changed, whether the fire inspection certificate carries over, and who signs off on renovations are separate questions. Here, custody is the signage.

My read is that foundation custody only improves the portability story if conformance tests ship alongside the spec. Without a test use, a provider can claim Open Responses compatibility while quietly dropping usage fields or chunking tool-call arguments in its own way. A vendor-neutral repository with no tests is a press release with a license file attached. I expect the next useful signal to be either a test suite or a changelog, and until one of those appears, the practical state of things has not moved.

Does the Open Responses spec make self-hosted LLM gateways easier?

For someone running their own gateway, the honest answer is that nothing changes at runtime today. A spec document does not restart your reverse proxy. What it can change is the assumption you are allowed to build on: that the request body going into /v1/chat/completions and the stream coming back have a fixed shape, so one gateway config can sit in front of more than one upstream provider without a rewrite per provider.

Does OpenAI API compatibility go away? No. The spec started from that shape, so compatibility with it is the entire premise. If a future version drifts from what your SDK already sends, that is a version you should read about before upgrading, not something a foundation transfer does to you silently.

The part worth checking before you point production traffic at a new provider is field-level behavior, not the spec’s name. Four things bite me every time:

  • Streaming deltas. Does the provider emit data: [DONE] at the end, or just stop? Does it send an empty choices array on the first frame the way some do?
  • Tool-call argument chunking. Arguments arrive as a JSON string split across frames. Some upstreams send one large chunk on the final frame, others drip it character by character. If your gateway concatenates by index incorrectly, you get truncated JSON and a parse error on a model that is working fine.
  • Error codes. A 429 with no Retry-After header is a different retry strategy from one that has it. A 400 for a malformed tool schema is not the same as a 500.
  • Token accounting. Whether usage appears at all in streaming responses, and whether prompt_tokens includes the system message. Invoicing and rate limits hang off those numbers.

The analogy I keep coming back to as a host: it is the difference between a VPS provider promising “a Linux VM” and one promising a specific kernel version with a specific init. The generic promise is fine until you need to know whether your startup script runs before or after networking. Both providers can claim the same spec and still hand you a machine that boots differently.

So the spec makes gateways easier in the same way a shared plug standard makes adapters easier: only once the devices actually get tested against it. Until then, the work stays in your adapter layer.

What it means in practice

The spec promises portability. What you actually get on day one is the same pile of edge cases you already had, now with a document claiming they should not exist.

finish_reason is my favourite example. Most providers return stop or length. Some return tool_calls, some return function_call, some return stop with a tool call in the payload and expect you to infer the rest. If your agent loop branches on that field, a new upstream can silently end a run early and you spend an hour blaming the model. Retry headers have the same problem. Retry-After in seconds, Retry-After as an HTTP date, x-ratelimit-reset-requests as a string like 1s, or nothing at all. Your backoff logic has to survive all four.

What I do on my own boxes is boring and it has saved me twice. I record one real tool-calling stream from the provider I trust, save it to fixtures/toolcall-stream.txt, then replay the same request against a new upstream and diff the frames:

curl -N -s https://newprovider/v1/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -d @fixtures/toolcall-request.json > /tmp/new.txt
diff <(sed 's/"id":"[^"]*"/"id":"X"/g' fixtures/toolcall-stream.txt) \
     <(sed 's/"id":"[^"]*"/"id":"X"/g' /tmp/new.txt)

IDs and timestamps differ every run, so normalise them first or the diff is noise. What you are looking for is structural: same number of frames, same delta keys, same terminator. That diff is the only honest portability test I know of, and it takes about ten minutes to set up.

Adoption is a client-library question more than a spec question. If the Python and Node SDKs your app imports, and the gateway plugin you run, both speak the format, the spec is real for you. If they do not, it is documentation you are reading for fun. Plenty of specs with foundation backing have died in that gap.

The migration cost also hides in the unhappy paths. Happy path is one POST and a parse. The days you lose are error handling: retries that fire twice, rate limits that fail open, a 400 that your code treats as retryable and hammers an upstream until it bans you. Keep that logic in one adapter file you own, not smeared across every call site, and swapping providers later stays a one-file diff.

What to expect next

The first thing I would watch for is a conformance suite, and a versioned changelog next to it. A spec repository with prose and no tests is a press release with extra steps. Drift comes back fast when there is nothing executable that fails when a provider bends a rule. My expectation, and it is only that, is a test use that runs a fixed set of requests against an endpoint and reports pass or fail on streaming shape, tool-call chunking, and finish_reason handling. If that ships within a couple of release cycles, the transfer means something. If a year passes and the repo holds only a README and a spec.md, the de facto shape stays whatever OpenAI ships, and the foundation name does no work.

Governance is the second thing. A foundation transfer usually settles ownership of the trademark and the repo. It rarely settles the question that matters to anyone writing client code: who approves a change, how long the comment window is, and whether one vendor can stall a merge it dislikes. A single upstream provider with a large voting block could keep the spec frozen at the shape it already ships, which is comfortable for compatibility and useless for portability. I do not know how this particular foundation is weighted. I would read the charter before I read the spec.

The likely outcome, as far as I can guess, is a thin compatibility layer over the OpenAI shape rather than a fresh design. That is the right call. Nobody rewrites every SDK and gateway to gain a cleaner schema. The value is in pinning down what is already in production.

What stays genuinely uncertain is self-hosted inference. Servers like vLLM, llama.cpp, and Ollama already emit something close to the OpenAI shape, mostly by translation. Whether they adopt a foundation-blessed format as a first-class response type or keep translating is an open question, and translation is where field-level bugs live. A first-class implementation is a much stronger portability signal than an adapter someone wrote to shut up a failing SDK.

Watch the changelog and the test runner. Everything else is commentary.

What to do this week

Pin your gateway and client SDK versions before you touch anything else. If you run LiteLLM, that is the image tag in your compose file; if it is a self-hosted vLLM box, it is the release you installed and the openai python package your app pulls in. Unpinned, a spec change becomes an incident on a Tuesday night instead of a red test in CI.

Then record one real streaming response and keep it. Tool calling with two tools and a multi-turn history is the useful case, because that is where the frames get complicated:

curl -sS -N https://gateway.internal/v1/chat/completions \
  -H 'content-type: application/json' \
  -d @recorded-tool-call.json > frames.ndjson

Replay that file against every provider and every gateway version you run, and diff the frames. Not the rendered text, the frames. You want to see whether delta.tool_calls[0].function.arguments arrives as one fragment or five, whether finish_reason is tool_calls or stop, whether the usage object is present on the final chunk and whether usage.total_tokens equals prompt plus completion. On my own boxes this is a twenty-line pytest file that asserts on the parsed frames, and it has caught more upstream drift than any changelog I have read.

Keep the provider-specific corrections in a single adapter file you own. One function that takes the raw response and returns your internal shape, plus one for the stream. When Cloudflare Workers AI or a local llama.cpp build returns an error body without error.code, you fix it in one place instead of hunting call sites. If you are on LangChain or LlamaIndex, that adapter is still worth writing outside the framework, because upgrading their abstractions is a bigger diff than patching your own wrapper.

Last: open the foundation’s spec repo and read the commits between the last tagged version and now. Check whether conformance tests exist and whether they run in CI on pull requests. The headline said the spec moved. The changelog is the only thing that tells you what moved inside it. Read that before you edit gateway.yaml, and if the repo has no changelog yet, treat every version bump as a breaking change until you have proven otherwise with your replay test.