OpenAI replaced the Python service that fronts every data read in ChatGPT with a Rust rewrite in the second quarter of 2026, a project two engineers completed using Codex and GPT-5.5. The result handles 95% of production traffic today, runs 6x more CPU efficient and 15x more memory efficient than the service it replaced, and the Python implementation is being deprecated entirely.
Key takeaways
- Habitat, OpenAI's online storage platform, now serves more than 70 million requests per second against over 500 petabytes of data for products used by a billion people each week.
- The Python-to-Rust rewrite was carried out by two engineers leaning on Codex and GPT-5.5, and produced 6x CPU and 15x memory efficiency gains.
- The harder lessons were operational rather than linguistic: asyncio scheduling delays, LIFO connection pooling that concentrated load on already-struggling pods, and thundering-herd behavior through Envoy and Istio.
What Habitat does and why it exists
Habitat sits between OpenAI's products and the storage systems underneath them β Azure Cosmos DB, blob storage, caching layers and internal databases. When a product needs data, Habitat resolves the schema, routes the request, checks authorization, handles encryption and serialization, shapes the request and manages connection pooling. Product engineers get one interface instead of learning each backend.
It started far smaller. Habitat launched in mid-2024 as a Python client library, essentially a shared package that services imported to talk to a database. That design was fast to build and, for a while, fast to improve: adding caching or compression meant publishing a new version and letting teams pick it up.
The problem with a library is that upgrades are distributed. OpenAI describes trying to spread critical data across multiple regional database accounts to survive a regional failure, which required new routing logic inside every consuming service, deployed one by one, shadowed against live traffic, then enabled behind a feature flag. Coordinating dozens of services took days, and at least one team rolled back to a version carrying an old bug β producing exactly the outage the work was meant to prevent. Habitat was extracted into a standalone service in mid-2025, leaving a thin client SDK in products and moving routing, deployment, monitoring and access control to one place.
The failures that showed up at scale
Running that service in Python at OpenAI's request volume surfaced problems that do not appear in benchmarks. Asyncio scheduling delay from CPU-heavy background work came to dominate tail latency even when downstream calls returned quickly. In one case, every pod parsed a Statsig feature-flag configuration as JSON once a minute with no jitter, across eight processes per pod, so all workers stalled at the same moment. The fix was mundane and instructive: a smaller targeted config, a longer refresh interval, and randomized timing.
A second failure was subtler. Python's aiohttp connection pool reuses connections in LIFO order by default, which means a slower, overloaded server returns its connection later and therefore gets selected more often β traffic concentrating on the pods least able to handle it. Switching to FIFO reuse broke the feedback loop and reduced steady-state request variance as a side effect. Thundering-herd behavior through Envoy and Istio rounded out the list.
OpenAI also made a design choice that reads as a constraint and functions as a guardrail: Habitat exposes a deliberately limited NoSQL API. Queries that would be expensive or unpredictable are simply not expressible, which keeps cost per request bounded as the number of consuming products grows.
Two engineers, one rewrite
The migration detail most likely to travel is the staffing. A rewrite of a service on the critical path for a billion weekly users, from an interpreted language to Rust, was done by two engineers using OpenAI's own coding tools. That is a concrete data point in an argument that has mostly been conducted with anecdotes: language migrations have historically been rejected not because the target language was wrong but because the porting labor could not be justified, and agentic coding tools change that arithmetic.
It is worth reading the efficiency numbers carefully. A 6x CPU and 15x memory improvement reflects Python's interpreter overhead on a request-routing workload dominated by serialization and I/O coordination, which is close to the best case for a Rust rewrite. Teams whose bottleneck is a database or an external API should not expect similar returns.
The Rust ecosystem's own supply-chain exposure is a separate concern for anyone following OpenAI's path, as the incident in which a poisoned crate stayed live for 86 minutes made clear. OpenAI's engineering write-up is the first of two, with the storage layer itself covered in a follow-up.
FAQ
Why did OpenAI rewrite Habitat from Python to Rust?
Interpreter overhead became unacceptable at the service's request volume, and the operational failures the team hit β asyncio scheduling delay, connection-pool metastability β were symptoms of pushing a Python service past its comfortable range. The Rust replacement is 6x more CPU efficient and 15x more memory efficient and now serves 95% of production requests.
Does this mean Python is unsuitable for production services?
No. Habitat ran in Python through three years of better-than-10x annual growth, which is the point: the language was the right call while the product shape was still moving, and the debt was paid down deliberately once the scale justified it. The article describes a sequencing decision, not a verdict on Python.
What storage systems sit behind Habitat?
Azure Cosmos DB is the primary database layer, alongside caching, blob storage and other internal data services. Products do not address those systems directly; they call Habitat, which resolves where the data lives and how to reach it.
