1d:I[70616,["/_next/static/chunks/0kcf5dt6yxdox.js","/_next/static/chunks/1su5n1uv1ti4s.js","/_next/static/chunks/0gvke1wv6udfg.js","/_next/static/chunks/1dfrle3jgmlzh.js","/_next/static/chunks/162lgvduyr-fa.js","/_next/static/chunks/2gcnnopam8e9w.js","/_next/static/chunks/2g60iab5a1pkg.js"],"Breadcrumb"] 1e:I[90762,["/_next/static/chunks/0kcf5dt6yxdox.js","/_next/static/chunks/1su5n1uv1ti4s.js","/_next/static/chunks/0gvke1wv6udfg.js","/_next/static/chunks/1dfrle3jgmlzh.js","/_next/static/chunks/162lgvduyr-fa.js","/_next/static/chunks/2gcnnopam8e9w.js","/_next/static/chunks/2g60iab5a1pkg.js"],"ArticleGrid"] 1f:T1948,

OpenAI replaced the Python service that fronts every data read in ChatGPT with a Rust rewrite in the second quarter of 2026, a project two engineers completed using Codex and GPT-5.5. The result handles 95% of production traffic today, runs 6x more CPU efficient and 15x more memory efficient than the service it replaced, and the Python implementation is being deprecated entirely.

Key takeaways

Habitat, OpenAI's online storage platform, now serves more than 70 million requests per second against over 500 petabytes of data for products used by a billion people each week.
The Python-to-Rust rewrite was carried out by two engineers leaning on Codex and GPT-5.5, and produced 6x CPU and 15x memory efficiency gains.
The harder lessons were operational rather than linguistic: asyncio scheduling delays, LIFO connection pooling that concentrated load on already-struggling pods, and thundering-herd behavior through Envoy and Istio.

What Habitat does and why it exists

Habitat sits between OpenAI's products and the storage systems underneath them — Azure Cosmos DB, blob storage, caching layers and internal databases. When a product needs data, Habitat resolves the schema, routes the request, checks authorization, handles encryption and serialization, shapes the request and manages connection pooling. Product engineers get one interface instead of learning each backend.

It started far smaller. Habitat launched in mid-2024 as a Python client library, essentially a shared package that services imported to talk to a database. That design was fast to build and, for a while, fast to improve: adding caching or compression meant publishing a new version and letting teams pick it up.

The problem with a library is that upgrades are distributed. OpenAI describes trying to spread critical data across multiple regional database accounts to survive a regional failure, which required new routing logic inside every consuming service, deployed one by one, shadowed against live traffic, then enabled behind a feature flag. Coordinating dozens of services took days, and at least one team rolled back to a version carrying an old bug — producing exactly the outage the work was meant to prevent. Habitat was extracted into a standalone service in mid-2025, leaving a thin client SDK in products and moving routing, deployment, monitoring and access control to one place.

The failures that showed up at scale

Running that service in Python at OpenAI's request volume surfaced problems that do not appear in benchmarks. Asyncio scheduling delay from CPU-heavy background work came to dominate tail latency even when downstream calls returned quickly. In one case, every pod parsed a Statsig feature-flag configuration as JSON once a minute with no jitter, across eight processes per pod, so all workers stalled at the same moment. The fix was mundane and instructive: a smaller targeted config, a longer refresh interval, and randomized timing.

A second failure was subtler. Python's aiohttp connection pool reuses connections in LIFO order by default, which means a slower, overloaded server returns its connection later and therefore gets selected more often — traffic concentrating on the pods least able to handle it. Switching to FIFO reuse broke the feedback loop and reduced steady-state request variance as a side effect. Thundering-herd behavior through Envoy and Istio rounded out the list.

OpenAI also made a design choice that reads as a constraint and functions as a guardrail: Habitat exposes a deliberately limited NoSQL API. Queries that would be expensive or unpredictable are simply not expressible, which keeps cost per request bounded as the number of consuming products grows.

Two engineers, one rewrite

The migration detail most likely to travel is the staffing. A rewrite of a service on the critical path for a billion weekly users, from an interpreted language to Rust, was done by two engineers using OpenAI's own coding tools. That is a concrete data point in an argument that has mostly been conducted with anecdotes: language migrations have historically been rejected not because the target language was wrong but because the porting labor could not be justified, and agentic coding tools change that arithmetic.

It is worth reading the efficiency numbers carefully. A 6x CPU and 15x memory improvement reflects Python's interpreter overhead on a request-routing workload dominated by serialization and I/O coordination, which is close to the best case for a Rust rewrite. Teams whose bottleneck is a database or an external API should not expect similar returns.

The Rust ecosystem's own supply-chain exposure is a separate concern for anyone following OpenAI's path, as the incident in which a poisoned crate stayed live for 86 minutes made clear. OpenAI's engineering write-up is the first of two, with the storage layer itself covered in a follow-up.

FAQ

Why did OpenAI rewrite Habitat from Python to Rust?

Interpreter overhead became unacceptable at the service's request volume, and the operational failures the team hit — asyncio scheduling delay, connection-pool metastability — were symptoms of pushing a Python service past its comfortable range. The Rust replacement is 6x more CPU efficient and 15x more memory efficient and now serves 95% of production requests.

Does this mean Python is unsuitable for production services?

No. Habitat ran in Python through three years of better-than-10x annual growth, which is the point: the language was the right call while the product shape was still moving, and the debt was paid down deliberately once the scale justified it. The article describes a sequencing decision, not a verdict on Python.

What storage systems sit behind Habitat?

Azure Cosmos DB is the primary database layer, alongside caching, blob storage and other internal data services. Products do not address those systems directly; they call Habitat, which resolves where the data lives and how to reach it.

a:["$","div",null,{"className":"mx-auto max-w-7xl px-4 py-8 sm:px-6","children":[["$","script",null,{"type":"application/ld+json","dangerouslySetInnerHTML":{"__html":"{\"@context\":\"https://schema.org\",\"@type\":\"BreadcrumbList\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https://ainewsway.com\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"#infrastructure\",\"item\":\"https://ainewsway.com/tag/infrastructure\"}]}"}}],["$","script",null,{"type":"application/ld+json","dangerouslySetInnerHTML":{"__html":"{\"@context\":\"https://schema.org\",\"@type\":\"ItemList\",\"@id\":\"https://ainewsway.com/tag/infrastructure#itemlist\",\"name\":\"#infrastructure Articles\",\"numberOfItems\":1,\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"url\":\"https://ainewsway.com/articles/two-engineers-and-codex-rewrote-the-service-behind-every-chatgpt-data-read-in-rust-mty63tkk\",\"name\":\"Two Engineers and Codex Rewrote the Service Behind Every ChatGPT Data Read in Rust\"}]}"}}],["$","$L1d",null,{"items":[{"label":"# infrastructure"}]}],["$","h1",null,{"className":"mb-8 text-3xl font-bold text-zinc-900 dark:text-zinc-100","children":["#","infrastructure"]}],[["$","$L1e",null,{"articles":[{"id":"ddc3ce91-8653-4be0-9436-36bd1ae44038","slug":"two-engineers-and-codex-rewrote-the-service-behind-every-chatgpt-data-read-in-rust-mty63tkk","title":"Two Engineers and Codex Rewrote the Service Behind Every ChatGPT Data Read in Rust","subtitle":"OpenAI's Habitat now serves 70 million requests a second, and the Python it replaced left a detailed list of scars","content":"$1f","excerpt":"OpenAI's storage platform moved from Python to Rust in Q2 2026, gaining 6x CPU and 15x memory efficiency. The migration took two engineers using Codex and GPT-5.5.","coverImage":"https://cdn.ainewsway.com/uploads/2026/09/243fe610-e169-453e-bf21-45dcb6feb855.webp","coverImageCaption":"Racks in a data center, the layer OpenAI's Habitat platform abstracts away for ChatGPT, Codex and its API products","authorId":"6773f67d-d7fa-4b19-8f38-384f10ac761b","status":"PUBLISHED","publishedAt":"2026-09-12T09:13:48.815Z","viewCount":1,"likeCount":0,"readingTime":5,"isFeatured":false,"isBreakingNews":false,"metaTitle":"OpenAI Rewrote Its Storage Service in Rust With 2 Engineers","metaDescription":"OpenAI's Habitat moved from Python to Rust in Q2 2026 for 6x CPU and 15x memory gains at 70M requests a second. Two engineers did it with Codex.","metaKeywords":["OpenAI Habitat","Python to Rust migration","ChatGPT infrastructure","Codex rewrite","distributed storage platform","asyncio tail latency","Azure Cosmos DB","scaling engineering"],"ogImage":null,"aiSummary":"OpenAI rewrote Habitat, the storage platform every ChatGPT data read passes through, from Python to Rust in the second quarter of 2026 using two engineers, Codex and GPT-5.5. The Rust service is 6x more CPU efficient and 15x more memory efficient and now handles 95% of production traffic across more than 70 million requests per second. The write-up details the asyncio and connection-pooling failures that forced the move.","aiTags":[],"sentiment":null,"articleType":"ANALYSIS","videoEmbed":null,"videoType":null,"seoScore":85,"isAutoGenerated":true,"isBackfill":false,"pipelineStage":null,"copyrightScore":null,"similarityScore":null,"matchedSourceId":null,"createdAt":"2026-09-12T09:13:23.829Z","updatedAt":"2026-09-12T12:05:08.617Z","contentUpdatedAt":"2026-09-12T12:05:08.616Z","deletedAt":null,"author":{"id":"6773f67d-d7fa-4b19-8f38-384f10ac761b","name":"Seung Jung","username":"Seung Jung","avatar":null},"categories":[{"id":"4bbc00ee-63c2-40b5-a588-8b0fb754c6f9","name":"SaaS & Cloud","slug":"saas-cloud"}],"tags":[{"id":"95546cf6-8b8e-4ec4-829b-f9988ab8674c","name":"OpenAI","slug":"openai"},{"id":"b5a82650-8d25-479c-ba03-ea80944a5d79","name":"Rust","slug":"rust"},{"id":"c7c539a8-3b82-4395-98ce-1a285d235daf","name":"ChatGPT","slug":"chatgpt"},{"id":"97461414-fad1-47bc-9c14-a3e2d70c304d","name":"Codex","slug":"codex"},{"id":"fef1d6da-dc95-437b-b9e1-bed51c645718","name":"Infrastructure","slug":"infrastructure"}],"translations":[{"id":"9c01fcd5-05fa-4369-aabc-a7657d5a87b0","language":"ko","title":"엔지니어 2명과 Codex가 모든 ChatGPT 데이터 읽기를 떠받치는 서비스를 Rust로 다시 썼다","subtitle":"OpenAI의 Habitat은 이제 초당 7000만 건을 처리한다. 교체된 Python은 상세한 상처 목록을 남겼다","excerpt":"OpenAI의 스토리지 플랫폼이 2026년 2분기에 Python에서 Rust로 옮겨가며 CPU 효율 6배, 메모리 효율 15배를 얻었다. 마이그레이션은 Codex와 GPT-5.5를 쓴 엔지니어 두 명이 해냈다.","metaTitle":"OpenAI, 엔지니어 2명으로 스토리지 서비스를 Rust로 재작성","metaDescription":"OpenAI의 Habitat이 2026년 2분기에 Python에서 Rust로 전환해 초당 7000만 요청에서 CPU 6배·메모리 15배 효율을 달성했다. 엔지니어 두 명이 Codex로 끝낸 작업이다."},{"id":"9fd6b720-9f62-4e11-8528-4feb2fa67817","language":"ja","title":"エンジニア2人とCodexが、ChatGPTの全データ読み取りを支えるサービスをRustで書き直した","subtitle":"OpenAIのHabitatは毎秒7000万リクエストを処理する。置き換えられたPythonは詳細な傷跡のリストを残した","excerpt":"OpenAIのストレージ基盤が2026年第2四半期にPythonからRustへ移行し、CPU効率6倍・メモリ効率15倍を得た。移行はCodexとGPT-5.5を使ったエンジニア2人が担った。","metaTitle":"OpenAI、エンジニア2人でストレージ基盤をRustに書き換え","metaDescription":"OpenAIのHabitatが2026年第2四半期にPythonからRustへ移行し、毎秒7000万リクエストでCPU6倍・メモリ15倍の効率を達成。エンジニア2人がCodexで完遂した。"},{"id":"acb7dac5-ec6a-40d5-9cff-d66c858ae197","language":"zh","title":"两名工程师加上Codex，用Rust重写了支撑ChatGPT所有数据读取的服务","subtitle":"OpenAI的Habitat如今每秒处理7000万次请求，被替换掉的Python留下了一份详尽的伤痕清单","excerpt":"OpenAI的存储平台在2026年第二季度从Python迁移到Rust，获得6倍CPU与15倍内存效率。迁移由两名使用Codex和GPT-5.5的工程师完成。","metaTitle":"OpenAI用两名工程师把存储服务重写为Rust","metaDescription":"OpenAI的Habitat在2026年第二季度从Python迁至Rust，在每秒7000万请求下取得6倍CPU与15倍内存效率。两名工程师借助Codex完成。"},{"id":"12b14054-da02-4786-9a17-9188942e54ce","language":"es","title":"Dos ingenieros y Codex reescribieron en Rust el servicio que sostiene cada lectura de datos de ChatGPT","subtitle":"Habitat, de OpenAI, atiende hoy 70 millones de solicitudes por segundo, y el Python que reemplazó dejó una lista detallada de cicatrices","excerpt":"La plataforma de almacenamiento de OpenAI pasó de Python a Rust en el segundo trimestre de 2026 y ganó 6 veces en eficiencia de CPU y 15 en memoria. La migración la hicieron dos ingenieros con Codex y GPT-5.5.","metaTitle":"OpenAI reescribió su servicio de almacenamiento en Rust con 2 ingenieros","metaDescription":"Habitat, de OpenAI, pasó de Python a Rust en el segundo trimestre de 2026 con 6 veces más eficiencia en CPU y 15 en memoria a 70 millones de solicitudes por segundo. Dos ingenieros lo lograron con Codex."}],"celebrities":[{"id":"b086135e-3126-46cd-b591-67f073dff1bb","nameKo":"OpenAI","nameEn":"OpenAI","slug":"openai","profileImage":"https://ui-avatars.com/api/?name=OpenAI&size=400&background=F97316&color=fff&bold=true"}]}]}],"$L20"]]}]

#infrastructure

Two Engineers and Codex Rewrote the Service Behind Every ChatGPT Data Read in Rust

Subscribe to Newsletter

#infrastructure

Two Engineers and Codex Rewrote the Service Behind Every ChatGPT Data Read in Rust