2026-09-12 — views
OpenAI Scales Habitat Storage to Handle 70M Requests Per Second
Read this because A case study in managing exponential growth by accepting short-term technical debt in Python to achieve rapid platform stability and decouple storage logic.
OpenAI engineers detail how they scaled their Python-based Habitat storage platform to serve over one billion users with unprecedented growth.
Unprecedented Growth and Scale
OpenAI has successfully scaled its online storage platform, Habitat, to support over one billion weekly active users across nearly 40 geographic regions. The system now processes more than 70 million requests every second and manages over 500 petabytes of data. This infrastructure is critical for all OpenAI products, including ChatGPT and Codex, ensuring fast and reliable data access for actions ranging from login verification to conversation initiation. If these requests fail or slow down, the user experience degrades significantly or the product becomes unusable.
Evolution from Library to Service
Habitat launched in 2023 as a simple Python client-side library to support GPTs. It was designed to abstract database management details, such as schema lookup, routing, authorization, and encryption, allowing product engineers to focus on application logic rather than underlying storage mechanics. However, by mid-2025, the client-side implementation reached its limits. As the number of OpenAI services increased, maintaining backward compatibility became infeasible. Coordinating deployments across dozens of services to update the client library proved brittle and inefficient. A single rollback by one team could cause outages that the engineering team had worked hard to prevent.
To resolve this operational fragility, OpenAI decoupled the storage logic into a standalone service. This centralization established a single point of control for deployments, observability, and platform enhancements. It also allowed for stricter enforcement of data security and privacy primitives, including access control policies and audit logging. The service acts as a chokepoint to protect user data from unauthorized access by external, internal, or agent actors.
Strategic Use of Python
Despite Python’s known overhead in high-throughput services, OpenAI chose to build Habitat as a Python service. This decision introduced higher network latency and increased CPU and memory scaling costs compared to local library execution. The engineering team acknowledged that Python would not be sustainable at 100x scale and that a future rewrite would likely be necessary. However, they viewed this as a strategic incursion of technical debt. The primary objective was to unblock product developers and achieve immediate platform stability. They calculated that the rapid advancement of their own coding models, such as Codex and GPT, would eventually simplify the path to migrating away from Python. This bet proved correct, allowing the team to prioritize stability over immediate resource optimization while building a robust infrastructure.
Practitioner note
Engineers should consider decoupling client-side logic into standalone services to reduce deployment fragility and centralize security controls. Accepting short-term technical debt in language choice can be a viable strategy if it accelerates platform stability and product velocity, provided a migration path is anticipated.