2026-10-25 — views
OpenAI Details Scaling Azure Cosmos DB for Habitat Storage Platform
Read this because A case study in balancing multi-tenancy efficiency with isolation needs at massive global scale.
OpenAI engineers explain how they optimized Azure Cosmos DB within Habitat to support over one billion users across 40 regions.
Scaling Habitat with Azure Cosmos DB
OpenAI has detailed the second part of its engineering journey to scale Habitat, its online storage platform, to serve over 1 billion ChatGPT users. The platform, which requires millisecond response times across 40 geographic regions, relies on Azure Cosmos DB as its underlying storage component. By working closely with the Azure team, OpenAI developed a viable multi-tenancy approach using co-managed containers to achieve excellent read performance at massive scale.
The Multi-Tenancy Tradeoff
As infrastructure scales, teams often transition from single-tenant systems, which offer isolation but require expensive over-provisioning, to multi-tenant systems managed by centralized database teams. While multi-tenancy improves resource efficiency, it introduces challenges such as noisy neighbors, where one tenant’s heavy load degrades performance for others. Habitat’s journey involved a circular evolution: starting with a single multi-tenant container, moving to single-tenant backing containers for isolation, and eventually reverting to a multi-tenant default to balance efficiency and reliability.
Addressing Partition Limitations
Initially, Habitat relied on Azure Cosmos DB primitives, allowing any client load through while depending on the stated 10,000 RU/s limit per logical partition key. This approach faced three main issues. First, logical partition key collisions occurred when different product use cases shared keys like user_id or workspace_id, constraining throughput. Second, noisy neighbors emerged because logical partitions share physical partition limits, allowing niche workloads to disrupt core flows. Third, weak partitions resulted from physical partition splits; as data volume grew, throughput capacity per partition shrank, reducing performance for storage-heavy use cases.
Implementing Workload Isolation
To mitigate these issues, OpenAI introduced workload isolation by using separate Azure Cosmos DB containers for critical new use cases. This allowed each container to have independent physical partitions, reducing noisy neighbor effects. Engineers also implemented scripts to synchronize total provisioned throughput with the number of physical partitions, preventing capacity shrinkage during splits. However, legacy critical use cases remained stuck in the shared default container, leading to increased pressure and production incidents. Consequently, OpenAI began provisioning separate containers for any new service or major feature to protect the default container from excessive load.
Practitioner note
Developers using Azure Cosmos DB at scale should monitor physical partition splits closely, as they can silently reduce effective throughput per partition. Implementing strict workload isolation through separate containers for high-traffic or critical services can prevent noisy neighbor issues and ensure reliable performance.