2026-10-10 — views
Ai2 Replaces Priority Scheduling with GPU Time Budgets to Solve Tragedy of the Commons
Read this because This case study demonstrates how economic incentives like time budgets can solve technical scheduling problems like resource squatting and priority inflation in shared GPU clusters.
Ai2 shifts from a flawed priority-based scheduler to a hierarchical fair-share system with GPU time budgets, improving cluster occupancy and research impact.
The Scheduling Pyramid and Legacy Issues
The AI Infrastructure team at Ai2 manages thousands of NVIDIA H100, B200, and B300 GPUs across clusters ranging from 88 to 1024 units. These resources serve approximately 150 internal researchers working on large-scale distributed training, robotics reinforcement learning, and scientific agentic use cases. The team views cluster management through a pyramid of metrics: availability, occupancy, impact, and utilization. Historically, they relied on a priority-based scheduler that allowed workloads to opt out of preemption. This approach led to predictable pathologies, including GPU squatting, where users parked no-op workloads to secure resources for future debugging. Additionally, priority inflation occurred as 100% of workloads adopted HIGH priority, starving lower-priority tasks. Engineers also spent significant time negotiating the shutdown of non-preemptable workloads during maintenance.
From Monopolies to Time Budgets
The legacy system created a tragedy of the commons, where individual users maximized their own outcomes at the expense of global performance. Initial attempts to fix this involved assigning GPU monopolies to important projects, but this caused idle time due to the seasonality of research needs. To solve this, Ai2 replaced the priority system with a model based on GPU time budgets and hierarchical fair-share allocation. Instead of assigning physical hardware, the system allocates a portion of GPU time. This shifts the debate from case-by-case operational tasks to a transparent administrative budgeting process. Managers allocate GPU time to projects and researchers based on strategic impact, allowing leadership to think like investors. Every request for GPU time must now be funded by a budget; otherwise, it is not protected from preemption. This makes gaming the scheduler more expensive than honestly advocating for a larger budget.
Hierarchical Fair-Share Implementation
Paired with the budgeting tool, Ai2 implemented a hierarchical fair-share scheduler. This system mirrors the research program structure, with weights determined by budgets set by managers rather than static quotas. The scheduler tracks occupancy over a sliding lookback window, defaulting to seven days. It sorts workloads from under-utilized allocations above those from over-utilized allocations, ensuring that every group receives its allocated GPU time over the week-long range, provided they actively submit workloads. This approach allows teams to burst beyond their allocation limits if they have unused capacity, reclaiming compute that would otherwise be lost. The new system has reportedly made the cluster feel like it has an extra 30% compute capacity by efficiently handling bursty workloads.
Practitioner note
Developers managing shared compute resources should consider implementing time-based budgets rather than static hardware allocations to align individual incentives with organizational goals. Watch for priority inflation and resource squatting in your current systems, as these are common indicators that a fair-share model with economic constraints is needed.