Platform engineers who last audited their GPU scheduling setup more than a year ago are likely running on infrastructure that’s been quietly superseded. Between Kubernetes v1.34 graduating Dynamic Resource Allocation to GA, NVIDIA donating the k8s-dra-driver to CNCF, and KAI Scheduler reaching CNCF Sandbox status, the stack for running AI workloads on Kubernetes shifted substantially in 2025–2026. Here’s what that actually means for teams with production clusters.

Why Device Plugins Were Never the Right Answer

The legacy path for GPU scheduling on Kubernetes was the device plugin API, introduced in Kubernetes 1.10. Device plugins are functional but blunt: they expose GPUs to the scheduler as opaque integer counts. You tell Kubernetes you need nvidia.com/gpu: 1, and the scheduler finds a node with a free unit. That’s it. No awareness of GPU topology, no sharing across pods, no way to express “I need two GPUs on the same NVLink fabric” or “this workload can run on half a GPU.”

The consequence is predictable: GPU clusters run at poor utilization. Workloads that need modest GPU resources claim a whole device. Jobs that could share hardware sit in separate pods on separate GPUs. Platform teams work around this with vendor-specific tooling bolted on at the node level — NVIDIA’s time-slicing config, hand-tuned MIG partitions — none of which the Kubernetes scheduler actually understands at scheduling time.

DRA was designed to fix the structural problem, not paper over it.

What DRA Actually Changes

Dynamic Resource Allocation introduced a new set of primitives in the resource.k8s.io API group. The key objects:

  • ResourceSlice: A driver-created object advertising available devices on a node — their attributes, topology, current allocation state. The scheduler reads these to understand what hardware is actually available, at a semantic level, not just a headcount.
  • ResourceClaim: What a workload requests. Instead of annotating a container spec with a GPU count, a workload creates a ResourceClaim describing its requirements using Common Expression Language (CEL) selectors. “I need a GPU with at least 40 GB of HBM and an NVLink peer within 2 hops.”
  • DeviceClass: Cluster-level categories defined by admins. Think of them as reusable device profiles — inference-small, training-large — that teams reference in ResourceClaims rather than re-specifying low-level selectors every time.

DRA graduated to stable in Kubernetes v1.34 and is enabled by default. v1.36, released in May 2026, added prioritized device lists: a workload can declare a preference order — “give me one H100 80GB; if none available, give me two A100 40GBs.” The scheduler evaluates these fallbacks without the workload needing to know cluster topology in advance.

Practically, this matters most for inference fleets where workload size varies. A quantized 7B model inference pod doesn’t need the same slice of GPU as a 70B FP16 pod. With device plugins, both claim a whole device. With DRA’s CEL selectors and prioritized lists, the platform can match hardware to actual requirements.

KAI Scheduler: DRA’s Production Complement

DRA solves the expression problem — workloads can now describe what they need with precision. KAI Scheduler solves the allocation problem — how do you decide, across a multi-tenant cluster with competing priorities, who gets what hardware when?

KAI Scheduler (a CNCF Sandbox project, Apache 2.0, v0.16.0 as of June 2026) is NVIDIA’s open-sourced derivative of the Run:ai scheduler, rebuilt as a Kubernetes-native scheduler. It ships several capabilities that the default kube-scheduler doesn’t have:

Gang scheduling. A distributed training job — say, a PyTorch FSDP run across 8 pods — either schedules all 8 simultaneously or waits. With the default scheduler, pods schedule greedily: you end up with 6 of 8 pods running, holding their GPUs, while the other 2 wait for capacity that isn’t coming. KAI’s gang scheduling prevents this deadlock entirely.

Topology-Aware Scheduling. KAI understands GPU topology — NVLink connectivity, NUMA zones, network switch boundaries — and uses this when placing pods. A workload that needs tight inter-GPU communication gets placed on nodes where GPUs can actually talk to each other efficiently. This showed up in KAI’s KubeCon NA 2025 talk (“Mind the Topology: Smarter Scheduling for AI Workloads”) and is particularly relevant for disaggregated serving architectures where prefill and decode stages run on different GPU sets.

Hierarchical fair-share with time decay. Multi-team clusters need usage accounting that doesn’t reward GPU hoarders or punish teams that run burst jobs. KAI’s queue hierarchy lets platform teams define fair-share policies across teams and projects. Time-based fairshare (added in v0.10.0, October 2025) factors in historical consumption with a decay curve — teams that burned a lot of GPU last week get lower priority this week, recovering as usage ages out.

GPU sharing. KAI supports fractional GPU allocation natively, complementing MIG partitioning and time-slicing configurations. Multiple inference pods can share a single physical GPU, with KAI tracking and enforcing the allocation without relying on node-level config alone.

What Platform Teams Actually Need to Do

The shift to DRA + KAI isn’t a flag flip. Here’s where the real work is.

Audit your device plugin dependencies. If your cluster relies on the NVIDIA device plugin for scheduling decisions, map which workloads use it and what they’re actually requesting. Most are using it as a headcount mechanism — those are the candidates for ResourceClaim migration. Device plugins still work alongside DRA in 1.34/1.35; you’re not forced to migrate everything at once.

Start with inference workloads. Training jobs have more predictable hardware requirements and are easier to over-provision. Inference workloads are where underutilization costs the most — latency variability from time-slicing is acceptable for many inference patterns, and fractional GPU allocation via KAI or DRA can meaningfully reduce per-request GPU cost. A cluster at 40% GPU utilization is spending more than half its GPU budget on idle hardware.

Define DeviceClasses before workloads start using ResourceClaims. If you let every team write their own CEL selectors from scratch, you’ll end up with 40 variations of “give me an H100.” Define a small set of cluster-level DeviceClasses (training-fp16-large, inference-a10g-small, etc.) that encode your hardware catalog. Workloads reference classes; admins manage the mapping between class and actual device attributes.

The KAI Helm install is straightforward; the queue hierarchy is not. KAI deploys via Helm into the kai-scheduler namespace. Getting it running takes under an hour. Getting the fairshare queue hierarchy right for a multi-team org takes real organizational work: what are your teams, what are their baseline allocations, what burst limits apply. Treat queue design as a platform product decision, not an afterthought.

The NVIDIA DRA Driver Donation and What It Signals

At KubeCon EU 2026 in Amsterdam, NVIDIA donated its k8s-dra-driver to CNCF. This is worth noting beyond the headline. The driver is the implementation layer that publishes NVIDIA GPU ResourceSlices to the Kubernetes API — it’s what makes NVIDIA hardware actually visible to DRA-aware schedulers. Moving it to CNCF governance means it’s no longer a proprietary NVIDIA artifact; it becomes infrastructure that the broader community can contribute to, audit, and depend on without a single-vendor trust dependency.

Combined with Google Cloud and AWS EKS both committing to DRA support, this signals that DRA is the cross-cloud standard for device scheduling going forward. Teams designing new GPU infrastructure should build around DRA, not device plugins.

Platform teams who get ahead of this migration — defining clean DeviceClasses, standing up KAI, migrating inference workloads off device-plugin-style scheduling — will have clusters that are measurably cheaper to run and more predictable to operate. Teams that wait are accumulating technical debt on infrastructure that the ecosystem is actively moving away from.