fw-ai/KAI-Scheduler

fw-ai/KAI-Scheduler

Description: KAI Scheduler is an open source Kubernetes Native scheduler for AI workloads at large scale

License: Apache-2.0

Stars: 0

Forks: 0

Open issues: 1

Created: 2026-09-22T22:46:48Z

Pushed: 2026-09-22T22:49:33Z

Default branch: main

Fork: yes

Parent repository: kai-scheduler/KAI-Scheduler

Archived: no

README:  

KAI Scheduler is a robust, efficient, and scalable Kubernetes scheduler that optimizes GPU resource allocation for AI and machine learning workloads.

Designed to manage large-scale GPU clusters, including thousands of nodes, and high-throughput of workloads, makes the KAI Scheduler ideal for extensive and demanding environments. KAI Scheduler allows administrators of Kubernetes clusters to dynamically allocate GPU resources to workloads.

KAI Scheduler supports the entire AI lifecycle, from small, interactive jobs that require minimal resources to large training and inference, all within the same cluster. It ensures optimal resource allocation while maintaining resource fairness between the different consumers. It can run alongside other schedulers installed on the cluster.

Latest News 🔥

• [2026/04] KubeCon EU 2026 Talk: Watch the recording of the presentation "GPU Reservations: Maximizing Utilization and Fairness Across Teams", to explore how KAI Scheduler manages GPU resource reservations to balance utilization and fairness across teams.
• [2025/11] KubeCon NA 2025 Talk: Watch the recording of the presentation "Lightning Talk: Mind the Topology: Smarter Scheduling for AI Workloads on Kubernetes" to learn how KAI's Topology-Aware Scheduling (TAS) optimizes placement for modern disaggregated serving architectures.
• [2025/11] Integration with [Grove](https://github.com/ai-dynamo/grove) & Dynamo: KAI's Topology-Aware and Hierarchical Gang Scheduling capabilities are integrated with Grove to orchestrate complex, multi-component workloads like disaggregated serving and agentic pipelines at scale. Read the blog post for more details.
• [2025/10] [v0.10.0 Release:](https://github.com/kai-scheduler/KAI-scheduler/releases/tag/v0.10.0) Major features released, including Topology-Aware Scheduling (TAS), Hierarchical PodGroups, and Time-based Fairshare.
• [2025/10] KubeRay Integration: KAI Scheduler is now natively integrated for Ray workloads on Kubernetes.
• [2025/08] Time-Based Fairshare: Proposal for Time-based Fairshare is discussed at batch-wg. Watch the recording.
• [2025/04] Project Introduction: Recording of the KAI Scheduler introduction presented at the batch-wg meeting.

Key Features

• [Batch Scheduling](docs/batch/README.md): Ensure all pods in a group are scheduled simultaneously or not at all.
• Bin Packing & Spread Scheduling: Optimize node usage either by minimizing fragmentation (bin-packing) or increasing resiliency and load balancing (spread scheduling).
• [Workload Priority](docs/priority/README.md): Prioritize workloads effectively within queues.
• Separation of workload priority and preemptibility: supports separation of workload priority and workloads preemptibility as two independent policies
• [Hierarchical Queues](docs/queues/README.md): Apply quotas, limits, priorities, and fairness policies across multi-level queue hierarchies for flexible organizational control.
• [Resource distribution](docs/fairness/README.md#resource-division-algorithm): Customize quotas, over-quota weights, limits, and priorities per queue.
• [Fairness Policies](docs/fairness/README.md#reclaim-strategies): Ensure equitable resource distribution using Dominant Resource Fairness (DRF) and resource reclamation across queues.
• Time-based Fairshare: Over-time fair usage of resources, considering historical usage, time decay, and other parameters for fine-tuning.
• Min-guaranteed-runtime: ensures a time period in which the scheduler must not preempt or reclaim a running workload, even if preemptible.
• Workload Consolidation: Reallocate running workloads intelligently to reduce fragmentation and increase cluster utilization.
• [Elastic Workloads](docs/elastic/README.md): Dynamically scale workloads within defined minimum and maximum pod or SubGroup thresholds.
• Dynamic Resource Allocation (DRA): Support vendor-specific hardware resources through Kubernetes ResourceClaims (e.g., GPUs from NVIDIA or AMD).

-...