Make Your Kubernetes Operator Handle 10× More Users

Kubesimplify · Intermediate ·☁️ DevOps & Cloud ·3mo ago

About this lesson

Your Kubernetes operator reconciles one object at a time — by default. With a single worker, 100 custom resources and a 5-second reconcile means customer #100 waits over 8 minutes just to get started. In this video, Shubham shows why this happens, how the controller-runtime work queue actually processes objects, and the one-line change in your SetupWithManager that turns a sequential controller into a parallel one using MaxConcurrentReconciles. You'll see it live on an EC2Instance CRD: 10 objects processed serially with 1 worker, then the same 10 objects processed in parallel with 10 workers — and the tradeoffs of tuning that number for your workload. This video is NOT about making a single reconciliation faster — it's about making sure your users don't wait in line behind other users' objects. ━━━━━━━━━━━━━━━━━━━━━━━ ⏱ Chapters ━━━━━━━━━━━━━━━━━━━━━━━ 0:00 The problem: customers waiting in a reconcile queue 1:09 What "faster" really means for an operator 2:04 Recap: the EC2Instance custom resource 4:20 Why the problem only shows up at scale 6:35 Inside the work queue — one worker, one object 8:03 Counter analogy: why a single worker bottlenecks 9:00 The fix: tell the controller to use more workers 10:26 Walkthrough of the reconciler code 17:29 Demo with 1 worker (sequential, 5s per object) 21:28 Scaling the simulated task to 10 seconds 23:46 Setting MaxConcurrentReconciles in S

Original Description

Your Kubernetes operator reconciles one object at a time — by default. With a single worker, 100 custom resources and a 5-second reconcile means customer #100 waits over 8 minutes just to get started. In this video, Shubham shows why this happens, how the controller-runtime work queue actually processes objects, and the one-line change in your SetupWithManager that turns a sequential controller into a parallel one using MaxConcurrentReconciles. You'll see it live on an EC2Instance CRD: 10 objects processed serially with 1 worker, then the same 10 objects processed in parallel with 10 workers — and the tradeoffs of tuning that number for your workload. This video is NOT about making a single reconciliation faster — it's about making sure your users don't wait in line behind other users' objects. ━━━━━━━━━━━━━━━━━━━━━━━ ⏱ Chapters ━━━━━━━━━━━━━━━━━━━━━━━ 0:00 The problem: customers waiting in a reconcile queue 1:09 What "faster" really means for an operator 2:04 Recap: the EC2Instance custom resource 4:20 Why the problem only shows up at scale 6:35 Inside the work queue — one worker, one object 8:03 Counter analogy: why a single worker bottlenecks 9:00 The fix: tell the controller to use more workers 10:26 Walkthrough of the reconciler code 17:29 Demo with 1 worker (sequential, 5s per object) 21:28 Scaling the simulated task to 10 seconds 23:46 Setting MaxConcurrentReconciles in S
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Related Reads

Chapters (11)

The problem: customers waiting in a reconcile queue
1:09 What "faster" really means for an operator
2:04 Recap: the EC2Instance custom resource
4:20 Why the problem only shows up at scale
6:35 Inside the work queue — one worker, one object
8:03 Counter analogy: why a single worker bottlenecks
9:00 The fix: tell the controller to use more workers
10:26 Walkthrough of the reconciler code
17:29 Demo with 1 worker (sequential, 5s per object)
21:28 Scaling the simulated task to 10 seconds
23:46 Setting MaxConcurrentReconciles in S
Up next
How to Code with Distrobox on the Steam Deck
Ian Wootten
Watch →