8 min read
Kubernetes CPU limits: 4x slower pods and OOMKills without a leak
I ran the same app with and without a CPU limit. The limit made it about 4x slower while average CPU looked fine, and in three memory tests only the capped pod was OOMKilled.
A CPU limit made the same app about 4x slower, and in three separate tests it got pods OOMKilled with no memory leak. I ran identical pods side by side on a local Kubernetes cluster, where the only difference in the YAML was limits.cpu. With a 500m limit, typical latency went from 23 ms to 87 ms, and the pod was throttled in half of all CPU windows while its average CPU stayed under the limit. In each of the three memory tests, the capped pod was OOMKilled and the uncapped pod never restarted.
limits.cpu (500m for the first three rows, 100m for the last two).Everything here, from the test app to the raw numbers, is open source: k8s-cpu-limits-analyzed on GitHub. This post is the short version.
What this means for your team
For almost every service, the fix is one deleted line. Keep the CPU request, because it reserves your share of the node. Keep a memory limit equal to the memory request, because a memory leak should not take the node down. Drop the CPU limit:
resources:
requests:
cpu: 250m
memory: 512Mi
limits:
memory: 512Mi
# no cpu limit
This is not a new idea. Tim Hockin, a Kubernetes co-founder, put it as "Always set memory limit == request. Never set CPU limit", and Google's GKE docs recommend an unbounded CPU limit. Most charts still ship one, usually as the request plus some headroom. What the tests add is evidence of what that headroom costs.
Four practical consequences:
-
Your dashboards can't see it. The usual CPU graph averages a minute. Throttling happens inside 100 ms windows. Watch the share of throttled windows per container instead:
rate(container_cpu_cfs_throttled_periods_total[5m]) / rate(container_cpu_cfs_periods_total[5m]) -
A memory "leak" can be a CPU limit. In the tests below, the OOMKilled pods were throttled in 96% and 100% of CPU windows, and the third was too starved to report. Check throttling before you raise
limits.memory. -
A limit does not protect your neighbors. Their own requests do. A busy pod without a limit only uses CPU that would otherwise sit idle, and gives it back when others need it.
-
Right-size after you drop the limit, not before. Usage recorded under a cap can never exceed the cap, so it tells you what the kernel allowed, not what the app wanted. The savings come from right-sized requests, not from deleting the limit line.
Before you roll it out, pin your runtime's CPU count (DOTNET_PROCESSOR_COUNT, GOMAXPROCS, or the JVM's -XX:ActiveProcessorCount), because these runtimes often size thread pools from the limit. Check that no LimitRange silently injects a CPU limit, and that kubelet has --system-reserved and --kube-reserved set so node daemons keep their share. The repo has a staged rollout plan and the PromQL to measure your own cluster first.
The exceptions are narrow: benchmarks that need a hard, repeatable ceiling, pods that pin whole cores on purpose, and multi-tenant platforms such as cloud providers. If you run one of those, you already know it.
Why a limit hurts even when you are under it
A request is a weight. When the node is busy, the Linux scheduler (CFS) splits CPU time between pods in proportion to their requests. When the node is quiet, a pod can use the spare cores. No limit is needed for any of that.
A limit is a separate budget per 100 ms window, shared by every thread in the pod. A 500m limit is 50 ms of CPU time per window. Four threads working at once burn that in about 12 ms, and then the pod waits for the rest of the window, even if the node is idle. Averaged over a minute, the pod looks comfortably under its limit.
Test 1: latency under a burst
Both pods requested 250m, and .NET was pinned to 4 CPUs on both, so the test compares the limit and nothing else. The load is a short burst: four threads for 20 ms, five times a second. That averages about 400m, under the 500m limit, but it uses the whole budget inside each window. This is deliberate. It is the pattern that throttles a pod while its CPU graph stays green.
Typical latency was 3.8 times higher with the limit, and the slowest 1% of requests went from 44 ms to 102 ms. There were no errors and no timeouts. Nothing in a normal dashboard would flag it.
A second test sent a short spike of 40 requests per second that did go over the cap. Typical latency barely moved there, because most of each request was a 30 ms sleep, but the slowest requests rose from 59 ms to 88 ms and the capped pod was throttled almost all the time.
I also ran a pod next to the unlimited app that just burns CPU in four tight loops, with no limit of its own. The unlimited app did not get slower: its slowest 1% was 52.9 ms alone and 49.2 ms with the busy neighbor. Its own request was enough.
Test 2: three ways a CPU limit ends in OOMKilled
CPU is the resource where you wait, not die. But wherever work arrives faster than a throttled pod can finish it, the difference sits in memory until the kernel kills the pod. The lab reproduces three common shapes. Nothing leaks in any of them.
The backlog. This is any consumer that buffers messages from Kafka or RabbitMQ, or any in-memory queue between two components. Jobs arrive at 20 per second, which needs about 800m of CPU against a 500m cap.
The capped pod was throttled in 96% of CPU windows before it died. A bounded worker pool does not prevent this, since it bounds concurrency, not the queue in front of it. Real backpressure only moves the deficit to the broker as lag.
GC starvation. No queue at all. Each request builds a graph of 10,000 small objects, holds it while working, then drops it. With a 100m cap, in-flight requests pile up, each garbage collection gets more expensive, and the collector competes with the workload for the same small budget. The capped pod became too throttled to answer its own stats endpoint, kept working through its backlog after the load had stopped, and was OOMKilled anyway. The uncapped pod finished with one request in flight, a 10.7 MiB managed heap and 0.16% of its time in GC pauses.
A plain API server. No queue, no buffer, no background worker. Each request allocates 8 MiB, awaits a 200 ms downstream call, does 40 ms of CPU work, frees everything and returns. Memory held is requests in flight times memory per request, and a CPU limit makes every request slower, so more of them are in flight at once. Under a 100m cap, the pod was throttled 100% of the time and OOMKilled after 24 seconds of load. The uncapped pod peaked at 11 requests in flight, held 40 MiB for them, and never restarted.
All three look the same on a graph: memory climbing into the limit. It looks like a leak, the usual fix is a bigger memory limit, and the actual cause is the CPU limit.
How I tested
The test app is a small .NET 10 service with endpoints for each workload, driven by an external load generator. Every comparison is an A/B pair of pods with the same app, the same request, the same memory limit and the same load. The latency tests ran on a local 8-CPU minikube on 16 August 2026. The OOMKilled tests ran on 18 and 19 August on local clusters (Rancher Desktop and minikube). The scripts, manifests, raw JSONL and results are in the repo, and each test runs with one command on a throwaway cluster.
Caveats
- Small, local runs. The latency numbers are one pass on one machine, not a production fleet. The burst test was designed to hit the blind spot. Your size of effect depends on how bursty your services are.
- The neighbor test had spare cores. The node had 8 CPUs and one busy neighbor, so nothing really had to compete. The run script now also fills the node completely, but that result is not in the recorded data.
- Seconds to death are specific to this setup. The pods compile the app at startup inside the .NET SDK image, which adds memory and CPU overhead. Which pod dies did not depend on it. How fast it died does.
- Big requests die fast. The API server dies in seconds because each request holds 8 MiB, like an export or image endpoint. A small JSON API under the same cap fails slower: rising latency, timeouts and probes going dark instead of a kill. Allocating after the
awaitinstead of before it also lets memory plateau. - Requests still matter. Burst above your request is a bonus, not a guarantee. Size the request for normal load. A limit only takes away the best case, even on an idle node.