cgroup OOM Killed Our API. The Node Had 12GB Free. Wrong Process Died.

📰 Medium · DevOps

Learn how cgroup OOM killers can terminate the wrong process even with ample node free memory and how to diagnose and fix the issue

intermediate Published 10 Jul 2026

Action Steps

Investigate cgroup v2 memory settings to understand OOM killer behavior
Diagnose issues by monitoring cgroup pressure and memory usage instead of node free memory
Apply realistic limits to cgroup configurations to prevent unnecessary OOM kills
Use Vertical Pod Autoscaler to dynamically set limits and prevent resource overallocation
Monitor Kubernetes events to distinguish between OOMKill and Evicted events

Who Needs to Know This

DevOps and software engineering teams can benefit from understanding cgroup memory management to prevent unexpected process terminations and optimize resource allocation

Key Insight

💡 cgroup OOM killers can be triggered even when the node has ample free memory, and diagnosing the issue requires monitoring cgroup pressure and memory usage

Key Takeaways

Learn how cgroup OOM killers can terminate the wrong process even with ample node free memory and how to diagnose and fix the issue

Full Article

Title: cgroup OOM Killed Our API. The Node Had 12GB Free. Wrong Process Died.

URL Source: https://blog.stackademic.com/cgroup-oom-killed-our-api-the-node-had-12gb-free-wrong-process-died-a0dd4ccee368?source=rss------devops-5

Published Time: 2026-07-10T01:23:38Z

Markdown Content:
[Sitemap](https://blog.stackademic.com/sitemap/sitemap.xml)

[Open in app](https://play.google.com/store/apps/details?id=com.medium.reader&referrer=utm_source%3DmobileNavBar&source=post_page---top_nav_layout_nav-----------------------------------------)

Sign up

[Sign in](https://blog.stackademic.com/m/signin?operation=login&redirect=https%3A%2F%2Fmedium.com%2F%40SoftwareNews%2Fcgroup-oom-killed-our-api-the-node-had-12gb-free-wrong-process-died-a0dd4ccee368&source=post_page---top_nav_layout_nav-----------------------global_nav------------------)

[](https://blog.stackademic.com/?source=post_page---top_nav_layout_nav-----------------------------------------)

Get app

[Write](https://blog.stackademic.com/m/signin?operation=register&redirect=https%3A%2F%2Fmedium.com%2Fnew-story&source=---top_nav_layout_nav-----------------------new_post_topnav------------------)

[Search](https://blog.stackademic.com/search?source=post_page---top_nav_layout_nav-----------------------------------------)

Sign up

[Sign in](https://blog.stackademic.com/m/signin?operation=login&redirect=https%3A%2F%2Fmedium.com%2F%40SoftwareNews%2Fcgroup-oom-killed-our-api-the-node-had-12gb-free-wrong-process-died-a0dd4ccee368&source=post_page---top_nav_layout_nav-----------------------global_nav------------------)

![Image 1: Unknown user](https://miro.medium.com/v2/resize:fill:32:32/1*dmbNkD5D-u45r44go_cf0g.png)

1. [cgroup v2 memory — two OOM killers](https://blog.stackademic.com/?source=post_page-----a0dd4ccee368---------------------------------------#688c "cgroup v2 memory — two OOM killers")
2. [Why the wrong process dies](https://blog.stackademic.com/?source=post_page-----a0dd4ccee368---------------------------------------#edf5 "Why the wrong process dies")
3. [Diagnosis — stop looking at node free memory](https://blog.stackademic.com/?source=post_page-----a0dd4ccee368---------------------------------------#aa6a "Diagnosis — stop looking at node free memory")
4. [The fix — limits that match reality](https://blog.stackademic.com/?source=post_page-----a0dd4ccee368---------------------------------------#e9bb "The fix — limits that match reality")
5. [cgroup v1 vs v2 — the migration footgun](https://blog.stackademic.com/?source=post_page-----a0dd4ccee368---------------------------------------#f086 "cgroup v1 vs v2 — the migration footgun")
6. [Java container limits — the heap trap everyone sets wrong](https://blog.stackademic.com/?source=post_page-----a0dd4ccee368---------------------------------------#f2eb "Java container limits — the heap trap everyone sets wrong")
7. [Vertical Pod Autoscaler — how we set limits now](https://blog.stackademic.com/?source=post_page-----a0dd4ccee368---------------------------------------#3419 "Vertical Pod Autoscaler — how we set limits now")
8. [Sidecar memory — the silent consumer](https://blog.stackademic.com/?source=post_page-----a0dd4ccee368---------------------------------------#c901 "Sidecar memory — the silent consumer")
9. [OOMKill vs Evicted — Kubernetes events matter](https://blog.stackademic.com/?source=post_page-----a0dd4ccee368---------------------------------------#9f44 "OOMKill vs Evicted — Kubernetes events matter")
10. [Load testing with cgroup limits applied](https://blog.stackademic.com/?source=post_page-----a0dd4ccee368---------------------------------------#f3d3 "Load testing with cgroup limits applied")
11. [Prometheus recording rule — cgroup pressure ratio](https://blog.stackademic.com/?source=post_page-----a0dd4ccee368---------------------------------------#de56 "Prometheus recording rule — cgroup pressure ratio")
12. [requests vs limits — the scheduling lie](https://blog.stackademic.com/?source=post_pa

Read full article → ← Back to Reads

cgroup OOM Killed Our API. The Node Had 12GB Free. Wrong Process Died.

Key Takeaways

Full Article

Related Videos