When a Linux server runs out of memory, the kernel's "OOM killer" picks a process using lots of memory and kills it so the rest of the system can keep running. That process is often your app or your database, so the site goes down, sometimes with no error at all. The system log records the kill. Then find what used the memory: a leak in the app, too many workers, a big job, or a server that's simply too small.
What's the problem
Your app or database stops suddenly, logs end without an error, or the system log shows the kernel killed a process for lack of memory. It may happen every few hours or days.
Why it happens
- A memory leak: the app keeps using more memory over time until it's killed.
- Too many workers or processes for the RAM available, such as too many PHP or Node workers.
- One large job: an import, an export, image processing, or a huge query result loaded all at once.
- The database and app share a small server and both grow.
- No swap space, so there's no buffer when memory runs short.
- A container memory limit set lower than the app needs.
How to fix it
- Confirm it was an out-of-memory kill. Run
dmesg -T | grep -i "killed process"or checkjournalctl -kfor the time it stopped. - See memory use now with
free -h, and which processes use the most withtop(pressMto sort by memory). - If memory grows steadily, the app has a leak. Restarting buys time; the fix is in the code.
- Reduce worker counts so they fit in RAM, or process big jobs in smaller batches.
- Add swap as a safety buffer on small servers.
- Give the database its own server, or upgrade the plan, once usage is normal and still too high.
What Preventionlabs checks
One click runs the full report on your domain. For this problem it shows whether your site loads, how fast, and where its redirects go; uptime checked every 5 minutes, with an email when the site goes down and when it is back. With Cloudflare connected, it also shows which pages returned server errors (5xx) to visitors, and when; recent deploys and account changes, with their times. It can't log in to your server, so memory, disk and process problems show up in your host's dashboard and logs.
Run your free reportYour first report is free, no card needed. Then A$50 a month, up to 10 domains. Read-only: it never changes anything.
Sources
- The Linux Kernel documentation: Documentation for /proc/sys/vm/official docs
If this is set to zero, the OOM killer will scan through the entire tasklist and select a task based on heuristics to kill. This normally selects a rogue memory-hogging task that frees up a large amount of memory when killed.
- procps (man7.org): free(1)manual page
free displays the total amount of free and used physical and swap memory in the system, as well as the buffers and caches used by the kernel.