Fix Docker Container Exited with Code 137 OOM Error

Symptoms

When a Docker container exits with code 137, it typically indicates that the container was killed by the kernel's Out of Memory (OOM) killer. Common symptoms include:

Container stops unexpectedly with exit code 137
Logs may show Killed or OOMKilled in docker inspect
Application inside container becomes unresponsive before exit
Host system may show high memory usage

Root Causes

Exit code 137 is the result of the Linux kernel's OOM killer terminating a process (usually the container's main process) when the system or container runs out of memory. This can happen due to:

Insufficient memory allocated to the container via --memory flag
Memory leak in the application running inside the container
Host system memory exhaustion
Incorrect memory swap settings

Step-by-Step Fix

1. Confirm OOM Kill

Run docker inspect <container_name> | grep -i oom to check if OOMKilled is true. Also check host logs: sudo dmesg | grep -i oom for kernel messages.

2. Check Current Memory Usage

Use docker stats --no-stream to see real-time memory usage of running containers. For the failed container, review logs: docker logs <container_name>.

3. Increase Container Memory Limit

If the container has a memory limit, increase it. For example, if previously run with --memory=512m, try --memory=1g. For unlimited memory, omit the flag (not recommended for production).

docker run -d --memory=1g --memory-swap=1g my_image

Note: --memory-swap should equal --memory to disable swap usage.

4. Adjust Docker Compose (if applicable)

In docker-compose.yml, add:

services:
  my_service:
    deploy:
      resources:
        limits:
          memory: 1g
        reservations:
          memory: 512m

5. Restart the Container

Remove the old container and run a new one with updated memory settings.

Alternative Fixes

Optimize application memory usage: Profile the application to reduce memory footprint or fix memory leaks.
Add swap space: Increase host swap to prevent OOM kills, though this may degrade performance.
Use memory cgroup: Enable cgroup v2 on the host for better memory accounting.
Set memory reservation: Use --memory-reservation to give the container a soft limit.

Prevention

Monitor container memory with tools like docker stats, Prometheus, or Grafana.
Set appropriate memory limits based on application profiling.
Implement health checks and auto-restart policies (--restart=always or unless-stopped).
Use orchestration tools (Kubernetes, Docker Swarm) to manage resource limits across nodes.
Regularly update Docker and host kernel to benefit from OOM handling improvements.

By following these steps, you can resolve exit code 137 and ensure your Docker containers run reliably.