Fix “Argument list too long” on Linux

Cause 1: ARG_MAX exceeded by a literal argument list

Every program you run gets its arguments and environment passed into a fixed-size kernel buffer. That cap is ARG_MAX, and on any modern 64-bit Linux it's usually 2 MB. You don't hit it with three filenames. You hit it when you do something like rm *.log in a directory holding 800,000 files, or pass a 3 MB list of paths to tar. The kernel returns E2BIG and the shell prints Argument list too long.

Real trigger I've seen more than once: a customer’s app writes one log per request into /var/log/myapp/. Three weeks later, cron tries rm /var/log/myapp/* and fails. Nobody notices until the disk fills.

Check your limit first so you know what you're working with:

getconf ARG_MAX
# typically 2097152 (2 MB)

# the real usable size is smaller once env vars are counted
xargs --show-limits </dev/null

The fix is always the same: stop handing one giant list to a single exec. Let find do the walking and the deleting.

# safest, fastest delete of a huge flat dir
find /var/log/myapp -maxdepth 1 -type f -name '*.log' -delete

# if your find lacks -delete (busybox, old systems)
find /var/log/myapp -maxdepth 1 -type f -name '*.log' -print0 \
  | xargs -0 -n 200 rm -f

If you're copying or moving instead of deleting, use -exec ... + which batches arguments up to the limit automatically:

find /old -type f -name '*.csv' -exec cp {} /new/ \;
# \; runs cp once per file — slow but bulletproof

find /old -type f -name '*.csv' -exec cp -t /new/ {} +
# + batches args, much faster than \;

Don't bother raising ARG_MAX. On Linux it's not tunable through sysctl, it's derived from the stack rlimit, and bumping ulimit -s to move it is a hack that breaks other things. Fix the invocation, not the limit.

Cause 2: Environment block bloat eats into ARG_MAX

Here's the one that gets people. The kernel limit is shared between arguments and the environment. If a process is started with a fat env, your usable argument space shrinks. I've watched someone spend an hour chasing an E2BIG on a command that worked fine in their own shell — the difference was a Docker container with ~1.5 MB of env vars injected by a badly configured orchestrator.

Prove it quickly:

# compare env size in both contexts
env | wc -c

# inside the failing container or service
podman exec mycontainer env | wc -c

If the second number is in the hundreds of KB, you found your culprit. The offending env is usually a single monster variable — a JSON blob, a base64 cert, a huge PATH from a botched source chain.

Fixes, in order of preference:

1. Move the big value out of the env and into a file the process reads at startup. Config belongs in config files, not env vars.
2. If it must be env, trim it. A 400 KB LS_COLORS or a duplicated PATH entry list is a bug, not a feature.
3. Run the heavy command with env -i and pass only what it needs:

env -i PATH=/usr/bin:/bin HOME=/root find /data -type f -delete

The env -i trick alone has saved me on more than one box where cron inherited a polluted environment from a parent shell.

Cause 3: Shell glob expansion before exec

This one is subtle because the program never sees the long list — the shell builds it first, then fails to exec. So rm *.txt in a huge dir fails inside bash, not inside rm. Same error, different layer.

You can spot it: the program's own error handling never fires, and strace shows no execve at all. The shell is the one printing the message.

Skip globs for bulk operations. Use find, or if you really want glob semantics, enable nullglob and lean on xargs:

# bash: don't do this on a huge dir
# rm /data/*.tmp

# do this instead
find /data -maxdepth 1 -type f -name '*.tmp' -delete

For non-delete workloads, xargs with a sane batch size is the workhorse. -n controls args per invocation, -P parallelizes, -0 handles filenames with spaces and newlines (which you should always assume exist):

find /data -type f -name '*.log' -print0 \
  | xargs -0 -n 100 -P 4 gzip

A note on -P: it's fine for gzip, md5sum, stat-style work. It is not fine for anything writing to a shared file or a database without transaction handling. I've seen people parallelize tee into one output file and wonder why it's corrupted. Don't.

Quick reference

Nine times out of ten, replacing a glob with find -delete ends the problem. The tenth time, check your environment size before you start blaming the kernel.