Shell Scripting: Expert Recipes for Linux, Bash, and More

xargs

When a command is called with many arguments (which are often filenames), the kernel has to put that data somewhere for the command to reference it. Sometimes (often when using the find command to trawl through large filesystems), this list is longer than the kernel will allow. This results in messages such as “rm: Argument list too long.” Many people are quite used to this and know the workaround is to use xargs, but not everybody understands quite how it works, which can have some significant implications.

$ rm 'find . -name "*.core*" -print'
bash: /bin/rm: Argument list too long
$ find . -name "*.core*" -print0 | xargs -0 rm
$

To get around the problem, xargs will read its standard input (which has no such restrictions) and pass those on in manageable chunks to the command it has been told to pass them to. This simple listfiles script shows the effect of this on the command being run. Here, the -L 3 option tells xargs to break the input down into three files per call. This is not how everybody thinks that xargs works, so the result may be surprising.

$ cat listfiles
#!/bin/bash
echo "Listfiles (PID $$) was called with $# arguments:"
i=1
while [ -a "$1" ]
do
  echo "${i}: $1"
  ((i++))
  shift
done

listfiles

$ find . -print
.
./etc
./etc/hosts
./etc/config.txt
./bin
./bin/ls
./sh
./listfiles
$ find . -print | xargs -L 3 ./listfiles Listfiles (PID 17088) was called ...