Skip to Content
Shell Scripting: Expert Recipes for Linux, Bash, and More
book

Shell Scripting: Expert Recipes for Linux, Bash, and More

by Steve Parker
August 2011
Beginner to intermediate
600 pages
14h 29m
English
Wrox
Content preview from Shell Scripting: Expert Recipes for Linux, Bash, and More

xargs

When a command is called with many arguments (which are often filenames), the kernel has to put that data somewhere for the command to reference it. Sometimes (often when using the find command to trawl through large filesystems), this list is longer than the kernel will allow. This results in messages such as “rm: Argument list too long.” Many people are quite used to this and know the workaround is to use xargs, but not everybody understands quite how it works, which can have some significant implications.

$ rm 'find . -name "*.core*" -print'
bash: /bin/rm: Argument list too long
$ find . -name "*.core*" -print0 | xargs -0 rm
$

To get around the problem, xargs will read its standard input (which has no such restrictions) and pass those on in manageable chunks to the command it has been told to pass them to. This simple listfiles script shows the effect of this on the command being run. Here, the -L 3 option tells xargs to break the input down into three files per call. This is not how everybody thinks that xargs works, so the result may be surprising.

download.eps
$ cat listfiles
#!/bin/bash
echo "Listfiles (PID $$) was called with $# arguments:"
i=1
while [ -a "$1" ]
do
  echo "${i}: $1"
  ((i++))
  shift
done

listfiles

$ find . -print
.
./etc
./etc/hosts
./etc/config.txt
./bin
./bin/ls
./sh
./listfiles
$ find . -print | xargs -L 3 ./listfiles Listfiles (PID 17088) was called ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

AirBnbBlueOriginElectronic ArtsHomeDepotNasdaqRakutenTata Consultancy Services

QuotationMarkO’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
QuotationMarkI wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
QuotationMarkI’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
QuotationMarkI'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Mark W.
Embedded Software Engineer

You might also like

Linux Command Line and Shell Scripting Techniques

Linux Command Line and Shell Scripting Techniques

Vedran Dakic, Jasmin Redzepagic

Publisher Resources

ISBN: 9781118166321Purchase bookDownloads