Chapter 1. Beginning bash

What’s a shell, and why should you care about it?

Any recent computer operating system (by recent, we mean since about 1970) has some sort of user interface—some way of specifying commands for the operating system to execute. But in lots of operating systems, that command interface was really built in and there was only one way to talk to the computer. Furthermore, an operating system’s command interface would let you execute commands, but that was about all. After all, what else was there to do?

The Unix operating system popularized the notion of separating the shell (the part of the system that lets you type commands) from everything else: the input/output system, the scheduler, memory management, and all of the other things the operating system takes care of for you (and that most users don’t want to care about). The shell was just one more program; it was a program whose job was executing other programs on behalf of users.

But that was the beginning of a revolution. The shell was just another program that ran on Unix; if you didn’t like the standard one, you could create your own. So by the end of Unix’s first decade, there were at least two competing shells: the Bourne shell, sh (which was a descendant of the original Thompson shell), plus the C shell, csh. By the end of Unix’s second decade, there were a few more alternatives: the Korn shell, ksh, and the first versions of the bash shell. By the end of Unix’s third decade, there were probably a dozen different shells.

You probably don’t sit around saying, “Should I use csh or bash or ksh today?” You’re probably happy with the standard shell that came with your Linux (or BSD or macOS or Solaris or HP/UX) system. But disentangling the shell from the operating system itself made it much easier for software developers (such as Brian Fox, the creator of bash, and Chet Ramey, the current developer and maintainer) to write better shells—you could create a new shell without modifying the operating system itself. It was much easier to get a new shell accepted, since you didn’t have to talk some operating system vendor into building the shell into their system; all you had to do was package the shell so that it could be installed just like any other program.

Still, you might be thinking that sounds like a lot of fuss for something that just takes commands and executes them. And you would be right—a shell that just let you type commands wouldn’t be very interesting. However, two factors drove the evolution of the Unix shell: user convenience and programming. And the result is a modern shell that does much more than just accept commands.

Modern shells are very convenient. For example, they remember commands that you’ve typed, and let you reuse those commands. Modern shells also let you edit those commands, so they don’t have to be the same each time. And modern shells let you define your own command abbreviations, shortcuts, and other features. For an experienced user, typing commands (e.g., with shorthand, shortcuts, and command completion) is a lot more efficient and effective than dragging things around in a fancy windowed interface.

But beyond simple convenience, shells are programmable. There are many sequences of commands that you type again and again. Whenever you do anything a second time, you should ask, “Can’t I write a program to do this for me?” You can. A shell is also a programming language that’s specially designed to work with your computer system’s commands. So, if you want to generate a thousand MP3 files from WAV files, you can write a shell program (or shell script). If you want to compress all of your system’s logfiles, you can write a shell script to do it. Whenever you find yourself doing a task repeatedly, you should try to automate it by writing a shell script. There are more powerful scripting languages, like Perl, Python, and Ruby, but the Unix shell (whatever flavor of shell you’re using) is a great place to start. After all, you already know how to type commands; why make things more complex?

1.1 Why bash?

Why is this book about bash, and not some other shell? Because bash is everywhere. It may not be the newest, and it’s arguably not the fanciest or the most powerful (though if not, it comes close), nor is it the only shell that’s distributed as open source software—but it is ubiquitous.

The reason has to do with history. The first shells were fairly good programming tools, but not very convenient for users. The C shell added a lot of user conveniences (like the ability to repeat a command you’d just typed), but as a programming language it was quirky. The Korn shell, which came along next (in the early ’80s), added a lot of user conveniences, improved the programming language, and looked like it was on the path to widespread adoption. But ksh wasn’t open source software at first; it was a proprietary software product, and was therefore difficult to ship with a free operating system like Linux. (The Korn shell’s license was changed in 2000, and again in 2005.)

In the late 1980s, the Unix community decided standardization was a good thing, and the POSIX working groups (organized by the IEEE) were formed. POSIX standardized the Unix libraries and utilities, including the shell. The standard shell was primarily based on the 1988 version of the Korn shell, with some C shell features and a bit of invention to fill in the gaps. bash was begun as part of the GNU Project’s effort to produce a complete POSIX system, which naturally needed a POSIX shell.

bash provided the programming features that shell programmers needed, plus the conveniences that command-line users liked. It was originally conceived as an alternative to the Korn shell, but as the free software movement became more important, and as Linux became more popular, bash quickly overshadowed ksh.

As a result, bash is the default user shell on every Linux distribution we know about (there are a few hundred Linux distros, so there are probably a few with some oddball default shell), as well as macOS (and the earlier OS X versions). It’s also available for just about every other Unix operating system, including BSD Unix and Solaris. In the rare cases where bash doesn’t ship with the operating system, it’s easy to install. It’s even available for Windows, via Cygwin and also the new Linux Subsystem (Ubuntu). bash is both a powerful programming language and a good user interface, and you won’t find yourself sacrificing keyboard shortcuts to get elaborate programming features.

You can’t possibly go wrong by learning bash. The most common default shells are the old Bourne shell and bash, which is mostly Bourne shell–compatible. One of these shells is certainly present on any modern, major Unix or Unix-like operating system. And as noted, if bash isn’t present you can always install it. But there are other shells. In the spirit of free software, the authors and maintainers of all of these shells share ideas. If you read the bash change logs, you’ll see many places where a feature was introduced or tweaked to match behavior on another shell. But most people won’t care. They’ll use whatever is already there and be happy with it. So if you are interested, by all means investigate other shells. There are many good alternatives, and you may find one you like better—though it probably won’t be as ubiquitous as bash.

1.2 The bash Shell

bash is a shell: a command interpreter. The main purpose of bash (or of any shell) is to allow you to interact with the computer’s operating system so that you can accomplish whatever you need to do. Usually that involves launching programs, so the shell takes the commands you type, determines from that input what programs need to be run, and launches them for you. You will also encounter tasks that involve a sequence of actions to perform that are recurring, or very complicated, or both. Shell programming, usually referred to as shell scripting, allows you to automate these tasks for ease of use, reliability, and reproducibility.

In case you’re new to bash, we’ll start with some basics. If you’ve used Unix or Linux at all, you probably aren’t new to bash—but you may not have known you were using it. bash is really just a language for executing commands—so the commands you’ve been typing all along (e.g., ls, cd, grep, cat) are, in a sense, bash commands. Some of these commands are built into bash itself; others are separate programs. For now, it doesn’t make a difference which are which.

We’ll end this chapter with a few recipes for getting bash. Most systems come with bash preinstalled, but a few don’t. Even if your system comes with bash, it’s always a good idea to know how to get and install it—new versions, with new features, are released from time to time.

If you’re already running bash, and are somewhat familiar with it, you may want to go straight to Chapter 2. You are not likely to read this book in order, and if you dip into the middle, you should find some recipes that demonstrate what bash is really capable of. But first, the basics.

1.3 Decoding the Prompt

Problem

You’d like to know what all the punctuation on your screen means.

Solution

All command-line shells have some kind of prompt to alert you that the shell is ready to accept your input. What the prompt looks like depends on many factors including your operating system type and version, shell type and version, distribution, and how someone else may have configured it. In the Bourne family of shells, a trailing $ in the prompt generally means you are logged in as a regular user, while a trailing # means you are root. The root account is the administrator of the system, equivalent to the System account on Windows (which is even more powerful than the Administrator account). root is all-powerful and can do anything on a typical Unix or Linux system.

Default prompts also often display the path to the directory that you are currently in; however, they usually abbreviate it, so a ~ means you are in your home directory. Some default prompts may also display your username and the name of the machine you are logged into. If that seems silly now, it won’t when you’re logged into five machines at once, possibly under different usernames.

Here is a typical Linux prompt for a user named jp on a machine called adams, sitting in the home directory. The trailing $ indicates this is a regular user, not root:

jp@adams:~$

Here’s the prompt after changing to the /tmp directory. Notice how ~, which really meant /home/jp, has changed to /tmp:

jp@adams:/tmp$

Discussion

The shell’s prompt is the thing you will see most often when you work at the command line, and there are many ways to customize it more to your liking. But for now, it’s enough to know how to interpret it. Of course, your default prompt may be different, but you should be able to figure out enough to get by for now.

There are some Unix or Linux systems where the power of root may be shared, using commands like su and sudo. Or root may not even be all-powerful, if the system is running some kind of mandatory access control (MAC) system such as the NSA’s SELinux.

1.4 Showing Where You Are

Problem

You are not sure what directory you are in, and the default prompt is not helpful.

Solution

Use the pwd builtin command, or set a more useful prompt (as described in Recipe 16.2). For example:

bash-4.3$ pwd
/tmp

bash-4.3$ export PS1='[\u@\h \w]$ '
[jp@solaris8 /tmp]$

Discussion

pwd stands for print working directory and takes two options. -L displays your logical path and is the default. -P displays your physical location, which may differ from your logical path if you have followed a symbolic link. Similarly, the cd command also provides -P and -L switches:

bash-4.3$ pwd
/tmp/dir2

bash-4.3$ pwd -L
/tmp/dir2

bash-4.3$ pwd -P
/tmp/dir1

1.5 Finding and Running Commands

Problem

You need to find and run a particular command under bash.

Solution

Try the type, which, apropos, locate, slocate, find, and ls commands.

Discussion

bash keeps a list of directories in which it should look for commands in an environment variable called PATH. The bash builtin type command searches your environment (including aliases, keywords, functions, builtins, directories in $PATH, and the command hash table) for executable commands matching its arguments and displays the type and location of any matches. It has several options, notably the -a flag, which causes it to print all matches instead of stopping at the first one. The which command is similar but only searches your $PATH (and csh aliases). It may vary from system to system (it’s usually a csh shell script on BSD, but a binary on Linux), and usually has a -a flag like type. Use these commands when you know the name of a command and need to know exactly where it’s located, or to see if it’s on this computer. For example:

$ type which
which is hashed (/usr/bin/which)

$ type ls
ls is aliased to `ls -F -h'

$ type -a ls
ls is aliased to `ls -F -h'
ls is /bin/ls

$ which which
/usr/bin/which

Almost all commands come with some form of help on how to use them. Usually there is online documentation called manpages, where “man” is short for manual. These are accessed using the man command, so man ls will give you documentation about the ls command. Many programs also have a built-in help facility, accessed by providing a “help me” argument such as -h or --help. Some programs, especially on other operating systems, will give you help if you don’t give them arguments. Some Unix commands will also do that, but a great many of them will not. This is due to the way that Unix commands fit together into something called pipelines, which we’ll cover later. But what if you don’t know or can’t remember the name of the command you need? apropos searches manpage names and descriptions for regular expressions supplied as arguments. This is incredibly useful when you don’t remember the name of the command you need. This is the same as man -k:

$ apropos music
cms (4) - Creative Music System device driver

$ man -k music
cms (4) - Creative Music System device driver

locate and slocate consult database files about the system (usually compiled and updated by a job run from the scheduler system cron) to find files or commands almost instantly. The location of the actual database files, what is indexed therein, and how often it is checked may vary from system to system. Consult your system’s manpages for details. slocate (secure locate) stores permission information (in addition to filenames and paths) so that it will not list programs to which the user does not have access. On most Linux systems, locate is a symbolic link to slocate; other systems may have separate programs, or may not have slocate at all. Here’s an example:

$ locate apropos
/usr/bin/apropos
/usr/share/man/de/man1/apropos.1.gz
/usr/share/man/es/man1/apropos.1.gz
/usr/share/man/it/man1/apropos.1.gz
/usr/share/man/ja/man1/apropos.1.gz
/usr/share/man/man1/apropos.1.gz

For details on the find command, see Chapter 9.

Last but not least, try using ls. Remember, if the command you wish to run is in your current directory, you must prefix it with a ./ since the current working directory is usually not in your $PATH for security reasons (see Recipes 14.3 and 14.10).

1.6 Getting Information About Files

Problem

You need more information about a file, such as what it is, who owns it, if it’s executable, how many hard links it has, or when it was last accessed or changed.

Solution

Use the ls, stat, file, or find commands:

$ touch /tmp/sample_file

$ ls /tmp/sample_file
/tmp/sample_file

$ ls -l /tmp/sample_file
-rw-r--r-- 1 jp         jp            0 Dec 18 15:03 /tmp/sample_file

$ stat /tmp/sample_file
File: "/tmp/sample_file"
Size: 0           Blocks: 0        IO Block: 4096   Regular File
Device: 303h/771d Inode:  2310201    Links: 1
Access: (0644/-rw-r--r--) Uid: (  501/      jp)   Gid: ( 501/        jp)
Access: Sun Dec 18 15:03:35 2005
Modify: Sun Dec 18 15:03:35 2005
Change: Sun Dec 18 15:03:42 2005

$ file /tmp/sample_file
/tmp/sample_file: empty

$ file -b /tmp/sample_file
empty

$ echo '#!/bin/bash -' > /tmp/sample_file

$ file /tmp/sample_file
/tmp/sample_file: Bourne-Again shell script text executable

$ file -b /tmp/sample_file
Bourne-Again shell script text executable

For much more on the find command, see Chapter 9.

Discussion

The command ls shows only filenames, while -l provides more details about each file. ls has many options; consult the manpage on your system for the ones it supports. Useful options include:

-a

Do not hide files starting with . (dot).

-A

Like -a, but skips the two common directories . (dot) and .. (dot dot), since they are present in virtually every directory.

-F

Show the type of file with one of several trailing type designators.

A slash (/) indicates that the file is a directory, an asterisk (*) means the file is executable, an at sign (@) indicates a symbolic link, an equals sign (=) is a socket, and a pipe or vertical bar (|) is a FIFO (first in, first out) buffer.

-l

Use the long listing format.

-L

Show information about the linked file, rather than the symbolic link itself.

-Q

Quote names (GNU extension, not supported on all systems).

-r

Reverse the sort order.

-R

Recurse through subdirectories.

-S

Sort by file size.

-1

Use the short format, but with only one file per line.

stat, file, and find all have many options that control the output format; see the manpages on your system for supported options. For example, these options produce output that is similar to ls -l:

$ ls -l /tmp/sample_file
-rw-r--r--  1 jp         jp                14 Dec 18 15:04 /tmp/sample_file

$ stat -c'%A %h %U %G %s %y %n' /tmp/sample_file
-rw-r--r-- 1 jp jp 14 Sun Dec 18 15:04:12 2005 /tmp/sample_file

$ find /tmp/ -name sample_file -printf '%m %n %u %g %t %p'
644 1 jp jp Sun Dec 18 15:04:12 2005 /tmp/sample_file

Not all operating systems and versions have all of these tools. For example, Solaris does not include stat by default.

It is also worth pointing out that directories are nothing more than files that the operating system knows to treat specially, so the commands shown here will work just fine on directories, though sometimes you may need to modify a command to get the behavior you want. For example, use ls -d to list information about the directory itself, rather than just ls (which lists the contents of the directory).

1.7 Showing All Hidden (Dot) Files in the Current Directory

Problem

You want to see only hidden (dot) files in a directory to edit a file you’ve forgotten the name of or remove obsolete files. ls -a shows all files, including normally hidden ones, but that is often too noisy, and ls -a .* does more than you think it will, or more than you want.

Solution

Use ls -d along with whatever other criteria you have. For example:

ls -d .*
ls -d .b*
ls -d .[!.]*
ls -d .*/

Since every normal directory contains a . and .., you don’t need to see those. You can use ls -A to list all the files in a directory except those two. For other commands where you list files with a wildcard (i.e., pattern), you can construct your wildcard in such a way that . and .. don’t match:

$ grep -l 'PATH' ~/.[!.]*
/home/jp/.bash_history
/home/jp/.bash_profile
$

Discussion

Due to the way the shell handles file wildcards, the sequence .* does not behave as you might expect or desire. The way filename expansion or globbing works is that any string containing the characters *, ?, or [ is treated as a pattern, and replaced by an alphabetically sorted list of filenames matching the pattern. * matches any string, including the null string, while ? matches any single character. Characters enclosed in [] specify a list or range of characters, any of which will match. There are also various extended pattern-matching operators that we’re not going to cover here (see “Pattern-Matching Characters” and “extglob Extended Pattern-Matching Operators” in Appendix A). So, *.txt means any file ending in .txt, while *txt means any file ending in txt (no dot). f?o would match foo or fao but not fooo. Given that, you might think that .* would match any file beginning with a dot.

The problem is that .* matches both the . and .. directories (present in every directory), which are then both displayed along with any other filenames beginning with a dot. When ls is given a directory name it doesn’t just list that directory name, but also the contents of that directory. Instead of getting just the dot files in the current directory, then, you get those files plus all the files and directories in the current directory (.), all the files and directories in the parent directory (..), and the names and contents of any subdirectories in the current directory that start with a dot. This can be very confusing, to say the least, and is usually more than you want.

You can experiment with the same ls command with -d and without, then try echo . *. The echo command simply shows you what the shell expands your .* into, which would become the arguments to the ls command.

Try echo .[!.]* also. .[!.]* is a filename expansion pattern where [] denotes a list of characters to match, but the leading ! negates the list. So here we are looking for a dot, followed by any character that is not a dot, followed by any number of any characters. You may also use ^ to negate a character class, but ! is specified in the POSIX standard and thus is more portable.

There is one other special case in the ls command that helps out here. If the -d option is specified and if the filename pattern ends with a slash, then only directories that match that pattern, rather than all filenames that match, are displayed by the ls command. For example:

$ ls -d .v*
.vim  .viminfo  .vimrc
$ ls -d .v*/
.vim
$

The first command shows the three filenames that begin with .v without listing (because of the -d option) the contents of any that might be a directory. The second command in this example uses a trailing slash on the pattern (.v*/) so the ls command only shows directories that match the pattern; in this case that is just the directory named .vim and no other.

Warning

If you see double slashes on the output of the ls -d .v*/ command, like this:

$ ls -d .v*/
.vim//
$

that’s likely because you may actually be running an alias for ls that includes the -F flag. Use a backslash in front of the command name to avoid any alias:

$ \ls -d .v*/
.vim/
$

Some combinations are just difficult to match. .[!.]* will miss a file named ..foo. You could add something like .??* to match anything starting with a dot that is also at least three characters long, but ls -d .[!.]* .??* will then display anything that matches both patterns twice. Or you can use .??* alone, but that will miss files like .a:

$ touch ..foo .a .normal_dot_file normal_file

$ ls -a
. .. ..foo .a .normal_dot_file normal_file

$ ls -d .??*
..foo .normal_dot_file

$ ls -d .[!.]*
.a .normal_dot_file

$ ls -d .[!.]* .??* | sort -u
..foo
.a
.normal_dot_file

Which you use depends on your needs and environment; there is no good one-size-fits-all solution.

Note

You can use echo * as an emergency substitute for ls if the ls command is corrupt or not available for some reason. This works because * is expanded by the shell to everything in the current directory, which results in a list similar to what you’d get with ls.

1.8 Using Shell Quoting

Problem

You need a rule of thumb for using command-line quoting.

Solution

Enclose a string in single quotes unless it contains elements that you want the shell to interpolate.

Discussion

Unquoted text and even text enclosed in double quotes is subject to shell expansion and substitution. Consider:

$ echo A coffee is $5?!
A coffee is ?!

$ echo "A coffee is $5?!"
-bash: !": event not found

$ echo 'A coffee is $5?!'
A coffee is $5?!

In the first example, $5 is treated as a variable to expand, but since it doesn’t exist it is set to null. In the second example, the same is true, but we never even get there because ! is treated as a history substitution, which fails in this case because it doesn’t match anything in the history. The third example works as expected.

To mix some shell expansions with some literal strings you may use the shell escape character \ or change your quoting. The exclamation point is a special case because the preceding backslash escape character is not removed. You can work around that by using single quotes or a trailing space, as shown here:

$ echo 'A coffee is $5 for' "$USER" '?!'
A coffee is $5 for jp ?!

$ echo "A coffee is \$5 for $USER?\!"
A coffee is $5 for jp?\!

$ echo "A coffee is \$5 for $USER?! "
A coffee is $5 for jp?!

Also, you can’t embed a single quote inside single quotes, even if using a backslash, since nothing (not even the backslash) is interpolated inside single quotes. But you can work around that by using double quotes with escapes, or by escaping a single quote outside of surrounding single quotes.

# We'll get a continuation prompt since we now have unbalanced quotes
$ echo '$USER won't pay $5 for coffee.'
> ^C

# WRONG
$ echo "$USER won't pay $5 for coffee."
jp won't pay for coffee.

# Works
$ echo "$USER won't pay \$5 for coffee."
jp won't pay $5 for coffee.

# Also works
$ echo 'I won'\''t pay $5 for coffee.'
I won't pay $5 for coffee.

1.9 Using or Replacing Builtins and External Commands

Problem

You want to replace a builtin command with your own function or external command, and you need to know exactly what your script is executing (e.g., /bin/echo or the builtin echo). Or you’ve created a new command and it may be conflicting with an existing external or builtin command.

Solution

Use the type and which commands to see if a given command exists and whether it is built in or external:

$ type cd
cd is a shell builtin

$ type awk
awk is /usr/bin/awk

$ which cd
/usr/bin/which: no cd in (/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/ \
local/sbin:/usr/bin/X11:/usr/X11R6/bin:/root/bin)

$ which awk
/usr/bin/awk

Discussion

A builtin command is just that; it is built into the shell itself, while an external command is an external file launched by the shell. The external file may be a binary, or it may be a shell script itself, and it is important to understand the difference for a couple of reasons. First, when you are using a given version of a particular shell, builtins will always be available but external programs may or may not be installed on a particular system. Second, if you give one of your own programs the same name as a builtin, you will be very confused about the results since the builtin will always take precedence (see Recipe 19.4). It is possible to use the enable command to turn builtin commands off and on, though we strongly recommend against doing so unless you are absolutely sure you understand what you are doing. enable -a will list all builtins and their enabled or disabled status.

One problem with builtin commands is that you generally can’t use a -h or --help option to get usage reminders, and if a manpage exists it’s often just a pointer to the large bash manpage. That’s where the help command, which is itself a builtin, comes in handy. help displays help about shell builtins:

help: help [-dms] [pattern ...]
    Display information about builtin commands.

    Displays brief summaries of builtin commands.  If PATTERN is
    specified, gives detailed help on all commands matching PATTERN,
    otherwise the list of help topics is printed.

    Options:
      -d	output short description for each topic
      -m	display usage in pseudo-manpage format
      -s	output only a short usage synopsis for each topic matching
    	PATTERN

    Arguments:
      PATTERN	Pattern specifying a help topic

    Exit Status:
    Returns success unless PATTERN is not found or an invalid option is given.

When you need to redefine a builtin you use the builtin command to avoid loops. For example, we can define a shell function (see Recipe 10.4) to change how the cd command works:

cd () {
    builtin cd "$@"
    echo "$OLDPWD --> $PWD"
}

To force the use of an external command instead of any function or builtin that would otherwise have precedence, use enable -n, which turns off shell builtins, or command, which ignores shell functions. For example, to use the test found in $PATH instead of the shell builtin version, type enable -n test and then run test. Or, use command ls to use the native ls command rather than any ls function you may have created.

1.10 Determining if You Are Running Interactively

Problem

You have some code you want to run only if you are (or are not) running interactively.

Solution

Use the case statement in Example 1-1.

Example 1-1. ch01/interactive

#!/usr/bin/env bash
# cookbook filename: interactive

case "$-" in
    *i*) # Code for interactive shell here
     ;;
    *)   # Code for noninteractive shell here
     ;;
esac

Discussion

$- is a string listing of all the current shell option flags. It will contain i if the shell is interactive.

You may also see code like the following (this will work, but the solution in Example 1-1 is the preferred method):

if [ -n "$PS1" ]; then
    echo This shell is interactive
else
    echo This shell is not interactive
fi

1.11 Setting bash as Your Default Shell

Problem

You’re using a BSD system, Solaris, or some other Unix variant for which bash isn’t the default shell. You’re tired of starting bash explicitly all the time, and want to make bash your default shell.

Solution

First, make sure bash is installed. Try typing bash --version at a command line. If you get a version, it’s installed:

$ bash --version
GNU bash, version 3.00.16(1)-release (i386-pc-solaris2.10)
Copyright (C) 2004 Free Software Foundation, Inc.
$

If you don’t see a version number, you may be missing a directory from your path. chsh -l or cat /etc/shells may give you a list of valid shells on some systems. Otherwise, ask your system administrator where bash is, or if it can be installed.

chsh -l provides a list of valid shells on Linux, but opens an editor and allows you to change settings on BSD. -l is not a valid option to chsh on macOS, but just running chsh will open an editor to allow you to change settings, and chpass -s shell will change your shell.

If bash is installed, use the chsh -s command to change your default shell: for example, chsh -s /bin/bash. If for any reason that fails, try chsh, passwd -e, passwd -l, chpass, or usermod -s /usr/bin/bash. If you still can’t change your shell ask your system administrator, who may need to edit the /etc/passwd file. On most systems, /etc/passwd will have lines of the form:

cam:pK1Z9BCJbzCrBNrkjRUdUiTtFOh/:501:100:Cameron Newham:/home/cam:/bin/bash
cc:kfDKDjfkeDJKJySFgJFWErrElpe/:502:100:Cheshire Cat:/home/cc:/bin/bash

As root, you can just edit the last field of the lines in the password file to the full pathname of whatever shell you choose. If your system has a vipw command, you should use it to ensure password file consistency.

Warning

Some systems will refuse to allow a login shell that is not listed in /etc/shells. If bash is not listed in that file, you will have to have your system administrator add it.

Discussion

Some operating systems, notably the BSD Unixes, typically place bash in the /usr partition. You may want to think twice about changing root’s shell on such systems. If the system runs into trouble while booting, and you have to work on it before /usr is mounted, you’ve got a real problem: there isn’t a shell for root to use. Therefore, it’s best to leave the default shell for root unchanged. However, there’s no reason not to make bash the default shell for regular user accounts. And it goes without saying that it’s bad practice to use the root account unless it’s absolutely necessary. Use your regular (user) account whenever possible. With commands like sudo, you should very rarely need a root shell.

If all else fails, you can probably replace your existing login shell with bash using exec, but this is not for the faint of heart. See “A7) How can I make bash my login shell?” in the bash FAQ.

1.12 Keeping bash Updated

Problem

This isn’t really a normal recipe, and it probably goes without saying, but it is a topic no one can afford to ignore and we wanted to say it anyway. You need to keep both bash and your entire system up-to-date with security patches.

Solution

Keeping your entire system up-to-date is out of the scope of this book; consult your system administrator and documentation.

How you keep bash up-to-date depends on how you got it in the first place. In the ideal case, it’s part of the system in general and updated when the system is updated. That may not be the case if you are using a very old system that is no longer supported, in which case you need to update the entire thing. If you are using your package system and the originating repository is still actively maintained, you should get updates from there—for example, from Extra Packages for Enterprise Linux (EPEL) or an Ubuntu Personal Package Archive (PPA).

If you installed from source, it will be up to you to update your source and rebuild as appropriate.

Discussion

We all know why we need to stay up-to-date, but we’ll cite one well-known reason anyway: CVE-2014-6271, better known as the shellshock vulnerability.

1.13 Getting bash for Linux

Problem

You want to get bash for your Linux system, or you want to make sure you have the latest version.

Solution

bash is included in virtually all modern Linux distributions. To make sure you have the latest version available for your distribution, use the distribution’s built-in packaging tools. You must be root or have sudo or the root password to upgrade or install applications.

Some Linux distributions (notably the Debian family) use the Debian Almquist shell, or dash, as /bin/sh because it is smaller and thus runs a bit faster than bash. That switchover caused a lot of confusion when scripts assumed that /bin/sh was really bash, as scripts using bash features with #!/bin/sh would fail. See Recipe 15.3 for more details.

For Debian and Debian-derived systems such as Ubuntu and Linux Mint, use one of the various graphical user interface (GUI) tools or a command-line tool such as apt-get, aptitude, or apt to make sure it is installed and current:

apt-get update && apt-get install bash bash-completion bash-doc

For Red Hat distributions, including Fedora, Community OS (CentOS), and Red Hat Enterprise Linux (RHEL), use the GUI Add/Remove Applications tool. For a command line only, use:

yum update bash

For SUSE, use either the GUI or terminal version of YaST. You may also use the command-line rpm tool.

Discussion

It’s impossible to cover every Linux distribution and difficult even to cover the major ones, as they are all evolving rapidly. Fortunately, much of that evolution is in the area of ease of use, so it should not be very difficult to figure out how to install software on your distribution of choice.

When using LiveCDs, software updates and installations will most likely fail due to the read-only media. Versions of such distributions that have been installed to a hard disk should be updatable.

If you are wondering what version of bash is available in a given Linux distribution, search for the distro on DistroWatch.com and consult the package table. For example, https://distrowatch.com/table.php?distribution=mint shows what you see in Table 1-1.

Table 1-1. Bash versions in Linux Mint
Package	18 sarah	17.3 rosa	16 petra	15 olivia	14 nadia	13 maya	12 lisa	11 katya	10 julia	…
bash (4.4)	4.3	4.3	4.2	4.2	4.2	4.2	4.2	4.2	4.1	…

1.14 Getting bash for xBSD

Problem

You want to get bash for your FreeBSD, NetBSD, or OpenBSD system, or you want to make sure you have the latest version.

Solution

According to Chet Ramey’s bash page:

Bash-4.3 is included as part of the FreeBSD ports collection, the OpenBSD packages collection, and the NetBSD packages collection.

To see if bash is installed, check the /etc/shells file. To install or update bash, use the pkg_add command. If you are an experienced BSD user, you may prefer using the ports collection, but we will not cover that here.

If you are wondering what version of bash is available in a given BSD distribution, search for the distro on DistroWatch.com and consult the package table. For example:

For FreeBSD, use the command:

pkg_add -vr bash

For NetBSD, browse to Application Software for NetBSD and locate the latest bash package for your version and architecture, then use a command such as:

pkg_add -vu ftp://ftp.netbsd.org/pub/NetBSD/packages/pkgsrc-2005Q3/NetBSD-2.0/ \
i386/All/bash-3.0pl16nb3.tgz

For OpenBSD, you use the pkg_add -vr command. You may have to adjust the FTP path for your version and architecture. Also, there may be a statically compiled version. For example:

pkg_add -vr ftp://ftp.openbsd.org/pub/OpenBSD/3.8/packages/i386/bash-3.0.16p1.tgz

Discussion

FreeBSD and OpenBSD place bash in /usr/local/bin/bash while NetBSD uses /usr/pkg/ bin/bash.

1.15 Getting bash for macOS

Problem

You want to get bash for your Mac, or you want to make sure you have the latest version.

Solution

According to Chet Ramey’s bash page:

Current versions of Mac OS X [now called macOS] (dating from Jaguar/Mac OS X 10.2) ship with bash-3.2 as /bin/sh. There are also precompiled OS X packages of bash-4.3 available from many web sites, though the source packages are usually more up-to-date. Bash for Darwin (the base for MacOS X) is available from MacPorts, Homebrew, or Fink.

Discussion

It is also possible to build a more recent version of bash from source, but this is recommended only for experienced users (see Appendix E).

1.16 Getting bash for Unix

Problem

You want to get bash for your Unix system, or you want to make sure you have the latest version.

Solution

If it’s not already installed or in your operating system’s program repository, check Chet Ramey’s bash page for binary downloads, or build it from source (see Appendix E).

Discussion

According to Chet Ramey’s bash page:

The OpenPKG project makes source RPMs of bash-4.3 available for a variety of Unix and Linux systems as a core part of the current release.

Solaris 2.x, Solaris 7/8/9/10/11 users can get a precompiled version of bash-4.3 from the Unixpackages site (subscription) or from OpenCSW. Oracle ships bash-3.2 as a supported part of Solaris 10 and bash-4.1 as part of Solaris 11. The version of Solaris/Illumos distributed as OpenIndiana includes bash-4.3 as of September 2016.

AIX users can get precompiled versions of bash-4.3 and older releases for various versions of AIX from Groupe Bull, and sources and binaries of bash-4.3 for various AIX releases from perzl.org. IBM makes bash-4.2 and bash-4.3 available for AIX 5L, AIX 6.1, and AIX 7.1 as part of the AIX toolbox for GNU/Linux applications. They use RPM format; you can get RPM for AIX from there, too.

HP-UX users can get bash-4.3 binaries and source code from the Software Porting and Archive Center for HP-UX.

1.17 Getting bash for Windows

Problem

You want to get bash for your Windows system, or you want to make sure you have the latest version.

Solution

Use Cygwin or Ubuntu on Windows, or a virtual machine. Or, don’t use bash.

Download Cygwin and run it. Follow the prompts and choose the packages to install, including bash, which is located in the shells category and is selected by default. Once Cygwin is installed, you will have to configure it. See the User’s Guide for details.

For Ubuntu on Windows you need a version of Windows 10 from summer 2016 or newer; then follow the install instructions, detailed in the Discussion.

To use a virtual machine, see Recipe 15.4.

Finally, though we hate to say it, maybe the right solution is to use native tools like PowerShell.

Discussion

Cygwin

Cygwin is a Linux-like environment for Windows that provides a Linux look and feel.

From the Cygwin site:

Cygwin is:

a large collection of GNU and Open Source tools which provide functionality similar to a Linux distribution on Windows.

a DLL (cygwin1.dll) which provides substantial POSIX API functionality.

Cygwin is not:

a way to run native Linux apps on Windows. You must rebuild your application from source if you want it to run on Windows.

a way to magically make native Windows apps aware of UNIX® functionality like signals, ptys, etc. Again, you need to build your apps from source if you want to take advantage of Cygwin functionality.

The Cygwin DLL currently works with all recent, commercially released x86 32 bit and 64 bit versions of Windows, starting with Windows Vista.

Note

The previous Cygwin version 2.5.2 was the last version supporting Windows XP and Server 2003.

Cygwin is a true Unix-like environment running on top of Windows. It is an excellent tool, but sometimes it might be overkill. For Windows native binaries of the GNU Text utils (not including bash), see http://unxutils.sourceforge.net/.

Ubuntu on Windows

Running Ubuntu on Windows is very interesting, but aside from the fact that it includes bash it is out of scope for this book, so we won’t cover it in detail. See the references listed in the See Also section for details.

Briefly:

Turn on Developer Mode (see http://bit.ly/2h21MSZ).
- Search for “Windows Features.”
- Choose “Turn Windows features on or off,” and enable “Windows Subsystem for Linux.”
  - This probably requires a reboot!!! Seriously!?!
Open the Command Prompt and type in bash.
- Download the Windows Subsystem for Linux from the Windows store.

Using PowerShell or other native tools

PowerShell is Microsoft’s answer to the power and flexibility of scriptable command-line tools and the replacement for the command.com and cmd.exe batch files. Other than the fact that it is the Windows native answer to shell scripting, it is out of scope for this book, so we won’t cover it.

Though they pale in comparison to any of the Unix/Linux tools, the old shell script languages were more powerful than many people knew. They may be appropriate for very simple tasks where any of the other solutions discussed here are overkill. See http://www.jpsdomain.org/windows/winshell.html for details.

For powerful character-based and GUI command-line shells with a more consistent interface but a DOS/Windows flavor, see http://jpsoft.com. None of the authors are affiliated with this company, but one is a long-time satisfied user.

1.18 Getting bash Without Getting bash

Problem

You want to try out a shell or a shell script on a system you don’t have the time or the resources to build or buy.

Or, you feel like reading a Zen-like recipe just about now.

Solution

Get an almost free shell account from http://polarhome.com, which has a tiny, symbolic one-time fee, or another vendor.

Since almost every Linux and BSD distribution has a LiveCD or LiveDVD image, which can also almost certainly be used as a LiveUSB, you can download and boot those to experiment. That’s also a good idea if you are thinking about switching, so you can verify that all your hardware is supported and works. The tricky part may be getting your system’s BIOS or UEFI to boot from the CD/DVD or USB. It used to be tricky to “burn” an ISO to a USB stick, but there are now many tools and detailed instructions on the web for your distro of choice.

Alternatively, you can use a virtualization solution; see Recipe 15.4.

Discussion

Polarhome provides many free services and almost free shell accounts. According to the website:

polarhome.com is a non commercial, educational effort for popularization of shell enabled operating systems and Internet services, offering shell accounts, development environment, mail and other online services on all available systems (currently on different flavours of Linux, MacOS X, OpenVMS, Solaris, OpenIndiana, AIX, QNX, IRIX, HP-UX, Tru64, SCO OpenServer, UnixWare, FreeBSD, OpenBSD, NetBSD, DragonFly/BSD, MirBSD, Ultrix, Minix, GNU Hurd, Syllable and OPENSTEP).

1.19 Learning More About bash Documentation

Problem

You’d like to read more about bash but don’t know where to start.

Solution

Well, you’re reading this book, which is a great place to start! The other O’Reilly books about bash and shell scripting are Learning the bash Shell, 3rd Edition, by Cameron Newham and Classic Shell Scripting by Nelson H. F. Beebe and Arnold Robbins.

Unfortunately, not all of the official bash documentation and support files are easily accessible online. The Bash Reference Manual is available on the GNU Project website, but much of the other material is harder to find or access. Our companion website has done all the work for you and provides the official bash reference documentation and other useful material online so it’s easy to refer to. Check it out, and refer others to it as needed.

Official documentation

The official bash FAQ is at ftp://ftp.cwru.edu/pub/bash/FAQ. See especially “H2) What kind of bash documentation is there?” The official reference guide is also strongly recommended; see below for details.

Chet Ramey’s bash page contains a ton of very useful information. Chet (the current bash maintainer) also maintains the following:

README: A file describing bash
NEWS: A file tersely listing the notable changes between the current and previous versions
CHANGES: A complete bash change history
INSTALL: Installation instructions
NOTES: Platform-specific configuration and operation notes
COMPAT: A list of compatibility issues between bash3 and bash1

The latest bash source code and documentation are always available at http://ftp.gnu.org/gnu/bash/.

We highly recommend downloading both the source and the documentation even if you are using prepackaged binaries (see Appendix B for an index of the included examples and source code). Here is a brief list of the documentation included in the source tarball’s ./doc directory (for example, for http://ftp.gnu.org/gnu/bash/bash-4.4.tar.gz/, bash-4.4/doc):

FAQ: A set of frequently asked questions about bash with answers
INTRO: A short introduction to bash
article.ms: An article Chet wrote about bash for The Linux Journal
bash.1: The bash manpage
bashbug.1: The bashbug manpage
builtins.1: A manpage that documents the builtins extracted from bash.1
bashref.texi: The Bash Reference Manual
bashref.info: The Bash Reference Manual processed by makeinfo
rbash.1: The restricted bash shell manpage
readline.3: The readline manpage

The .ps files are PostScript versions of the files listed here. The .html files are HTML versions of the manpage and reference manual. The .0 files are formatted manual pages. The .txt versions are ASCII—the output of groff -Tascii.

In the document tarball (for example, http://ftp.gnu.org/gnu/bash/bash-doc-4.4.tar.gz), you will find formatted versions of:

bash-doc-4.4:

bash.0: The bash manpage (also .pdf, .ps, .html)
bashbug.0: The bashbug manpage
bashref: The GNU Bash Reference Manual (also .pdf, .ps, .html, .dvi)
builtins.0: The builtins manpage
rbash.0: The restricted bash shell manpage