Classic Shell Scripting

Chapter Summary

We recommend reading the book in order, as each chapter builds upon the concepts and material covered in the chapters preceding it. Here is a chapter-by-chapter summary:

Chapter 1

Here we provide a brief history of Unix. In particular, the computing environment at Bell Labs where Unix was developed motivated much of the Software Tools philosophy. This chapter also presents the principles for good Software Tools that are then expanded upon throughout the rest of the book.

Chapter 2

This chapter starts off the discussion. It begins by describing compiled languages and scripting languages, and the tradeoffs between them. Then it moves on, covering the very basics of shell scripting with two simple but useful shell scripts. The coverage includes commands, options, arguments, shell variables, output with echo and printf, basic I/O redirection, command searching, accessing arguments from within a script, and execution tracing. It closes with a look at internationalization and localization; issues that are increasingly important in today's "global village."

Chapter 3

Here we introduce text searching (or "matching") with regular expressions. We also cover making changes and extracting text. These are fundamental operations that form the basis of much shell scripting.

Chapter 4

In this chapter we describe a number of the text processing software tools that are used over and over again when shell scripting. Two of the most important tools presented here are sort and uniq, which serve as powerful ways to organize and reduce data. This chapter also looks at reformatting paragraphs, counting text units, printing files, and retrieving the first or last lines of a file.

Chapter 5

This chapter shows several small scripts that demonstrate combining simple Unix utilities to make more powerful, and importantly, more flexible tools. This chapter is largely a cookbook of problem statements and solutions, whose common theme is that all the solutions are composed of linear pipelines.

Chapter 6

This is the first of two chapters that cover the rest of the essentials of the shell language. This chapter looks at shell variables and arithmetic, the important concept of an exit status, and how decision making and loops are done in the shell. It rounds off with a discussion of shell functions.

Chapter 7

This chapter completes the description of the shell, focusing on input/output, the various substitutions that the shell performs, quoting, command-line evaluation order, and shell built-in commands.

Chapter 8

Here we demonstrate combinations of Unix tools to carry out more complex text processing jobs. The programs in this chapter are larger than those in Chapter 5, but they are still short enough to digest in a few minutes. Yet they accomplish tasks that are quite hard to do in conventional programming languages such as C, C++, or Java©.

Chapter 9

This chapter describes the essentials of the awk language. awk is a powerful language in its own right. However, simple, and sometimes, not so simple, awk programs can be used with other programs in the software toolbox for easy data extraction, manipulation, and formatting.

Chapter 10

This chapter introduces the primary tools for working with files. It covers listing files, making temporary files, and the all-important find command for finding files that meet specific criteria. It looks at two important commands for dealing with disk space utilization, and then discusses different programs for comparing files.

Chapter 11

Here we tie things together by solving an interesting and moderately challenging task.

Chapter 12

This chapter uses the problem of doing spellchecking to show how it can be solved in different ways. It presents the original Unix shell script pipeline, as well as two small scripts to make the freely available ispell and aspell commands more usable for batch spellchecking. It closes off with a reasonably sized yet powerful spellchecking program written in awk, which nicely demonstrates the elegance of that language.

Chapter 13

This chapter moves out of the realm of text processing and into the realm of job and system management. There are a small number of essential utilities for managing processes. In addition, this chapter covers the sleep command, which is useful in scripts for waiting for something to happen, as well as other standard tools for delayed or fixed-time-of-day command processing. Importantly, the chapter also covers the trap command, which gives shell scripts control over Unix signals.

Chapter 14

Here we describe some of the more useful extensions available in both ksh and bash that aren't in POSIX. In many cases, you can safely use these extensions in your scripts. The chapter also looks at a number of "gotchas" waiting to trap the unwary shell script author. It covers issues involved when writing scripts, and possible implementation variances. Furthermore, it covers download and build information for ksh and bash. It finishes up by discussing shell initialization and termination, which differ among different shell implementations.

Chapter 15

In this chapter we provide a cursory introduction to shell scripting security issues.

Appendix A

This chapter describes how to write a manual page. This necessary skill is usually neglected in typical Unix books.

Appendix B

Here we describe the Unix byte-stream filesystem model, contrasting it with more complex historical filesystems and explaining why this simplicity is a virtue.

Appendix C

This chapter provides several lists of Unix commands. We recommend that you learn these commands and what they do to improve your skills as a Unix developer.

Bibliography

Here we list further sources of information about shell scripting with Unix.

Glossary

The Glossary provides definitions for the important terms and concepts introduced in this book.