Book description
For system administrators, programmers, and end users, shell command or carefully crafted shell script can save you time and effort, or facilitate consistency and repeatability for a variety of common tasks. This cookbook provides more than 300 practical recipes for using bash, the popular Unix shell that enables you to harness and customize the power of any Unix or Linux system.
Ideal for new and experienced users alike—including proficient Windows users and sysadmins—this updated second edition helps you solve a wide range of problems. You’ll learn ways to handle input/output, file manipulation, program execution, administrative tasks, and many other challenges. Each recipe includes one or more scripting examples and a discussion of why the solution works.
You’ll find recipes for problems including:
- Standard output and input, and executing commands
- Shell variables, shell logic, and arithmetic
- Intermediate shell tools and advanced scripting
- Searching for files with find, locate, and slocate
- Working with dates and times
- Creating shell scripts for various end-user tasks
- Working with tasks that require parsing
- Writing secure shell scripts
- Configuring and customizing bash
Publisher resources
Table of contents
- Preface
-
Beginning bash
- 1.1. Why bash?
- 1.2. The bash Shell
- 1.3. Decoding the Prompt
- 1.4. Showing Where You Are
- 1.5. Finding and Running Commands
- 1.6. Getting Information About Files
- 1.7. Showing All Hidden (Dot) Files in the Current Directory
- 1.8. Using Shell Quoting
- 1.9. Using or Replacing Builtins and External Commands
- 1.10. Determining if You Are Running Interactively
- 1.11. Setting bash as Your Default Shell
- 1.12. Keeping bash Updated
- 1.13. Getting bash for Linux
- 1.14. Getting bash for xBSD
- 1.15. Getting bash for macOS
- 1.16. Getting bash for Unix
- 1.17. Getting bash for Windows
- 1.18. Getting bash Without Getting bash
- 1.19. Learning More About bash Documentation
-
Standard Output
- 2.1. Writing Output to the Terminal/Window
- 2.2. Writing Output but Preserving Spacing
- 2.3. Writing Output with More Formatting Control
- 2.4. Writing Output Without the Newline
- 2.5. Saving Output from a Command
- 2.6. Saving Output to Other Files
- 2.7. Saving Output from the ls Command
- 2.8. Sending Output and Error Messages to Different Files
- 2.9. Sending Output and Error Messages to the Same File
- 2.10. Appending Rather than Clobbering Output
- 2.11. Using Just the Beginning or End of a File
- 2.12. Skipping a Header in a File
- 2.13. Throwing Output Away
- 2.14. Saving or Grouping Output from Several Commands
- 2.15. Connecting Two Programs by Using Output as Input
- 2.16. Saving a Copy of Output Even While Using It as Input
- 2.17. Connecting Two Programs by Using Output as Arguments
- 2.18. Using Multiple Redirects on One Line
- 2.19. Saving Output When Redirect Doesn’t Seem to Work
- 2.20. Swapping STDERR and STDOUT
- 2.21. Keeping Files Safe from Accidental Overwriting
- 2.22. Clobbering a File on Purpose
- Standard Input
-
Executing Commands
- 4.1. Running Any Executable
- 4.2. Running Several Commands in Sequence
- 4.3. Running Several Commands All at Once
- 4.4. Telling Whether a Command Succeeded or Not
- 4.5. Running a Command Only if Another Command Succeeded
- 4.6. Using Fewer if Statements
- 4.7. Running Long Jobs Unattended
- 4.8. Displaying Error Messages When Failures Occur
- 4.9. Running Commands from a Variable
- 4.10. Running All Scripts in a Directory
-
Basic Scripting: Shell Variables
- 5.1. Documenting Your Script
- 5.2. Embedding Documentation in Shell Scripts
- 5.3. Promoting Script Readability
- 5.4. Separating Variable Names from Surrounding Text
- 5.5. Exporting Variables
- 5.6. Seeing All Variable Values
- 5.7. Using Parameters in a Shell Script
- 5.8. Looping Over Arguments Passed to a Script
- 5.9. Handling Parameters with Spaces
- 5.10. Handling Lists of Parameters with Spaces
- 5.11. Counting Arguments
- 5.12. Consuming Arguments
- 5.13. Getting Default Values
- 5.14. Setting Default Values
- 5.15. Using null as a Valid Default Value
- 5.16. Using More than Just a Constant String for Default
- 5.17. Giving an Error Message for Unset Parameters
- 5.18. Changing Pieces of a String
- 5.19. Getting the Absolute Value of a Number
- 5.20. Using bash for basename
- 5.21. Using bash for dirname
- 5.22. Using Alternate Values for Comma Separated Values
- 5.23. Using Array Variables
- 5.24. Converting Between Upper- and Lowercase
- 5.25. Converting to Camel Case
-
Shell Logic and Arithmetic
- 6.1. Doing Arithmetic in Your Shell Script
- 6.2. Branching on Conditions
- 6.3. Testing for File Characteristics
- 6.4. Testing for More than One Thing
- 6.5. Testing for String Characteristics
- 6.6. Testing for Equality
- 6.7. Testing with Pattern Matches
- 6.8. Testing with Regular Expressions
- 6.9. Changing Behavior with Redirections
- 6.10. Looping for a While
- 6.11. Looping with a read
- 6.12. Looping with a Count
- 6.13. Looping with Floating-Point Values
- 6.14. Branching Many Ways
- 6.15. Parsing Command-Line Arguments
- 6.16. Creating Simple Menus
- 6.17. Changing the Prompt on Simple Menus
- 6.18. Creating a Simple RPN Calculator
- 6.19. Creating a Command-Line Calculator
-
Intermediate Shell Tools I
- 7.1. Sifting Through Files for a String
- 7.2. Getting Just the Filename from a Search
- 7.3. Getting a Simple True/False from a Search
- 7.4. Searching for Text While Ignoring Case
- 7.5. Doing a Search in a Pipeline
- 7.6. Paring Down What the Search Finds
- 7.7. Searching with More Complex Patterns
- 7.8. Searching for an SSN
- 7.9. Grepping Compressed Files
- 7.10. Keeping Some Output, Discarding the Rest
- 7.11. Keeping Only a Portion of a Line of Output
- 7.12. Reversing the Words on Each Line
- 7.13. Summing a List of Numbers
- 7.14. Counting String Values with awk
- 7.15. Counting String Values with bash
- 7.16. Showing Data as a Quick and Easy Histogram
- 7.17. An Easy Histogram with bash
- 7.18. Showing a Paragraph of Text After a Found Phrase
-
Intermediate Shell Tools II
- 8.1. Sorting Your Output
- 8.2. Sorting Numbers
- 8.3. Sorting IP Addresses
- 8.4. Cutting Out Parts of Your Output
- 8.5. Removing Duplicate Lines
- 8.6. Compressing Files
- 8.7. Uncompressing Files
- 8.8. Checking a tar Archive for Unique Directories
- 8.9. Translating Characters
- 8.10. Converting Uppercase to Lowercase
- 8.11. Converting DOS Files to Linux Format
- 8.12. Removing Smart Quotes
- 8.13. Counting Lines, Words, or Characters in a File
- 8.14. Rewrapping Paragraphs
- 8.15. Doing More with less
-
Finding Files: find, locate, slocate
- 9.1. Finding All Your MP3 Files
- 9.2. Handling Filenames Containing Odd Characters
- 9.3. Speeding Up Operations on Found Files
- 9.4. Finding Files Across Symbolic Links
- 9.5. Finding Files Irrespective of Case
- 9.6. Finding Files by Date
- 9.7. Finding Files by Type
- 9.8. Finding Files by Size
- 9.9. Finding Files by Content
- 9.10. Finding Existing Files and Content Fast
- 9.11. Finding a File Using a List of Possible Locations
-
Additional Features for Scripting
- 10.1. “Daemon-izing” Your Script
- 10.2. Reusing Code with Includes and Sourcing
- 10.3. Using Configuration Files in a Script
- 10.4. Defining Functions
- 10.5. Using Functions: Parameters and Return Values
- 10.6. Trapping Interrupts
- 10.7. Redefining Commands with alias
- 10.8. Avoiding Aliases and Functions
- 10.9. Counting Elapsed Time
- 10.10. Writing Wrappers
-
Working with Dates and Times
- 11.1. Formatting Dates for Display
- 11.2. Supplying a Default Date
- 11.3. Automating Date Ranges
- 11.4. Converting Dates and Times to Epoch Seconds
- 11.5. Converting Epoch Seconds to Dates and Times
- 11.6. Getting Yesterday or Tomorrow with Perl
- 11.7. Figuring Out Date and Time Arithmetic
- 11.8. Handling Time Zones, Daylight Saving Time, and Leap Years
- 11.9. Using date and cron to Run a Script on the Nth Day
- 11.10. Logging with Dates
- End-User Tasks as Shell Scripts
-
Parsing and Similar Tasks
- 13.1. Parsing Arguments for Your Shell Script
- 13.2. Parsing Arguments with Your Own Error Messages
- 13.3. Parsing Some HTML
- 13.4. Parsing Output into an Array
- 13.5. Parsing Output with a Function Call
- 13.6. Parsing Text with a read Statement
- 13.7. Parsing with read into an Array
- 13.8. Reading an Entire File
- 13.9. Getting Your Plurals Right
- 13.10. Taking It One Character at a Time
- 13.11. Cleaning Up an SVN Source Tree
- 13.12. Setting Up a Database with MySQL
- 13.13. Isolating Specific Fields in Data
- 13.14. Updating Specific Fields in Datafiles
- 13.15. Trimming Whitespace
- 13.16. Compressing Whitespace
- 13.17. Processing Fixed-Length Records
- 13.18. Processing Files with No Line Breaks
- 13.19. Converting a Datafile to CSV
- 13.20. Parsing a CSV Datafile
-
Writing Secure Shell Scripts
- 14.1. Avoiding Common Security Problems
- 14.2. Avoiding Interpreter Spoofing
- 14.3. Setting a Secure $PATH
- 14.4. Clearing All Aliases
- 14.5. Clearing the Command Hash
- 14.6. Preventing Core Dumps
- 14.7. Setting a Secure $IFS
- 14.8. Setting a Secure umask
- 14.9. Finding World-Writable Directories in Your $PATH
- 14.10. Adding the Current Directory to the $PATH
- 14.11. Using Secure Temporary Files
- 14.12. Validating Input
- 14.13. Setting Permissions
- 14.14. Leaking Passwords into the Process List
- 14.15. Writing setuid or setgid Scripts
- 14.16. Restricting Guest Users
- 14.17. Using chroot Jails
- 14.18. Running as a Non-root User
- 14.19. Using sudo More Securely
- 14.20. Using Passwords in Scripts
- 14.21. Using SSH Without a Password
- 14.22. Restricting SSH Commands
- 14.23. Disconnecting Inactive Sessions
-
Advanced Scripting
- 15.1. Finding bash Portably for #!
- 15.2. Setting a POSIX $PATH
- 15.3. Developing Portable Shell Scripts
- 15.4. Testing Scripts Using Virtual Machines
- 15.5. Using for Loops Portably
- 15.6. Using echo Portably
- 15.7. Splitting Output Only When Necessary
- 15.8. Viewing Output in Hex
- 15.9. Using bash Net-Redirection
- 15.10. Finding My IP Address
- 15.11. Getting Input from Another Machine
- 15.12. Redirecting Output for the Life of a Script
- 15.13. Working Around “Argument list too long” Errors
- 15.14. Logging to syslog from Your Script
- 15.15. Using logger Correctly
- 15.16. Sending Email from Your Script
- 15.17. Automating a Process Using Phases
- 15.18. Doing Two Things at Once
- 15.19. Running an SSH command on multiple hosts
-
Configuring and Customizing bash
- 16.1. bash Startup Options
- 16.2. Customizing Your Prompt
- 16.3. A Prompt Before Your Program Runs
- 16.4. Changing Your $PATH Permanently
- 16.5. Changing Your $PATH Temporarily
- 16.6. Setting Your $CDPATH
- 16.7. When Programs Are Not Found
- 16.8. Shortening or Changing Command Names
- 16.9. Adjusting Shell Behavior and Environment
- 16.10. Adjusting readline Behavior Using .inputrc
- 16.11. Keeping a Private Stash of Utilities by Adding ~/bin
- 16.12. Using Secondary Prompts: $PS2, $PS3, $PS4
- 16.13. Synchronizing Shell History Between Sessions
- 16.14. Setting Shell History Options
- 16.15. Creating a Better cd Command
- 16.16. Creating and Changing Into a New Directory in One Step
- 16.17. Getting to the Bottom of Things
- 16.18. Adding New Features to bash Using Loadable Builtins
- 16.19. Improving Programmable Completion
- 16.20. Using Initialization Files Correctly
- 16.21. Creating Self-Contained, Portable rc Files
- 16.22. Getting Started with a Custom Configuration
-
Housekeeping and Administrative Tasks
- 17.1. Renaming Many Files
- 17.2. Using GNU Texinfo and info on Linux
- 17.3. Unzipping Many ZIP Files
- 17.4. Recovering Disconnected Sessions Using screen
- 17.5. Sharing a Single bash Session
- 17.6. Logging an Entire Session or Batch Job
- 17.7. Clearing the Screen When You Log Out
- 17.8. Capturing File Metadata for Recovery
- 17.9. Creating an Index of Many Files
- 17.10. Using diff and patch
- 17.11. Counting Differences in Files
- 17.12. Removing or Renaming Files Named with Special Characters
- 17.13. Prepending Data to a File
- 17.14. Editing a File in Place
- 17.15. Using sudo on a Group of Commands
- 17.16. Finding Lines That Appear in One File but Not in Another
- 17.17. Keeping the Most Recent N Objects
- 17.18. Writing to a Circular Log
- 17.19. Circular Backups
- 17.20. Grepping ps Output Without Also Getting the grep Process Itself
- 17.21. Finding Out Whether a Process Is Running
- 17.22. Adding a Prefix or Suffix to Output
- 17.23. Numbering Lines
- 17.24. Writing Sequences
- 17.25. Emulating the DOS Pause Command
- 17.26. Commifying Numbers
- Working Faster by Typing Less
-
Tips and Traps: Common Goofs for Novices
- 19.1. Forgetting to Set Execute Permissions
- 19.2. Fixing “No such file or directory” Errors
- 19.3. Forgetting That the Current Directory Is Not in the $PATH
- 19.4. Naming Your Script “test”
- 19.5. Expecting to Change Exported Variables
- 19.6. Forgetting Quotes Leads to “command not found” on Assignments
- 19.7. Forgetting that Pattern Matching Alphabetizes
- 19.8. Forgetting that Pipelines Make Subshells
- 19.9. Making Your Terminal Sane Again
- 19.10. Deleting Files Using an Empty Variable
- 19.11. Seeing Odd Behavior from printf
- 19.12. Testing bash Script Syntax
- 19.13. Debugging Scripts
- 19.14. Avoiding “command not found” When Using Functions
- 19.15. Confusing Shell Wildcards and Regular Expressions
-
Reference Lists
- bash Invocation
- Prompt String Customizations
- ANSI Color Escape Sequences
- Builtin Commands
- bash Reserved Words
- Builtin Shell Variables
- set Options
- shopt Options
- Test Operators
- I/O Redirection
- echo Options and Escape Sequences
- printf
- Date and Time String Formatting with strftime
- Pattern-Matching Characters
- extglob Extended Pattern-Matching Operators
- tr Escape Sequences
- readline Init File Syntax
- Table of ASCII Values
- Examples Included with bash
- Command-Line Processing
- Revision Control
- Building bash from Source
- Index
Product information
- Title: bash Cookbook, 2nd Edition
- Author(s):
- Release date: October 2017
- Publisher(s): O'Reilly Media, Inc.
- ISBN: 9781491975336
You might also like
book
Bash Cookbook
Create simple to advanced shell scripts and enhance your system functionality with effective recipes Key Features …
book
bash Cookbook
The key to mastering any Unix system, especially Linux and Mac OS X, is a thorough …
book
Linux Cookbook, 2nd Edition
This handy cookbook teaches new-to-intermediate Linux users the essential skills necessary to manage a Linux system, …
video
Bash Shell Scripting, 2nd Edition
8 Hours of Video Instruction Overview Bash is the best way for you to efficiently and …