book

Beginning Shell Scripting

Name: Beginning Shell Scripting
ISBN: 9780764583209

by John C. Welch, Eric Foster-Johnson, Micah Anderson

April 2005

Beginner

528 pages

12h 22m

English

Wrox

Read now

Unlock full access

Copyright
About the Authors
Credits
Introduction
Whom This Book Is ForWhat Does This Book Cover?How This Book Is StructuredWhat Do You Need to Use This Book?ConventionsSource CodeErratap2p.wrox.com
1. Introducing Shells
1.1. What Is a Shell?1.2. Why Use Shells?1.3. What Kind of Shells Are There?1.3.1. The Bourne Shell1.3.2. The C Shell1.3.3. The Korn Shell1.3.4. Bash, the Bourne Again Shell1.3.5. tcsh, the T C Shell1.3.6. Other Shells1.3.7. Graphical Shells1.3.8. The Official POSIX Shell1.3.9. Default Shells1.3.10. Choosing a Shell1.3.11. Changing Your Default Shell1.3.12. Running a Shell from Within Another Shell1.3.13. Finding Out More About the Shell1.4. How Do Shells Fit into a Graphical Environment?1.4.1. Running Shells on Linux1.4.1.1. Running the GNOME Shell Window1.4.1.2. Running the KDE Shell Window1.4.1.3. Active Select and Paste1.4.2. Running Shells on Mac OS X1.4.3. Running Shells on Unix Systems1.4.4. Running Shells on Windows—Isn't command.com Enough?1.4.4.1. Installing Cygwin on Windows1.4.4.2. Installing the Korn Shell on Windows1.4.5. Running Shells on PDAs and Other Systems1.5. Entering Commands1.5.1. Determining Which Shell You Are Running1.5.2. Command-Line Options1.5.2.1. Combining Options1.5.2.2. Strange Options1.6. Command Editing1.6.1. Command Substitution1.6.1.1. Repeating Previous Commands1.6.1.2. Repeating Parts of Previous Commands1.6.1.3. Using the Up and Down Arrows1.6.2. Viewing the Command History1.6.3. Calling Up an Editor1.6.4. Using File-Name Completion1.7. Working with Wildcards1.7.1. The * Wildcard1.7.2. The ? Wildcard1.8. Running Commands in the Background1.9. Summary
2. Introducing Shell Scripts
2.1. What Are Shell Scripts?2.2. Alternatives to Shells: Other Scripting Languages2.2.1. Perl2.2.2. Python2.2.3. Tcl2.2.4. MS-DOS Batch Files2.3. Tools to Edit Shell Scripts2.3.1. Using Legacy Editors2.3.1.1. Delving into emacs2.3.1.2. Learning vi Modes2.3.2. Moving to Graphical Text Editors2.3.2.1. Cross-Platform Editors2.3.2.2. Mac OS X Editors2.3.2.3. Linux Editors2.3.2.4. Unix Editors2.3.2.5. Microsoft Windows Editors2.4. Writing Scripts2.4.1. Remembering Commands So You Don't Have To2.4.2. Outputting Text2.4.3. Variables2.4.3.1. Variables and the C Shell2.4.3.2. Combining Variables2.4.3.3. Building Up Commands with Variables2.4.3.4. De-assigning Variables2.4.4. Gathering Input2.4.5. Commenting Your Scripts2.4.6. Continuing Lines2.5. Summary2.6. Exercises
3. Controlling How Scripts Run
3.1. Referencing Variables3.2. Looping and Iteration3.2.1. Looping over Files3.2.2. Looping for a Fixed Number of Iterations3.2.3. Looping Like a C Program—the bash Shell3.2.4. Looping in the C Shell3.2.5. Nested Loops3.3. Checking Conditions with if3.3.1. Or Else, What?3.3.2. What Is Truth?3.3.3. Redirecting Output3.3.4. Using elif (Short for else if)3.4. Nesting if Statements3.5. Testing with the test Command3.5.1. Comparing Numbers3.5.2. Comparing Text Strings3.5.3. Testing Files3.5.4. Using the Binary and Not Operators3.5.5. Creating Shorthand Tests with [3.6. Making Complex Decisions with case3.6.1. Handling Problematic Input3.6.2. Using case with the C Shell3.7. Looping While a Condition Is True3.8. Looping Until a Condition Is True3.9. Summary3.10. Exercises
4. Interacting with the Environment
4.1. Examining Environment Variables4.1.1. Reading Environment Variables4.1.1.1. Handling Local and Environment Variables4.1.1.2. Listing the Environment with the C Shell4.1.1.3. Testing the Environment4.1.2. Setting Environment Variables4.1.2.1. Setting Environment Variables in the C Shell4.1.2.2. Reading Values into the Current Environment4.2. Customizing Your Account4.2.1. How the Bourne Shell Starts Up4.2.2. How the Korn Shell Starts Up4.2.3. How the C Shell Starts Up4.2.4. How the T C Shell Starts Up4.2.5. How Bash Starts Up4.3. Handling Command-Line Arguments4.3.1. Reading Command-Line Arguments with the Bourne Shell4.3.2. Reading Command-Line Arguments with the C Shell4.4. Making Scripts Executable4.4.1. Marking Files Executable4.4.2. Setting the #! Magic Line4.5. Summary4.6. Exercises
5. Scripting with Files
5.1. Combining Files into Archives5.2. Working with File Modes5.3. Testing Files with the test Command5.4. Dealing with Mac OS X Files5.4.1. The Legacies of NeXT5.4.2. Mobile File Systems and Mac OS X5.4.2.1. Target Disk Mode5.4.2.2. Mobile File Systems and Disk Images5.4.3. Naming Issues5.5. HFS+ Versus UFS: The Mac OS X Holy War5.5.1. The Terror of the Resource Fork5.6. Working with Here Files and Interactive Programs5.6.1. Displaying Messages with Here Files5.6.2. Customizing Here Files5.6.3. Driving Interactive Programs with Here Files5.6.4. Turning Off Variable Substitution5.7. Summary5.8. Exercises
6. Processing Text with sed
6.1. Introducing sed6.2. sed Versions6.3. Installing sed6.3.1. Bootstrap Installation6.3.2. Configuring and Installing sed6.4. How sed Works6.4.1. Invoking sed6.4.2. Editing Commands6.4.3. Invoking sed with the -e Flag6.4.4. The -n, --quiet, and --silent Flags6.4.5. sed Errors6.5. Selecting Lines to Operate On6.5.1. Address Ranges6.5.2. Address Negation6.5.3. Address Steps6.6. Substitution6.6.1. Substitution Flags6.6.2. Using an Alternative String Separator6.6.3. Address Substitution6.7. Advanced sed Invocation6.7.1. The comment Command6.7.2. The insert, append, and change Commands6.8. Advanced Addressing6.8.1. Regular Expression Addresses6.8.2. Character Class Keywords6.8.3. Regular Expression Address Ranges6.8.4. Combining Line Addresses with regexps6.9. Advanced Substitution6.9.1. Referencing Matched regexps with &6.9.2. Back References6.10. Hold Space6.11. More sed Resources6.11.1. Common One-Line sed Scripts6.11.2. Common sed Commands6.11.3. Less Common sed Commands6.11.4. GNU sed-Specific sed Extensions6.12. Summary6.13. Exercises

7. Processing Text with awk
7.1. What Is awk (Gawk/Mawk/Nawk/Oawk)?7.1.1. Gawk, the GNU awk7.1.2. What Version Do I Have Installed?7.1.3. Installing gawk7.2. How awk Works7.3. Invoking awk7.4. The print Command7.4.1. Using Field Separators7.4.2. Using the printf Command7.4.3. Using printf Format Modifiers7.4.4. Using the sprintf Command7.5. Using Variables in awk7.5.1. User-Defined Variables7.5.2. Built-in Variables7.5.2.1. The FS Variable7.5.2.2. FS Regular Expressions7.5.2.3. The NR Variable7.6. Control Statements7.6.1. if Statements7.6.2. Comparison Operators7.6.3. Arithmetic Functions7.6.4. Output Redirection7.6.5. While Loops7.6.6. For Loops7.7. Functions7.8. Resources7.9. Summary7.10. Exercises
8. Creating Command Pipelines
8.1. Working with Standard Input and Output8.1.1. Redirecting Standard Input and Output8.1.2. Redirecting Standard Error8.1.3. Redirecting Both Standard Output and Standard Error8.1.4. Appending to Files8.1.5. Truncating Files8.1.6. Sending Output to Nowhere Fast8.2. Piping Commands8.2.1. Piping with Unix Commands8.2.2. Creating Pipelines8.2.3. Using tee to Send the Output to More Than One Process8.3. Summary8.4. Exercises
9. Controlling Processes
9.1. Exploring Processes9.1.1. Checking Process IDs9.1.2. Reading the /proc File System9.1.3. Killing Processes9.2. Launching Processes9.2.1. Running Commands in the Foreground9.2.2. Running Commands in the Background9.2.3. Running Commands in Subshells9.2.4. Running Commands with the exec Command9.3. Capturing the Output of Processes9.3.1. Using Backticks for Command Substitution9.3.1.1. Using Parentheses in Place of Backticks9.3.1.2. Setting Variables from Commands9.3.1.3. Performing Math with Commands9.3.1.4. Reading Files into Variables9.3.2. Capturing Program Return Codes9.4. Summary9.5. Exercises
10. Shell Scripting Functions
10.1. Defining Functions10.1.1. Adding Names to Blocks of Code10.1.2. Function Declaration Errors10.2. Using Functions10.2.1. Declaring before Use10.2.2. Function Files10.2.3. Common Usage Errors10.2.4. Undeclaring Functions10.3. Using Arguments with Functions10.4. Using Return Codes with Functions10.5. Variable Scope: Think Globally, Act Locally10.6. Understanding Recursion10.7. Summary10.8. Exercises
11. Debugging Shell Scripts
11.1. Deciphering Error Messages11.1.1. Finding Missing Syntax11.1.2. Finding Syntax Errors11.2. Tracking Down Problems with Debugging Techniques11.2.1. Look Backward11.2.2. Look for Obvious Mistakes11.2.3. Look for Weird Things11.2.4. Look for Hidden Assumptions11.2.5. Divide and Conquer11.2.6. Break the Script into Pieces11.2.7. Trace the Execution11.2.8. Get Another Set of Eyes11.3. Running Scripts in Debugging Mode11.3.1. Disabling the Shell11.3.2. Displaying the Script Commands11.3.3. Combining the -n and -v Options11.3.4. Tracing Script Execution11.4. Avoiding Errors with Good Scripting11.4.1. Tidy Up Your Scripts11.4.2. Comment Your Scripts11.4.3. Create Informative Error Messages11.4.4. Simplify Yourself Out of the Box11.4.5. Test, Test, and Test Again11.5. Summary11.6. Exercises
12. Graphing Data with MRTG
12.1. Working with MRTG12.2. Monitoring Other Data with MRTG12.3. Installing MRTG12.4. Writing Scripts for MRTG12.5. Configuring MRTG12.5.1. Configuring the Global Values12.5.2. Configuring MRTG Targets for Your Scripts12.5.3. Customizing MRTG Output12.5.3.1. Configuring Target HTML Outputs12.5.3.2. Configuring Graphs12.6. Running MRTG12.6.1. Viewing Your First MRTG Output12.6.2. Configuring cron12.6.3. Maximizing MRTG Performance12.7. Monitoring Your Computer with MRTG12.7.1. Graphing Memory Usage12.7.2. Graphing CPU Usage12.7.3. Graphing Disk Usage12.8. Monitoring Networks with MRTG12.9. Monitoring Applications with MRTG12.10. Summary12.11. Exercises
13. Scripting for Administrators
13.1. Why Write Scripts?13.2. Scripting Complicated Commands13.3. Troubleshooting Your Systems13.4. Removing Minor Annoyances13.5. Cleaning Up Data13.6. Automating Daily Work13.7. Summary13.8. Exercises
14. Scripting for the Desktop
14.1. Scripting Office Applications14.1.1. Scripting the OpenOffice.org Suite14.1.1.1. Scripting OpenOffice with the Shell14.1.1.2. Scripting OpenOffice.org in Basic14.1.2. Scripting AbiWord14.1.3. Scripting NEdit14.2. Scripting for the Desktop on Mac OS X14.2.1. Open Scripting Architecture14.2.2. AppleScript Basics14.2.2.1. AppleScript Dictionaries14.2.2.2. AppleScript's Shell Support14.2.2.2.1. Going from Shell to AppleScript14.2.2.2.2. Going from AppleScript to Shell14.2.3. Mac OS X Terminal Window Settings14.3. Scripting Multimedia14.3.1. Scripting the XMMS Music Player14.3.2. Scripting Rhythmbox14.3.3. Scripting the Totem Movie Player14.4. Scripting Other Desktop Applications14.5. Where to Go from Here14.6. Summary14.7. Exercises
A. Answers to Exercises
A.1.A.1.1.A.1.1.1. Chapter 2A.1.1.2. Chapter 3A.1.1.3. Chapter 4A.1.1.4. Chapter 5A.1.1.5. Chapter 6A.1.1.6. Chapter 7A.1.1.7. Chapter 8A.1.1.8. Chapter 9A.1.1.9. Chapter 10A.1.1.10. Chapter 11A.1.1.11. Chapter 12A.1.1.12. Chapter 13A.1.1.13. Chapter 14
B. Useful Commands
B.1. Navigating the SystemB.1.1. exitB.1.1.1. DescriptionB.1.1.2. ExampleB.1.1.3. OptionsB.1.2. fileB.1.2.1. DescriptionB.1.2.2. ExamplesB.1.2.3. OptionsB.1.3. killB.1.3.1. DescriptionB.1.3.2. ExamplesB.1.3.3. OptionsB.1.4. manB.1.4.1. DescriptionB.1.4.2. ExampleB.1.4.3. OptionsB.1.5. nohupB.1.5.1. DescriptionB.1.5.2. ExampleB.1.5.3. OptionsB.1.6. printenvB.1.6.1. DescriptionB.1.6.2. ExampleB.1.6.3. OptionsB.1.7. psB.1.7.1. DescriptionB.1.7.2. ExamplesB.1.7.3. OptionsB.1.8. sleepB.1.8.1. DescriptionB.1.8.2. ExamplesB.1.8.3. OptionsB.1.9. typeB.1.9.1. DescriptionB.1.9.2. ExamplesB.1.9.3. OptionsB.1.10. unameB.1.10.1. DescriptionB.1.10.2. ExamplesB.1.10.3. OptionsB.1.11. whoB.1.11.1. DescriptionB.1.11.2. ExamplesB.1.11.3. OptionsB.1.12. whoamiB.1.12.1. DescriptionB.1.12.2. ExamplesB.1.12.3. OptionsB.2. Working with Files and DirectoriesB.2.1. basenameB.2.1.1. DescriptionB.2.1.2. ExamplesB.2.1.3. OptionsB.2.2. catB.2.2.1. DescriptionB.2.2.2. ExamplesB.2.2.3. OptionsB.2.3. chmodB.2.3.1. DescriptionB.2.3.2. ExamplesB.2.3.3. OptionsB.2.4. chownB.2.4.1. DescriptionB.2.4.2. ExampleB.2.4.3. OptionsB.2.5. cpB.2.5.1. DescriptionB.2.5.2. ExamplesB.2.5.3. OptionsB.2.6. dfB.2.6.1. DescriptionB.2.6.2. ExamplesB.2.6.3. OptionsB.2.7. duB.2.7.1. DescriptionB.2.7.2. ExamplesB.2.7.3. OptionsB.2.8. findB.2.8.1. DescriptionB.2.8.2. ExamplesB.2.8.3. OptionsB.2.9. grepB.2.9.1. DescriptionB.2.9.2. ExamplesB.2.9.3. OptionsB.2.10. headB.2.10.1. DescriptionB.2.10.2. ExamplesB.2.10.3. OptionsB.2.11. lsB.2.11.1. DescriptionB.2.11.2. ExamplesB.2.11.3. OptionsB.2.12. mkdirB.2.12.1. DescriptionB.2.12.2. ExamplesB.2.12.3. OptionsB.2.13. mvB.2.13.1. DescriptionB.2.13.2. ExamplesB.2.13.3. OptionsB.2.14. rmB.2.14.1. DescriptionB.2.14.2. ExamplesB.2.14.3. OptionsB.2.15. rmdirB.2.15.1. DescriptionB.2.15.2. ExamplesB.2.15.3. OptionB.2.16. tailB.2.16.1. DescriptionB.2.16.2. ExamplesB.2.16.3. OptionsB.2.17. touchB.2.17.1. DescriptionB.2.17.2. ExamplesB.2.17.3. OptionsB.3. Manipulating TextB.3.1. awkB.3.1.1. DescriptionB.3.1.2. ExampleB.3.1.3. OptionsB.3.2. echoB.3.2.1. DescriptionB.3.2.2. ExamplesB.3.2.3. OptionsB.4. Transforming DataB.4.1. cutB.4.1.1. DescriptionB.4.1.2. ExampleB.4.1.3. OptionsB.4.2. sedB.4.2.1. DescriptionB.4.2.2. ExampleB.4.2.3. OptionsB.4.3. sortB.4.3.1. DescriptionB.4.3.2. ExamplesB.4.3.3. OptionsB.4.4. stringsB.4.4.1. DescriptionB.4.4.2. ExampleB.4.4.3. OptionsB.4.5. trB.4.5.1. DescriptionB.4.5.2. ExampleB.4.5.3. OptionsB.5. Resolving ExpressionsB.5.1. bcB.5.1.1. DescriptionB.5.1.2. ExampleB.5.1.3. OptionsB.5.2. exprB.5.2.1. DescriptionB.5.2.2. ExamplesB.5.2.3. Options

Content preview from Beginning Shell Scripting

Chapter 5. Scripting with Files

Shells were originally designed to work with files. That's because a huge portion of what you need to do on a Unix system relates to files. Unix, for example, stores most configuration settings in files, and usually text files at that. This is different from Windows, for example, which stores many configuration settings in the Registry.

This means that many simple commands can manipulate the configuration of many applications, especially server applications. All of these commands work with files. In addition to the normal operations of reading from files, writing to files, and so on, shell scripts interact with files in a number of ways, including:

Combining files into archives.
Testing files with the test command.
Dealing with files on Mac OS X, especially for mobile users.
Outputting files stored within the scripts. These are called here files. You can use here files to drive interactive programs.

Shells provide a number of built-in commands to work with files, but for the most part, your scripts will call non–built-in commands for most work with files. That's because Unix and Unix-like systems offer a rich set of commands for working with files.

You can also use input redirection to read the contents of files. Output redirection allows you to send the output of a command to a file. And command pipelines allow you to feed the results of one command into another. These topics are covered in Chapter 8.

The following sections show all sorts of additional ways ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9780764583209Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Beginning Shell Scripting

by John C. Welch, Eric Foster-Johnson, Micah Anderson

Chapter 5. Scripting with Files

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.