book

UNIX® Shells by Example Fourth Edition

Name: UNIX® Shells by Example Fourth Edition
Author: Ellie Quigley
ISBN: 013147572X

by Ellie Quigley

September 2004

Beginner

1200 pages

22h 47m

English

Pearson

Read now

Unlock full access

Copyright
Dedication
Preface
ACKNOWLEDGMENTS
1. Introduction to UNIX/Linux Shells
1.1. What Is UNIX? What Is Linux? A Little History1.1.1. A Little Bit About UNIX1.1.2. Why Linux?The Free Software Foundation1.2. Definition and Function of a Shell1.2.1. The UNIX Shells1.2.2. The Linux ShellsWhat Is POSIX?1.3. History of the Shell1.3.1. Uses of the Shell1.3.2. Responsibilities of the Shell1.4. System Startup and the Login Shell1.4.1. Parsing the Command Line1.4.2. Types of Commands1.5. Processes and the Shell1.5.1. What Processes Are Running?The ps CommandThe pstree/ptree Command1.5.2. What Are System Calls?1.5.3. Creating ProcessesThe fork System CallThe wait System CallThe exec System CallThe exit System CallKilling Processes1.6. The Environment and Inheritance1.6.1. Ownership and Permissions1.6.2. The File Creation Mask1.6.3. Changing Permissions and OwnershipThe chmod CommandThe chown Command1.6.4. The Working Directory1.6.5. Variables1.6.6. Redirection and PipesFile DescriptorsRedirectionPipes1.6.7. The Shell and Signals1.7. Executing Commands from Scripts
2. Shell Programming QuickStart
2.1. Taking a Peek at Shell Scripts2.2. Sample Scripts: Comparing the Major ShellsBefore Getting StartedThe Purpose2.3. The C and TC Shell Syntax and Constructs2.3.1. The C/TC Shell Script2.4. The Bourne Shell Syntax and Constructs2.4.1. The Bourne Shell Script2.5. The Korn Shell Constructs2.5.1. The Korn Shell Script2.6. The Bash Shell Constructs2.6.1. The Bash Shell Script
3. Regular Expressions and Pattern Matching
3.1. Regular Expressions3.1.1. Definition and Example3.1.2. Regular Expression Metacharacters3.2. Combining Regular Expression Metacharacters3.2.1. More Regular Expression Metacharacters
4. The grep Family
4.1. The grep Command4.1.1. The Meaning of grep4.1.2. How grep Works4.1.3. Metacharacters4.1.4. grep and Exit Status4.2. grep Examples with Regular Expressions4.3. grep with Options4.4. grep with Pipes4.4.1. grep Review4.5. egrep (Extended grep)4.5.1. egrep Examples4.5.2. egrep Review4.6. fgrep (Fixed grep or Fast grep)4.7. Linux and GNU grep4.7.1. Basic and Extended Regular ExpressionsThe POSIX Class4.8. GNU Basic grep (grep –G) with Regular Expressions4.9. grep –E or egrep (GNU Extended grep)4.9.1. grep –E and egrep Examples4.9.2. Anomalies with Regular and Extended Variants of grep4.10. Fixed grep (grep –F and fgrep)4.11. Recursive grep (rgrep, grep –R)4.12. GNU grep with Options4.13. grep with Options (UNIX and GNU)4.13.1. GNU grep Options ExamplesLAB 1: grep EXERCISE
5. sed, the Streamlined Editor
5.1. What Is sed ?5.2. Versions of sed5.3. How Does sed Work?5.4. Regular Expressions5.5. Addressing5.6. Commands and Options5.6.1. How to Modify a File with sed5.6.2. GNU sed Options5.7. Error Messages and Exit Status5.8. Metacharacters5.9. sed Examples5.9.1. Printing: The p Command5.9.2. Deleting: The d Command5.9.3. Substitution: The s Command5.9.4. Range of Selected Lines: The Comma5.9.5. Multiple Edits: The e Command5.9.6. Reading from Files: The r Command5.9.7. Writing to Files: The w Command5.9.8. Appending: The a Command5.9.9. Inserting: The i Command5.9.10. Changing: The c Command5.9.11. Next: The n Command5.9.12. Transform: The y Command5.9.13. Quit: The q Command5.9.14. Holding and Getting: The h and g Commands5.9.15. Holding and Exchanging: The h and x Commands5.10. sed Scripting5.10.1. sed Script Examples5.10.2. sed ReviewLAB 2: sed EXERCISE
6. The awk Utility
6.1. What's awk? What's nawk? What's gawk?6.1.1. What Does awk Stand For?6.1.2. Which awk?6.2. awk's Format6.2.1. Input from Files6.2.2. Input from Commands6.3. How awk Works6.4. Formatting Output6.4.1. The print FunctionEscape Sequences6.4.2. The OFMT Variable6.4.3. The printf Function6.5. awk Commands from Within a File6.6. Records and Fields6.6.1. RecordsThe Record SeparatorThe $0 VariableThe NR Variable6.6.2. Fields6.6.3. Field SeparatorsThe Input Field SeparatorChanging the Field Separator at the Command LineUsing More Than One Field SeparatorThe Output Field Separator6.7. Patterns and Actions6.7.1. Patterns6.7.2. Actions6.8. Regular Expressions6.8.1. Matching on an Entire Line6.8.2. The match OperatorThe POSIX Character Class6.9. awk Commands in a Script File6.10. Review6.10.1. Simple Pattern Matching6.10.2. Simple Actions6.10.3. Regular Expressions in Pattern and Action Combinations6.10.4. Input Field Separators6.10.5. awk ScriptingLAB 3: awk EXERCISE6.11. Comparison Expressions6.11.1. Relational and Equality Operators6.11.2. Conditional Expressions6.11.3. Computation6.11.4. Logical Operators and Compound Patterns6.11.5. Range Patterns6.11.6. A Data Validation Program6.12. Review6.12.1. Equality Testing6.12.2. Relational Operators6.12.3. Logical Operators (&&, ||)6.12.4. Logical NOT Operator (!)6.12.5. Arithmetic Operators6.12.6. Range Operator6.12.7. Conditional Operator6.12.8. Assignment OperatorsLAB 4: awk EXERCISE6.13. Variables6.13.1. Numeric and String ConstantsInitialization and Type Coercion6.13.2. User-Defined VariablesIncrement and Decrement OperatorsUser-Defined Variables at the Command LineThe –v Option (nawk)Field VariablesBuilt-In Variables6.13.3. BEGIN Patterns6.13.4. END Patterns6.14. Redirection and Pipes6.14.1. Output Redirection6.14.2. Input Redirection (getline)The getline Function6.15. Pipes6.15.1. Closing Files and PipesThe system Function6.16. Review6.16.1. Increment and Decrement Operators6.16.2. Built-In Variables6.16.3. BEGIN Patterns6.16.4. END Patterns6.16.5. awk Script with BEGIN and END6.16.6. The printf Function6.16.7. Redirection and Pipes6.16.8. Opening and Closing a PipeLAB 5: nawk EXERCISE6.17. Conditional Statements6.17.1. if Statements6.17.2. if/else Statements6.17.3. if/else and else if Statements6.18. Loops6.18.1. while Loop6.18.2. for Loop6.18.3. Loop Controlbreak and continue Statements6.19. Program Control Statements6.19.1. next Statement6.19.2. exit Statement6.20. Arrays6.20.1. Subscripts for Associative ArraysUsing Variables As Array IndexesThe Special-for LoopUsing Strings As Array SubscriptsUsing Field Values As Array SubscriptsArrays and the split FunctionThe delete FunctionMultidimensional Arrays (nawk)6.20.2. Processing Command Arguments (nawk)ARGVARGC6.21. awk Built-In Functions6.21.1. String FunctionsThe sub and gsub FunctionsThe index FunctionThe length FunctionThe substr FunctionThe match FunctionThe split FunctionThe sprintf Function6.22. Built-In Arithmetic Functions6.22.1. Integer Function6.22.2. Random Number GeneratorThe rand FunctionThe srand Function6.23. User-Defined Functions (nawk)6.24. ReviewLAB 6: nawk EXERCISE6.25. Odds and Ends6.25.1. Fixed FieldsEmpty FieldsNumbers with $, Commas, or Other Characters6.25.2. Multiline Records6.25.3. Generating Form Letters6.25.4. Interaction with the ShellLAB 7: nawk EXERCISE6.26. awk Built-In Functions6.26.1. String FunctionsThe toupper and tolower Functions (gawk only)6.26.2. Time Functions with gawkThe systime FunctionThe strftime Function6.26.3. Command-Line Arguments6.26.4. Reading Input (getline)6.26.5. Control Functions6.26.6. User-Defined Functions6.26.7. awk/gawk Command-Line Options
7. The Interactiven Bourne Shell
7.1. Introduction7.2. The Environment7.2.1. The Initialization FilesThe /etc/profile FileThe .profile File7.2.2. The PromptsThe Primary PromptThe Secondary Prompt7.2.3. The Search Path7.2.4. The hash Command7.2.5. The dot Command7.3. The Command Line7.3.1. The Exit Status7.3.2. Multiple Commands at the Command LineGrouping Commands7.3.3. Conditional Execution of Commands7.3.4. Commands in the Background7.4. Shell Metacharacters (Wildcards)7.5. Filename Substitution7.5.1. The Asterisk7.5.2. The Question Mark7.5.3. The Square Brackets7.5.4. Escaping Metacharacters7.6. Variables7.6.1. Local VariablesSetting Local VariablesThe Scope of Local VariablesSetting Read-Only Variables7.6.2. Environment VariablesSetting Environment Variables7.6.3. Listing Set Variables7.6.4. Unsetting Variables7.6.5. Printing the Values of Variables: The echo Command7.6.6. Variable Expansion Modifiers7.6.7. Positional Parameters7.6.8. Other Special Variables7.7. Quoting7.7.1. The Backslash7.7.2. Single Quotes7.7.3. Double Quotes7.8. Command Substitution7.9. An Introduction to Functions7.9.1. Defining Functions7.9.2. Listing and Unsetting Functions7.10. Standard I/O and Redirection7.10.1. The exec Command and Redirection7.11. Pipes7.12. The here document and Input7.12.1. Now What?
8. Programming the Bourne Shell
8.1. Introduction8.1.1. The Steps in Creating a Shell ScriptThe First LineCommentsExecutable Statements and Bourne Shell ConstructsMaking the Script ExecutableA Scripting Session8.2. Reading User Input8.3. Arithmetic8.3.1. Integer Arithmetic and the expr Command8.3.2. Floating-Point Arithmetic8.4. Positional Parameters and Command-Line Arguments8.4.1. The set Command and Positional Parameters8.4.2. How $* and $@ Differ8.5. Conditional Constructs and Flow Control8.5.1. Testing Exit Status: The test Command8.5.2. The if Command8.5.3. The exit Command and the ? Variable8.5.4. Checking for Null Values8.5.5. The if/else Command8.5.6. The if/elif/else Command8.5.7. File Testing8.5.8. The null Command8.5.9. The case Command8.5.10. Creating Menus with the here document and case Command8.6. Looping Commands8.6.1. The for Command8.6.2. The $* and $@ Variables in Wordlists8.6.3. The while Command8.6.4. The until Command8.6.5. Looping Control CommandsThe shift CommandThe break CommandThe continue Command8.6.6. Nested Loops and Loop Control8.6.7. I/O Redirection and SubshellsRedirecting the Output of a Loop to a FilePiping the Output of a Loop to a UNIX Command8.6.8. Running Loops in the Background8.6.9. The exec Command and Loops8.6.10. IFS and Loops8.7. Functions8.7.1. Unsetting Functions8.7.2. Function Arguments and the Return Value8.7.3. Functions and the dot CommandStoring Functions8.8. Trapping Signals8.8.1. Resetting Signals8.8.2. Ignoring Signals8.8.3. Listing Traps8.8.4. Traps in Functions8.8.5. Debugging8.9. The Command Line8.9.1. Processing Command-Line Options with getoptsgetopts Scripts8.9.2. The eval Command and Parsing the Command Line8.10. Shell Invocation Options8.10.1. The set Command and Options8.10.2. Shell Built-In CommandsLAB 8: BOURNE SHELL—GETTING STARTEDLAB 9: METACHARACTERSLAB 10: REDIRECTIONLAB 11: FIRST SCRIPTLAB 12: COMMAND-LINE ARGUMENTSLAB 13: GETTING USER INPUTLAB 14: CONDITIONAL STATEMENTSLAB 15: CONDITIONALS AND FILE TESTINGLAB 16: THE case STATEMENTLAB 17: LOOPSLAB 18: FUNCTIONS

9. The Interactive C and TC Shells
9.1. Introduction9.1.1. C/TC Shell Startup9.2. The Environment9.2.1. Initialization FilesThe .cshrc and .tcshrc FilesThe .login File9.2.2. The Search Path9.2.3. The rehash Command9.2.4. The hashstat Command9.2.5. The source Command9.2.6. The Shell PromptsThe Primary Prompt.The Secondary Prompt.9.3. The C/TC Shell Command Line9.3.1. The Exit Status9.3.2. Command Grouping9.3.3. Conditional Execution of Commands9.3.4. Commands in the Background9.3.5. Command-Line HistorySetting HistorySaving HistoryDisplaying HistoryRe-executing Commands9.4. Aliases9.4.1. Listing Aliases9.4.2. Creating Aliases9.4.3. Deleting Aliases9.4.4. Alias Loop9.5. Manipulating the Directory Stack9.5.1. The pushd and popd Commands9.6. Job Control9.6.1. The Ampersand and Background Jobs9.6.2. The Suspend Key Sequence and Background Jobs9.6.3. The jobs Command9.6.4. The Foreground and Background Commands9.7. Shell Metacharacters9.8. Filename Substitution9.8.1. The Asterisk9.8.2. The Question Mark9.8.3. The Square Brackets9.8.4. The Curly Braces9.8.5. Escaping Metacharacters9.8.6. Tilde Expansion9.8.7. Filename Completion: The filec Variable9.8.8. Turning Off Metacharacters with noglob9.9. Redirection and Pipes9.9.1. Redirecting Input9.9.2. The here document9.9.3. Redirecting Output9.9.4. Appending Output to an Existing File9.9.5. Redirecting Output and Error9.9.6. Separating Output and Errors9.9.7. The noclobber Variable9.10. Variables9.10.1. Curly Braces9.10.2. Local VariablesSetting Local VariablesThe set CommandRead-Only Variables (tcsh)Built-In Local Variables9.10.3. Environment VariablesPrinting Environment Variables9.10.4. ArraysThe shift Command and ArraysCreating an Array from a String9.10.5. Special VariablesPathname Variable Modifiers9.11. Command Substitution9.11.1. Wordlists and Command Substitution9.12. Quoting9.12.1. The Backslash9.12.2. Single Quotes9.12.3. Double Quotes9.12.4. The Quoting GameQuoting with the :q ModifierQuoting with the :x Modifier9.13. New Features of the Interactive TC Shell9.13.1. Versions of tcsh9.13.2. The Shell PromptsThe Primary Prompt.The Secondary Prompt.9.14. The TC Shell Command Line9.14.1. The Command Line and Exit StatusExit Status and the printexitvalue Variable9.14.2. TC Shell Command-Line HistoryThe history VariableSaving History and the savehist VariableDisplaying HistoryAccessing Commands from the History File9.14.3. The Built-In Command-Line EditorsThe bindkey Built-In CommandThe vi Built-In EditorThe emacs Built-In EditorBinding Keys9.15. TC Shell Command, Filename, and Variable Completion9.15.1. The autolist Variable9.15.2. The fignore Variable9.15.3. The complete Shell Variable9.15.4. Programming CompletionsTypes of Completions9.16. TC Shell Spelling Correction9.17. TC Shell Aliases9.17.1. Listing Aliases9.17.2. Creating Aliases9.17.3. Deleting Aliases9.17.4. Alias Loop9.17.5. Special tcsh Aliases9.18. TC Shell Job Control9.18.1. The jobs Command and the listjobs Variable9.18.2. Foreground and Background Commands9.18.3. Scheduling Jobs9.19. Printing the Values of Variables in the TC Shell9.19.1. The echo Command9.19.2. The printf Command9.19.3. Curly Braces and Variables9.19.4. Uppercase and Lowercase Modifiers9.20. TC Shell Built-In Commands9.20.1. Special Built-In T/TC Shell Variables9.20.2. TC Shell Command-Line SwitchesLAB 19: THE TC SHELL—GETTING STARTEDLAB 20: HISTORYLAB 21: SHELL METACHARACTERSLAB 22: REDIRECTIONLAB 23: VARIABLES AND ARRAYS
10. Programming the C and TC Shells
10.1. Introduction10.1.1. The Steps in Creating a Shell ScriptThe First LineCommentsMaking the Script ExecutableAn Example Scripting Session10.2. Reading User Input10.2.1. The $< Variable10.2.2. Creating a Wordlist from the Input String10.3. Arithmetic10.3.1. Arithmetic Operators10.3.2. Floating-Point Arithmetic10.4. Debugging Scripts10.5. Command-Line Arguments10.5.1. Positional Parameters and argv10.6. Conditional Constructs and Flow Control10.6.1. Testing Expressions10.6.2. Precedence and Associativity10.6.3. The if Statement10.6.4. Testing and Unset or Null Variables10.6.5. The if/else Statement10.6.6. Logical Expressions10.6.7. The if Statement and a Single Command10.6.8. The if/else if Statement10.6.9. Exit Status and the Status Variable10.6.10. Exiting from a Shell Script10.6.11. Using an Alias to Create an Error Message10.6.12. Using the Status Variable in a Script10.6.13. Evaluating Commands Within Conditionals10.6.14. The goto10.6.15. File Testing with the C Shell10.6.16. The test Command and File Testing10.6.17. Nesting Conditionals10.6.18. File Testing with the TC Shell10.6.19. The filetest Built-In (tcsh)10.6.20. Additional TC Shell File-Testing Operators10.6.21. The switch CommandNesting Switches10.6.22. The here document and Menus10.7. Looping Commands10.7.1. The foreach Loop10.7.2. The while Loop10.7.3. The repeat Command10.7.4. Looping Control CommandsThe shift CommandThe break CommandNested Loops and the repeat CommandThe continue Command10.8. Interrupt Handling10.9. setuid Scripts10.10. Storing Scripts10.11. Built-In CommandsLAB 24: C/TC SHELLS—GETTING STARTEDLAB 25: SHELL METACHARACTERSLAB 26: REDIRECTIONLAB 27: FIRST SCRIPTLAB 28: GETTING USER INPUTLAB 29: COMMAND-LINE ARGUMENTSLAB 30: CONDITIONALS AND FILE TESTINGLAB 31: THE switch STATEMENTLAB 32: LOOPS
11. The Interactive Korn Shell
11.1. Introduction11.1.1. Startup11.2. The Environment11.2.1. The Initialization FilesThe /etc/profile FileThe .profile FileThe ENV FileThe set –o Options11.2.2. The PromptsThe Primary PromptThe Secondary Prompt11.2.3. The Search Path11.2.4. The dot Command11.3. The Command Line11.3.1. The Order of Processing Commands11.3.2. The Exit Status11.3.3. Multiple Commands and Command Grouping11.3.4. Conditional Execution of Commands11.3.5. Commands in the Background11.3.6. Command-Line HistoryThe history Command/Redisplay CommandsRe-executing Commands with the r Command11.3.7. Command-Line EditingThe vi Built-In EditorThe emacs Built-In EditorFCEDIT and Editing Commands11.4. Commenting and Filename Expansion11.5. Aliases11.5.1. Listing Aliases11.5.2. Creating Aliases11.5.3. Deleting Aliases11.5.4. Tracked Aliases11.6. Job Control11.7. Metacharacters11.8. Filename Substitution (Wildcards)11.8.1. The Asterisk11.8.2. The Question Mark11.8.3. The Square Brackets11.8.4. Escaping Metacharacters11.8.5. Tilde and Hyphen Expansion11.8.6. New ksh Metacharacters11.8.7. The noglob Variable11.9. Variables11.9.1. Local VariablesSetting and Referencing Local VariablesThe Scope of Local VariablesSetting Read-Only Variables11.9.2. Environment VariablesSetting Environment VariablesSpecial Environment Variables11.9.3. Listing Set Variables11.9.4. Unsetting Variables11.9.5. Printing the Values of Variables11.9.6. Escape Sequences11.9.7. Variable Expressions and Expansion Modifiers11.9.8. Variable Expansion of Substrings11.9.9. Variable Attributes: The typeset Command11.9.10. Positional Parameters11.9.11. Other Special Variables11.10. Quoting11.10.1. The Backslash11.10.2. Single Quotes11.10.3. Double Quotes11.11. Command Substitution11.12. Functions11.12.1. Defining Functions11.12.2. Functions and Aliases11.12.3. Listing Functions11.12.4. Unsetting Functions11.13. Standard I/O and Redirection11.13.1. The exec Command and Redirection11.13.2. Redirection and the Child Shell11.14. Pipes11.14.1. The here document and Redirecting Input11.15. Timing Commands11.15.1. The time Command11.15.2. The TMOUT Variable
12. Programming the Korn Shell
12.1. Introduction12.1.1. The Steps in Creating a Shell ScriptThe First LineCommentsExecutable Statements and Korn Shell ConstructsNaming and Storing ScriptsMaking a Script ExecutableUsing a Script As an Argument to kshA Scripting Session12.2. Reading User Input12.2.1. read and File Descriptors12.2.2. Reading Through Files12.3. Arithmetic12.3.1. The Integer Type12.3.2. Using Different Bases12.3.3. Listing Integers12.3.4. Arithmetic Operators and the let Command12.4. Positional Parameters and Command-Line Arguments12.4.1. The set Command and Positional Parameters12.5. Conditional Constructs and Flow Control12.5.1. Testing Exit Status and the $? Variable12.5.2. The Old test Command12.5.3. The New test Command12.5.4. File Testing with Binary Operators12.5.5. Expression Testing with Logical Operators12.5.6. File Testing with Flags12.5.7. The if Command12.5.8. Using the Old-Style Bourne test12.5.9. Using the New-Style Korn test12.5.10. Using the Old-Style Bourne test with Numbers12.5.11. The let Command and Testing Numbers12.5.12. The if/else Command12.5.13. The if/elif/else Command12.5.14. The exit Command12.5.15. The null Command12.5.16. The case CommandThe case Command and the here document12.6. Looping Commands12.6.1. The for Command12.6.2. The $* and $@ Variables in Wordlists12.6.3. The while Command12.6.4. The until Command12.6.5. The select Command and Menus12.6.6. Looping Control CommandsThe shift CommandThe break CommandThe continue Command12.6.7. Nested Loops and Loop Control12.6.8. I/O Redirection and LoopsRedirecting the Output of a Loop to a FilePiping the Output of a Loop to a UNIX Command12.6.9. Running Loops in the Background12.6.10. The exec Command and Loops12.6.11. The IFS and Loops12.7. Arrays12.7.1. Creating Arrays with the set Command12.8. Functions12.8.1. Defining Functions12.8.2. Listing and Unsetting Functions12.8.3. Local Variables and the Return Value12.8.4. Exported Functions12.8.5. Function Options and the typeset CommandAutoloaded Functions12.9. Trapping Signals12.9.1. Pseudo or Fake Signals12.9.2. Resetting Signals12.9.3. Ignoring Signals12.9.4. Listing Traps12.9.5. Traps and Functions12.10. Coprocesses12.11. Debugging12.12. The Command Line12.12.1. Processing Command-Line Options with getoptsSample getopts Scripts12.13. Security12.13.1. Privileged Scripts12.13.2. Restricted Shells12.14. Built-In Commands12.15. Korn Shell Invocation ArgumentsLAB 33: KORN SHELL—GETTING STARTEDLAB 34: HISTORYLAB 35: ALIASES AND FUNCTIONSLAB 36: SHELL METACHARACTERSLAB 37: TILDE EXPANSION, QUOTES, AND COMMAND SUBSTITUTIONLAB 38: REDIRECTIONLAB 39: JOB CONTROLLAB 40: WRITING THE info SHELL SCRIPTLAB 41: VARIABLE EXPANSION OF SUBSTRINGSLAB 42: THE lookup SCRIPTLAB 43: USING typesetLAB 44: THE if/else CONSTRUCT AND THE let COMMANDLAB 45: THE case STATEMENTLAB 46: THE select LOOPLAB 47: AUTOLOADING FUNCTIONS
13. The Interactive Bash Shell
13.1. Introduction13.1.1. Versions of bash13.1.2. StartupChanging the Shell at the Command Line13.2. The Environment13.2.1. The Initialization FilesThe /etc/profile FileThe ~/.bash_profile FileThe BASH_ENV (ENV) VariableThe .bashrc FileThe /etc/bashrc FileThe ~/.profile FileThe ~/.bash–logout FileOptions to Prevent Startup Files from Being ExecutedThe .inputrc File13.2.2. Setting bash Options with the Built-In set and shopt CommandsThe set –o OptionsThe shopt Built-In (Version 2.x+)13.2.3. The PromptsThe Primary PromptSetting the Prompt with Special Escape SequencesThe Secondary Prompt13.2.4. The Search Path13.2.5. The hash Command13.2.6. The source or dot Command13.3. The Command Line13.3.1. The Order of Processing Commands13.3.2. Built-In Commands and the help Command13.3.3. Changing the Order of Command-Line Processing13.3.4. The Exit Status13.3.5. Multiple Commands at the Command Line13.3.6. Command Grouping13.3.7. Conditional Execution of Commands13.3.8. Commands in the Background13.4. Job Control13.4.1. Job Control Commands and OptionsNew jobs OptionsThe disown Built-In13.5. Command-Line Shortcuts13.5.1. Command and Filename Completion13.5.2. History13.5.3. Accessing Commands from the History FileThe Arrow KeysThe history Built-In CommandThe fc CommandRe-executing History Commands (Bang! Bang!)13.5.4. Command-Line EditingThe vi Built-In EditorThe emacs Built-In EditorFCEDIT and Editing Commands13.6. Aliases13.6.1. Listing Aliases13.6.2. Creating Aliases13.6.3. Deleting Aliases13.7. Manipulating the Directory Stack13.7.1. The dirs Built-In Command13.7.2. The pushd and popd Commands13.8. Metacharacters (Wildcards)13.9. Filename Substitution (Globbing)13.9.1. The Asterisk13.9.2. The Question Mark13.9.3. The Square Brackets13.9.4. Brace Expansion13.9.5. Escaping Metacharacters13.9.6. Tilde and Hyphen Expansion13.9.7. Controlling Wildcards (Globbing)13.9.8. Extended Filename Globbing (bash 2.x)13.10. Variables13.10.1. Types of Variables13.10.2. Naming Conventions13.10.3. The declare Built-In13.10.4. Local Variables and ScopeSetting Local VariablesSetting Read-Only Variables13.10.5. Environment VariablesSetting Environment Variables.13.10.6. Unsetting Variables13.10.7. Printing the Values of VariablesThe echo CommandThe printf Command.13.10.8. Variable Expansion Modifiers (Parameter Expansion)13.10.9. Variable Expansion of Substrings13.10.10. Positional Parameters13.10.11. Other Special Variables13.11. Quoting13.11.1. The Backslash13.11.2. Single Quotes13.11.3. Double Quotes13.12. Command Substitution13.13. Arithmetic Expansion13.14. Order of Expansion13.15. Arrays13.16. Functions13.16.1. Defining Functions13.16.2. Listing and Unsetting Functions13.17. Standard I/O and Redirection13.17.1. The exec Command and Redirection13.18. Pipes13.18.1. The here document and Redirecting Input13.19. Shell Invocation Options13.19.1. The set Command and Options13.19.2. The shopt Command and Options13.20. Shell Built-In CommandsLAB 48: bash SHELL—GETTING STARTEDLAB 49: JOB CONTROLLAB 50: COMMAND COMPLETION, HISTORY, AND ALIASESLAB 51: SHELL METACHARACTERSLAB 52: REDIRECTIONLAB 53: VARIABLES
14. Programming the Bash Shell
14.1. Introduction14.1.1. The Steps in Creating a Shell ScriptThe First LineCommentsExecutable Statements and bash Shell ConstructsMaking the Script ExecutableA Scripting Session14.2. Reading User Input14.2.1. Variables (Review)14.2.2. The read Command14.3. Arithmetic14.3.1. Integers (declare and let Commands)The declare CommandListing IntegersRepresenting and Using Different BasesThe let Command14.3.2. Floating-Point Arithmetic14.4. Positional Parameters and Command-Line Arguments14.4.1. Positional Parameters14.4.2. The set Command and Positional ParametersHow $* and $@ Differ14.5. Conditional Constructs and Flow Control14.5.1. Exit Status14.5.2. The Built-In test and let CommandsThe test Command with Single BracketsThe test Command with Double BracketsThe let Command and Arithmetic with Double Parentheses14.5.3. The if CommandThe exit Command and the ? VariableChecking for Null ValuesNested if Commands14.5.4. The if/else Command14.5.5. The if/elif/else Command14.5.6. File Testing14.5.7. The null Command14.5.8. The case CommandCreating Menus with the here document and case Command14.6. Looping Commands14.6.1. The for Command14.6.2. The $* and $@ Variables in Wordlists14.6.3. The while Command14.6.4. The until Command14.6.5. The select Command and Menus14.6.6. Looping Control CommandsThe shift CommandThe break CommandThe continue CommandNested Loops and Loop Control14.6.7. I/O Redirection and SubshellsRedirecting the Output of a Loop to a FilePiping the Output of a Loop to a UNIX/Linux Command14.6.8. Running Loops in the Background14.6.9. The IFS and Loops14.7. Functions14.7.1. Unsetting Functions14.7.2. Exporting Functions14.7.3. Function Arguments and the Return ValueArgumentsThe Built-In local FunctionThe Built-In return Function14.7.4. Functions and the source (or dot) CommandStoring Functions14.8. Trapping Signals14.8.1. Resetting Signals14.8.2. Ignoring Signals14.8.3. Listing Traps14.8.4. Traps and Functions14.9. Debugging14.10. The Command Line14.10.1. Processing Command-Line Options with getoptsSample getopts ScriptsSpecial getopts Variables14.10.2. The eval Command and Parsing the Command Line14.11. bash Options14.11.1. Shell Invocation Options14.11.2. The set Command and Options14.11.3. The shopt Command and Options14.12. Shell Built-In CommandsLAB 54: bash SHELL—FIRST SCRIPTLAB 55: COMMAND-LINE ARGUMENTSLAB 56: GETTING USER INPUTLAB 57: CONDITIONAL STATEMENTSLAB 58: CONDITIONALS AND FILE TESTINGLAB 59: THE case STATEMENTLAB 60: LOOPSLAB 61: FUNCTIONS
15. Debugging Shell Scripts
15.1. Introduction15.2. Style Issues15.3. Types of Errors15.3.1. Runtime Errors15.3.2. Naming Conventions15.3.3. Insufficient Permissions15.3.4. Path Problems15.3.5. The Shbang Line15.3.6. Sneaky Aliases15.4. Probable Causes for Syntax Errors15.4.1. Undefined and Misspelled Variables15.4.2. Incomplete Programming StatementsMissing KeywordsIndentationif/endif Errorscase and switch ErrorsLooping ErrorsOperator ErrorsMisusing an OperatorQuoting ErrorsQuoting MetacharactersWhat You Should Know About QuotesThe BackslashSingle QuotesDouble QuotesBackquotesCombining QuotesSetting the Shell VariableProblems with the here documentFile-Testing ErrorsChecking for File Existence in the Five Shells15.4.3. Common Error Messages from the Big 5 ShellsCommon C/TC Shell Error MessagesCommon Bourne Shell Error MessagesCommon Korn Shell Error MessagesCommon Bash Error Messages15.4.4. Logic Errors and RobustnessLogical Operator ErrorsRelational Operator ErrorsBranching ErrorsExit Status with an if ConditionalLacking Robustness15.5. Tracing with Shell Options and the set Command15.5.1. Debugging Bourne Shell ScriptsBourne Shell Debugging Options15.5.2. Debugging C/TC Shell Scripts15.5.3. Debugging Korn Shell Scripts15.5.4. Debugging Bash ScriptsBash Invocation OptionsThe set Command and OptionsThe shopt Command and Options15.6. Summary
16. The System Administrator and the Shell
16.1. Introduction16.2. The Superuser16.3. Becoming a Superuser with the su Command16.3.1. Running Scripts As RootRunning a Script in the Current Directory16.3.2. Scripts That Run As Root (setuid Programs)16.4. Boot Scripts16.4.1. A Little TerminologyThe Kernel and initWhat Is a Run Level?Single-User ModeBoot ScriptsWhat Is a Daemon?16.4.2. An Example Boot Script—The cron UtilityCreating a cron Instruction File (crontab File)The cron Boot Script16.4.3. Writing a Portable Script16.4.4. User-Specific Initialization FilesPossible Files for /etc/skel16.4.5. System-Wide Initialization Files/etc/profile/etc/bashrc/etc/csh.login/etc/csh.cshrc16.5. Summary
Bibliography
A. Useful UNIX/Linux Utilities for Shell Programmers
apropos—searches the whatis database for stringsarch—prints the machine architecture (see uname -m)at, batch—executes commands at a later timeawk—pattern scanning and processing languagebanner—makes postersbasename—with a directory name delivers portions of the pathnamebash—GNU Bourne Again Shellbc—processes precision arithmeticbdiff—compares two big filescal—displays a calendarcat—concatenates and displays fileschfn—changes the finger informationchmod—changes the permissions mode of a filechown—changes owner of filechsh—changes your login shellclear—clears the terminal screencmp—compares two filescompress—compress, uncompress, zcat compress, uncompress files, or display expanded filescp—copies filescpio—copies file archives in and outcron—the clock daemoncrypt—encodes or decodes a filecut—removes selected fields or characters from each line of a filedate—displays the date and time or sets the datedd—converts a file while copying itdiff—compares two files for differencesdos, xdos, dosexec, dosdebug—a Linux DOS emulator that runs MS-DOS and MS-DOS programs under Linuxdf—summarizes free disk spacedu—summarizes disk usageecho—echoes argumentsegrep—searches a file for a pattern using full regular expressionsexpr—evaluates arguments as an expressionfgrep—searches a file for a character stringfile—determines the type of a file by looking at its contentsfind—finds filesfinger—displays information about local and remote usersfmt—simple text formattersfold—folds long linesftp—file transfer programfree—displays amount of free and used memory in the systemfuser—identifies processes using files or socketsgawk—pattern scanning and processing languagegcc, g++—GNU project C and C++ Compiler (v2.7)getopt(s)—parses command-line optionsgrep—searches a file for a patterngroups—prints group membership of usergzip, gunzip, zcat—compresses or expands fileshead—outputs the first ten lines of a file(s)host—prints information about specified hosts or zones in DNSid—prints the username, user ID, group name, and group IDjsh—the standard, job control shellkill—sends a signal to terminate one or more processeskillall—kills processes by nameless—opposite of moreline—reads one lineln—creates hard links to fileslogname—gets the name of the user running the processlook—displays lines beginning with a given stringlp—sends output to a printer (AT&T)lpr—sends output to a printer (UCB)lpstat—prints information about the status of the LP print service (AT&T)lpq—prints information about the status of the printer (UCB)ls—lists contents of directorymail, rmail—reads mail or sends mail to usersmailx—interactive message processing systemmake—maintains, updates, and regenerates groups of related programs and filesman—formats and displays the online manual pagesmanpath—determines user's search path for man pagesmesg—permits or denies messages resulting from the write commandmkdir—creates a directorymore—browses or pages through a text filemtools—utilities to access DOS disks in UNIXmv—moves or renames filesnawk—pattern scanning and processing languagenewgrp—logs into a new groupnews—prints news itemsnice—runs a command at low prioritynohup—makes commands immune to hangups and quitsod—octal dumppack, pcat, unpack—compresses and expands filespasswd—changes the login password and password attributespaste—merges same lines of several files or subsequent lines of one filepcat—(see pack)pine—a Program for Internet News and E-mailpg—displays files one page at a timepr—prints filesping—reports if a remote system is reachable and aliveps—reports process statuspstree—displays a tree of processespwd—displays the present working directory namequota—displays users' disk usage and limitsrcp—remote file copyrdate—gets the date and time via the networkrgrep—a recursive, highlighting grep programrlogin—remote loginrm—removes files from directoriesrmdir—removes a directoryrsh—starts a remote shellruptime—shows the host status of local machinesrwho—who is logged in on local machinesscript—creates a typescript of a terminal sessionsed—streamlined editorsize—prints section sizes in bytes of object filessleep—suspends execution for some number of secondssort—sorts and/or merges filesspell—finds spelling errorssplit—splits a file into piecesstrings—finds any printable strings in an object or binary filestty—sets the options for a terminalsu—become superuser or another usersum—calculates a checksum for a filesync—updates the superblock and sends changed blocks to disktabs—sets tab stops on a terminaltail—displays the tail end of a filetalk—allows you to talk to another usertar—stores and retrieves files from an archive file, normally a tape devicetee—replicates the standard outputtelnet—communicates with a remote hosttest—evaluates an expressiontime—displays a summary of time used by this shell and its childrentimex—times a command; reports process data and system activitytop—displays top CPU processestouch—updates access time and/or modification time of a filetput—initializes a terminal or queries the terminfo databasetr—translates characterstrue—provides successful exit statustsort—topological sorttty—gets the name of the terminalumask—sets file-creation mode mask for permissionsuname—prints name of current machineuncompress—restores files to their original state after they have been compressed using the compress commanduniq—reports on duplicate lines in a fileunits—converts quantities expressed in standard scales to other scalesunpack—expands files created by packuucp—copies files to another system, UNIX-to-UNIX system copyuuencode, uudecode—encodes a binary file into ASCII text in order to send it through e-mail, or converts it back into its original formwc—counts lines, words, and characterswhat—extracts SCCS version information from a file by printing information found after the @(#) patternwhich—locates a command and displays its pathname or alias (UCB)whereis—locates the binary, source, and manual page files for a command (UCB)who—displays who is logged on the systemwrite—writes a message to another userxargs—constructs an argument list(s) and executes a commandzcat—uncompresses a compressed file to standard output; Same as uncompress –czipinfo—lists detailed information about a ZIP archivezmore—file perusal filter for crt viewing of compressed text
B. Comparison of the Shells
B.1. The Shells ComparedB.2. tcsh versus cshB.3. bash versus sh
CD-ROM Warranty
About the CD-ROM

Content preview from UNIX® Shells by Example Fourth Edition

Chapter 6. The `awk` Utility

What's `awk`? What's `nawk`? What's `gawk`?

Awk is a UNIX programming language used for manipulating data and generating reports. Nawk is a newer version of awk, and gawk is the GNU version used on Linux.

The data may come from standard input, one or more files, or as output from a process. Awk can be used at the command line for simple operations, or it can be written into programs for larger applications. Because awk can manipulate data, it is an indispensable tool used in shell scripts and for managing small databases.

Awk scans a file (or input) line by line, from the first to the last line, searching for lines that match a ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 013147572XPurchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

UNIX® Shells by Example Fourth Edition

by Ellie Quigley

Chapter 6. The `awk` Utility

What's `awk`? What's `nawk`? What's `gawk`?

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.