book

Linux® Command Line and Shell Scripting Bible

Name: Linux® Command Line and Shell Scripting Bible
Author: Richard Blum
ISBN: 9780470251287

by Richard Blum

May 2008

Beginner

839 pages

16h 37m

English

Wiley

Read now

Unlock full access

Copyright
About the Author
Credits
Acknowledgments
Introduction
Who Should Read This BookHow This Book Is OrganizedConventions and FeaturesNotes and CautionsMinimum RequirementsWhere to Go fromHere
I. The Linux Command Line
1. Starting with Linux Shells
1.1. What Is Linux?1.1.1. Looking into the Linux kernel1.1.1.1. System memory management1.1.1.2. Software program management1.1.1.3. Hardware management1.1.1.4. Filesystem management1.1.2. The GNU utilities1.1.2.1. The core GNU utilities1.1.2.2. The shell1.1.3. The Linux desktop environment1.1.3.1. The X Windows system1.1.3.2. The KDE desktop1.1.3.3. The GNOME desktop1.1.3.4. Other desktops1.2. Linux Distributions1.2.1. Core Linux distributions1.2.2. Specialized Linux distributions1.2.3. The Linux LiveCD1.3. Summary
2. Getting to the Shell
2.1. Terminal Emulation2.1.1. Graphics capabilities2.1.1.1. Character sets2.1.1.2. Control codes2.1.1.3. Block mode graphics2.1.1.4. Vector graphics2.1.1.5. Display buffering2.1.1.6. Color2.1.2. The keyboard2.2. The terminfo Database2.3. The Linux Console2.4. The xterm Terminal2.4.1. Command line parameters2.4.2. The xterm main menu2.4.2.1. X event commands2.4.2.2. Output capturing2.4.2.3. Keyboard settings2.4.3. The VT options menu2.4.3.1. VT features2.4.3.2. VT commands2.4.3.3. Current screen commands2.4.4. The VT fonts menu2.4.4.1. Set the font2.4.4.2. Display the font2.4.4.3. Specify the font2.5. The Konsole Terminal2.5.1. Command line parameters2.5.2. Sessions2.5.3. The menu bar2.5.3.1. Session2.5.3.2. Edit2.5.3.3. View2.5.3.4. Bookmarks2.5.3.5. Settings2.5.3.6. Help2.6. The GNOME Terminal2.6.1. The command line parameters2.6.2. Tabs2.6.3. The menu bar2.6.3.1. File2.6.3.2. Edit2.6.3.3. View2.6.3.4. Terminal2.6.3.5. Tabs2.6.3.6. Help2.7. Summary
3. Basic bash Shell Commands
3.1. Starting the Shell3.2. The Shell Prompt3.3. The bash Manual3.4. Filesystem Navigation3.4.1. The Linux filesystem3.4.2. Traversing directories3.4.2.1. Absolute filepaths3.4.2.2. Relative filepaths3.5. File and Directory Listing3.5.1. Basic listing3.5.2. Modifying the information presented3.5.3. The complete parameter list3.5.4. Filtering listing output3.6. File Handling3.6.1. Creating files3.6.2. Copying files3.6.3. Linking files3.6.4. Renaming files3.6.5. Deleting files3.7. Directory Handling3.7.1. Creating directories3.7.2. Deleting directories3.8. Viewing File Contents3.8.1. Viewing file statistics3.8.2. Viewing the file type3.8.3. Viewing the whole file3.8.3.1. The cat command3.8.3.2. The more command3.8.3.3. The less command3.8.4. Viewing parts of a file3.8.4.1. The tail command3.8.4.2. The head command3.9. Summary
4. More bash Shell Commands
4.1. Monitoring Programs4.1.1. Peeking at the processes4.1.1.1. Unix-style parameters4.1.1.2. BSD-style parameters4.1.1.3. The GNU long parameters4.1.2. Real-time process monitoring4.1.3. Stopping processes4.1.3.1. The kill command4.1.3.2. The killall command4.2. Monitoring Disk Space4.2.1. Mounting media4.2.1.1. The mount command4.2.1.2. The umount command4.2.2. Using the df command4.2.3. Using the du command4.3. Working with Data Files4.3.1. Sorting data4.3.2. Searching for data4.3.3. Compressing data4.3.3.1. The bzip2 utility4.3.3.2. The gzip utility4.3.3.3. The zip utility4.3.4. Archiving data4.4. Summary

5. Using Linux Environment Variables
5.1. What Are Environment Variables?5.1.1. Global environment variables5.1.2. Local environment variables5.2. Setting Environment Variables5.2.1. Setting local environment variables5.2.2. Setting global environment variables5.3. Removing Environment Variables5.4. Default Shell Environment Variables5.5. Setting the PATH Environment Variable5.6. Locating System Environment Variables5.6.1. Login shell5.6.1.1. The /etc/profile file5.6.1.2. The $HOME startup files5.6.2. Interactive shell5.6.3. Non-interactive shell5.7. Variable Arrays5.8. Using Command Aliases5.9. Summary
6. Understanding Linux File Permissions
6.1. Linux Security6.1.1. The /etc/passwd file6.1.2. The /etc/shadow file6.1.3. Adding a new user6.1.4. Removing a user6.1.5. Modifying a user6.1.5.1. usermod6.1.5.2. passwd and chpasswd6.1.5.3. chsh, chfn, and chage6.2. Using Linux Groups6.2.1. The /etc/group file6.2.2. Creating new groups6.2.3. Modifying groups6.3. Decoding File Permissions6.3.1. Using file permission symbols6.3.2. Default file permissions6.4. Changing Security Settings6.4.1. Changing permissions6.4.2. Changing ownership6.5. Sharing Files6.6. Summary
7. Working with Editors
7.1. The vim Editor7.1.1. The basics of vim7.1.2. Editing data7.1.3. Copy and paste7.1.4. Search and substitute7.2. The emacs Editor7.2.1. Using emacs on the console7.2.1.1. The basics of emacs7.2.1.2. Editing data7.2.1.3. Copying and pasting7.2.1.4. Searching and replacing7.2.1.5. Using buffers in emacs7.2.1.6. Using windows in console mode emacs7.2.2. Using emacs in X Windows7.3. The KDE Family of Editors7.3.1. The KWrite editor7.3.2. The Kate editor7.4. The GNOME Editor7.4.1. Starting gedit7.4.2. Basic gedit features7.4.3. Setting preferences7.4.3.1. View7.4.3.2. Editor7.4.3.3. Font & Colors7.4.3.4. Syntax Highlighting7.4.3.5. Plugins7.5. Summary
II. Shell Scripting Basics
8. Basic Script Building
8.1. Using Multiple Commands8.2. Creating a Script File8.3. Displaying Messages8.4. Using Variables8.4.1. Environment variables8.4.2. User variables8.4.3. The backtick8.5. Redirecting Input and Output8.5.1. Output redirection8.5.2. Input redirection8.6. Pipes8.7. Performing Math8.7.1. The expr command8.7.2. Using brackets8.7.3. A floating-point solution8.7.3.1. The basics of bc8.7.3.2. Using bc in scripts8.8. Exiting the Script8.8.1. Checking the exit status8.8.2. The exit command8.9. Summary
9. Using Structured Commands
9.1. Working with the if-then Statement9.2. The if-then-else Statement9.3. Nesting ifs9.4. The test Command9.4.1. Numeric comparisons9.4.2. String comparisons9.4.2.1. String equality9.4.2.2. String order9.4.2.3. String size9.4.3. File comparisons9.4.3.1. Checking directories9.4.3.2. Checking if an object exists9.4.3.3. Checking for a file9.4.3.4. Can you read it?9.4.3.5. Checking for empty files9.4.3.6. Checking if you can write to a file9.4.3.7. Checking if you can run a file9.4.3.8. Checking ownership9.4.3.9. Checking file date9.5. Compound Condition Testing9.6. Advanced if-then Features9.6.1. Using double parentheses9.6.2. Using double brackets9.7. The case Command9.8. Summary
10. More Structured Commands
10.1. The for Command10.1.1. Reading values in a list10.1.2. Reading complex values in a list10.1.3. Reading a list from a variable10.1.4. Reading values from a command10.1.5. Changing the field separator10.1.6. Reading a directory using wildcards10.2. The C-Style for Command10.2.1. The C language for command10.2.2. Using multiple variables10.3. The while Command10.3.1. Basic while format10.3.2. Using multiple test commands10.4. The until Command10.5. Nesting Loops10.6. Looping on File Data10.7. Controlling the Loop10.7.1. The break command10.7.1.1. Breaking out of a single loop10.7.1.2. Breaking out of an inner loop10.7.1.3. Breaking out of an outer loop10.7.2. The continue command10.8. Processing the Output of a Loop10.9. Summary
11. Handling User Input
11.1. Command Line Parameters11.1.1. Reading parameters11.1.2. Reading the program name11.1.3. Testing parameters11.2. Special Parameter Variables11.2.1. Counting parameters11.2.2. Grabbing all the data11.3. Being Shifty11.4. Working With Options11.4.1. Finding your options11.4.1.1. Processing simple options11.4.1.2. Separating options from parameters11.4.1.3. Processing options with values11.4.2. Using the getopt command11.4.2.1. The command format11.4.2.2. Using getopt in your scripts11.4.3. The more advanced getopts11.5. Standardizing Options11.6. Getting User Input11.6.1. Basic reading11.6.2. Timing out11.6.3. Silent reading11.6.4. Reading from a file11.7. Summary
12. Presenting Data
12.1. Understanding Input and Output12.1.1. Standard file descriptors12.1.1.1. STDIN12.1.1.2. STDOUT12.1.1.3. STDERR12.1.2. Redirecting errors12.1.2.1. Redirecting just errors12.1.2.2. Redirecting errors and data12.2. Redirecting Output in Scripts12.2.1. Temporary redirections12.2.2. Permanent redirections12.3. Redirecting Input in Scripts12.4. Creating Your Own Redirection12.4.1. Creating output file descriptors12.4.2. Redirecting file descriptors12.4.3. Creating input file descriptors12.4.4. Creating a read/write file descriptor12.4.5. Closing file descriptors12.5. Listing Open File Descriptors12.6. Suppressing Command Output12.7. Using Temporary Files12.7.1. Creating a local temporary file12.7.2. Creating a temporary file in /tmp12.7.3. Creating a temporary directory12.8. Logging Messages12.9. Summary
13. Script Control
13.1. Handling Signals13.1.1. Linux signals revisited13.1.2. Generating signals13.1.2.1. Interrupting a process13.1.2.2. Pausing a process13.1.3. Trapping signals13.1.4. Trapping a script exit13.1.5. Removing a trap13.2. Running Scripts in Background Mode13.2.1. Running in the background13.2.2. Running multiple background jobs13.2.3. Exiting the terminal13.3. Running Scripts without a Console13.4. Job Control13.4.1. Viewing jobs13.4.2. Restarting stopped jobs13.5. Being Nice13.5.1. The nice command13.5.2. The renice command13.6. Running Like Clockwork13.6.1. Scheduling a job using the at command13.6.1.1. The at command format13.6.1.2. Retrieving job output13.6.1.3. Listing pending jobs13.6.1.4. Removing jobs13.6.2. Using the batch command13.6.3. Scheduling regular scripts13.6.3.1. The cron table13.6.3.2. Building the cron table13.6.3.3. The anacron program13.7. Start At the Beginning13.7.1. Starting your scripts at boot13.7.1.1. The boot process13.7.1.2. Defining your scripts13.7.2. Starting with a new shell13.8. Summary
III. Advanced Shell Scripting
14. Creating Functions
14.1. Basic Script Functions14.1.1. Creating a function14.1.2. Using functions14.2. Returning a Value14.2.1. The default exit status14.2.2. Using the return command14.2.3. Using function output14.3. Using Variables in Functions14.3.1. Passing parameters to a function14.3.2. Handling variables in a function14.3.2.1. Global variables14.3.2.2. Local variables14.4. Array Variables and Functions14.4.1. Passing arrays to functions14.4.2. Returning arrays from functions14.5. Function Recursion14.6. Creating a Library14.7. Using Functions on the Command Line14.7.1. Creating functions on the command line14.7.2. Defining functions in the .bashrc file14.7.2.1. Directly defining functions14.7.2.2. Sourcing function files14.8. Summary
15. Adding Color to Scripts
15.1. Creating Text Menus15.1.1. Create the menu layout15.1.2. Create the menu functions15.1.3. Add the menu logic15.1.4. Putting it all together15.1.5. Using the select command15.2. Adding Color15.2.1. The ANSI escape codes15.2.2. Displaying ANSI escape codes15.2.3. Using colors in scripts15.3. Doing Windows15.3.1. The dialog package15.3.1.1. The msgbox widget15.3.1.2. The yesno widget15.3.1.3. The inputbox widget15.3.1.4. The textbox widget15.3.1.5. The menu widget15.3.1.6. The fselect widget15.3.2. The dialog options15.3.3. Using the dialog command in a script15.4. Getting Graphic15.4.1. The KDE environment15.4.1.1. kdialog widgets15.4.1.2. Using kdialog15.4.2. The GNOME environment15.4.2.1. zenity widgets15.4.2.2. Using zenity in scripts15.5. Summary
16. Introducing sed and gawk
16.1. Text Manipulation16.1.1. The sed editor16.1.1.1. Defining an editor command in the command line16.1.1.2. Using multiple editor commands in the command line16.1.1.3. Reading editor commands from a file16.1.2. The gawk program16.1.2.1. The gawk command format16.1.2.2. Reading the program script from the command line16.1.2.3. Using data field variables16.1.2.4. Using multiple commands in the program script16.1.2.5. Reading the program from a file16.1.2.6. Running scripts before processing data16.1.2.7. Running scripts after processing data16.2. The sed Editor Basics16.2.1. More substitution options16.2.1.1. Substitution flags16.2.1.2. Replacement characters16.2.2. Using addresses16.2.2.1. Numeric line addressing16.2.2.2. Using text pattern filters16.2.2.3. Grouping commands16.2.3. Deleting lines16.2.4. Inserting and appending text16.2.5. Changing lines16.2.6. The transform command16.2.7. Printing revisited16.2.7.1. Printing lines16.2.7.2. Printing line numbers16.2.7.3. Listing lines16.2.8. Using files with sed16.2.8.1. Writing to a file16.2.8.2. Reading data from a file16.3. Summary
17. Regular Expressions
17.1. What Are Regular Expressions?17.1.1. A definition17.1.2. Types of regular expressions17.2. Defining BRE Patterns17.2.1. Plain text17.2.2. Special characters17.2.3. Anchor characters17.2.3.1. Starting at the beginning17.2.3.2. Looking for the ending17.2.3.3. Combining anchors17.2.4. The dot character17.2.5. Character classes17.2.6. Negating character classes17.2.7. Using ranges17.2.8. Special character classes17.2.9. The asterisk17.3. Extended Regular Expressions17.3.1. The question mark17.3.2. The plus sign17.3.3. Using braces17.3.4. The pipe symbol17.3.5. Grouping expressions17.4. Regular Expressions in Action17.4.1. Counting directory files17.4.2. Validating a phone number17.4.3. Parsing an e-mail address17.5. Summary
18. Advanced sed
18.1. Multiline Commands18.1.1. The next commands18.1.1.1. The single-line next command18.1.1.2. Combining lines of text18.1.2. The multiline delete command18.1.3. The multiline print command18.2. The Hold Space18.3. Negating a Command18.4. Changing the Flow18.4.1. Branching18.4.2. Testing18.5. Pattern Replacement18.5.1. The ampersand18.5.2. Replacing individual words18.6. Using sed in Scripts18.6.1. Using wrappers18.6.2. Redirecting sed output18.7. Creating sed Utilities18.7.1. Double spacing lines18.7.2. Double spacing files that may have blanks18.7.3. Numbering lines in a file18.7.4. Printing last lines18.7.5. Deleting lines18.7.5.1. Deleting consecutive blank lines18.7.5.2. Deleting leading blank lines18.7.5.3. Deleting trailing blank lines18.7.6. Removing HTML tags18.8. Summary
19. Advanced gawk
19.1. Using Variables19.1.1. Built-in variables19.1.1.1. The field and record separator variables19.1.1.2. Data variables19.1.2. User-defined variables19.1.2.1. Assigning variables in scripts19.1.2.2. Assigning variables in the command line19.2. Working with Arrays19.2.1. Defining array variables19.2.2. Iterating through array variables19.2.3. Deleting array variables19.3. Using Patterns19.3.1. Regular expressions19.3.2. The matching operator19.3.3. Mathematical expressions19.4. Structured Commands19.4.1. The if statement19.4.2. The while statement19.4.3. The do-while statement19.4.4. The for statement19.5. Formatted Printing19.6. Built-in Functions19.6.1. Mathematical functions19.6.2. String functions19.6.3. Time functions19.7. User-Defined Functions19.7.1. Defining a function19.7.2. Using your functions19.7.3. Creating a function library19.8. Summary
IV. Alternative Linux Shells
20. The ash Shell
20.1. What Is the ash Shell?20.2. The Original ash Shell20.2.1. The Original ash command line parameters20.2.2. The original ash built-in commands20.2.2.1. The bltin command20.2.2.2. The hash command20.2.2.3. The lc command20.2.3. The ash shell files20.3. The dash Shell20.3.1. The dash command line parameters20.3.2. The dash environment variables20.3.2.1. Default environment variables20.3.2.2. Positional parameters20.3.2.3. User-defined environment variables20.3.3. The dash built-in commands20.3.3.1. The printf command20.3.3.2. The ulimit command20.4. Scripting in dash20.4.1. Creating ash and dash scripts20.4.2. Things that won't work20.4.2.1. Using arithmetic20.4.2.2. The test command20.4.2.3. The echo statement options20.4.2.4. The function command20.5. Summary
21. The tcsh Shell
21.1. What Is the tcsh Shell?21.2. The tcsh Shell Components21.2.1. The tcsh command line parameters21.2.2. The tcsh files21.2.3. The tcsh login files21.2.4. Shell startup files21.2.5. The logout files21.2.6. The tcsh environment variables21.2.7. Shell variables21.2.8. Environment variables21.2.9. Setting variables in tcsh21.2.10. Using the set command21.2.11. Using the setenv command21.2.12. The tcsh built-in commands21.3. Scripting in tcsh21.3.1. Working with variables21.3.2. Array variables21.3.3. Handling mathematical operations21.3.4. Structured commands21.3.5. The if statements21.3.6. The foreach statement21.3.7. The while statement21.3.8. The switch command21.4. Summary
22. The Korn Shell
22.1. The Korn Shell History22.2. The Parts of the ksh93 Shell22.2.1. Command line parameters22.2.2. Default files22.2.3. Environment variables22.2.3.1. Random numbers22.2.3.2. Defining the variable type22.2.3.3. Using arrays22.2.3.4. Compound variables22.2.3.5. Special variable subscripts22.2.4. Built-in commands22.3. Scripting in the ksh93 Shell22.3.1. Mathematical operations22.3.1.1. Performing calculations22.3.1.2. Mathematical functions22.3.2. Structured commands22.3.2.1. The if-then-else statement22.3.2.2. The while and until loops22.3.3. Command redirection22.3.4. Discipline functions22.4. Summary
23. The zsh Shell
23.1. History of the zsh Shell23.2. Parts of the zsh Shell23.2.1. Shell options23.2.1.1. Shell state23.2.1.2. Shell emulation23.2.1.3. Initialization23.2.1.4. Scripts and functions23.2.2. The zsh shell files23.2.2.1. The shell startup files23.2.2.2. The shell login files23.2.2.3. The interactive shell files23.2.2.4. The shell logout files23.2.3. Environment variables23.2.4. Built-in commands23.2.4.1. Core built-in commands23.2.4.2. Add-in modules23.2.4.3. Viewing, adding, and removing modules23.3. Scripting with zsh23.3.1. Mathematical operations23.3.1.1. Performing calculations23.3.1.2. Mathematical functions23.3.2. Structured commands23.3.3. Functions23.4. Summary
V. Advanced Topics
24. Using a Database
24.1. The MySQL Database24.1.1. Installing MySQL24.1.2. Completing the MySQL configuration24.1.2.1. Initializing the database files24.1.2.2. Starting the MySQL server24.1.2.3. Securing the MySQL user account24.1.3. The MySQL client interface24.1.3.1. Connecting to the server24.1.3.2. The mysql commands24.1.4. Creating MySQL database objects24.1.4.1. Creating a database24.1.4.2. Creating a user account24.2. The PostgreSQL Database24.2.1. Installing PostgreSQL24.2.2. The PostgreSQL command interface24.2.2.1. Connecting to the server24.2.2.2. The psql commands24.2.3. Creating PostgreSQL database objects24.2.3.1. Creating a database object24.2.3.2. Creating user accounts24.3. Working with Tables24.3.1. Creating a table24.3.2. Inserting and deleting data24.3.3. Querying data24.4. Using the Database in Your Scripts24.4.1. Connecting to the databases24.4.1.1. Finding the programs24.4.1.2. Logging into the server24.4.2. Sending commands to the server24.4.3. Formatting data24.4.3.1. Assigning output to a variable24.4.3.2. Using formatting tags24.5. Summary
25. Using the Web
25.1. The Lynx Program25.1.1. Installing Lynx25.1.2. The lynx command line25.1.3. The Lynx configuration file25.1.4. The Lynx environment variables25.1.5. Capturing data from Lynx25.2. The cURL Program25.2.1. Installing cURL25.2.2. The cURL command line25.2.3. Exploring with curl25.3. Networking with zsh25.3.1. The TCP module25.3.2. The client/server paradigm25.3.3. Client/server programming with zsh25.3.3.1. The server program25.3.3.2. The client program25.3.3.3. Running the programs25.4. Summary
26. Using E-Mail
26.1. The Basics of Linux E-Mail26.1.1. E-Mail in Linux26.1.2. The Mail Transfer Agent26.1.3. The Mail Delivery Agent26.1.4. The Mail User Agent26.1.4.1. Mailx26.1.4.2. Mutt26.1.4.3. Graphical e-mail clients26.2. Setting Up Your Server26.2.1. sendmail26.2.1.1. Parts of the sendmail program26.2.1.2. The sendmail.cf file26.2.2. Postfix26.2.2.1. Parts of the Postfix system26.2.2.2. Postfix configuration files26.3. Sending a Message with Mailx26.4. The Mutt Program26.4.1. Installing Mutt26.4.2. The Mutt command line26.4.3. Using Mutt26.5. Summary
27. Shell Scripts for Administrators
27.1. Monitoring System Statistics27.1.1. Monitoring disk free space27.1.1.1. The required functions27.1.1.2. Creating the script27.1.1.3. Running the script27.1.2. Catching disk hogs27.1.2.1. The required functions27.1.2.2. Creating the script27.1.2.3. Running the script27.1.3. Watching CPU and memory usage27.1.3.1. The required functions27.1.3.2. Creating the capture script27.1.3.3. Generating the report script27.1.3.4. Running the script27.2. Performing Backups27.2.1. Archiving data files27.2.1.1. The required functions27.2.1.2. Creating a daily archive script27.2.1.3. Creating an hourly archive script27.2.2. Storing backups off-site27.2.2.1. The required functions27.2.2.2. Creating the script27.2.2.3. Running the script27.3. Summary
A. Quick Guide to bash Commands
A.1. Built-In CommandsA.2. Bash CommandsA.3. Environment Variables
B. Quick Guide to sed and gawk
B.1. The sed EditorB.1.1. Starting the sed editorB.1.2. sed commandsB.1.2.1. SubstitutionB.1.2.2. AddressingB.1.2.3. Deleting linesB.1.2.4. Inserting and appending textB.1.2.5. Changing linesB.1.2.6. Transform commandB.1.2.7. Printing linesB.1.2.8. Writing to a fileB.1.2.9. Reading from a fileB.2. The gawk programB.2.1. The gawk command formatB.2.2. Using gawkB.2.2.1. Reading the program script from the command lineB.2.2.2. Using multiple commands in the program scriptB.2.2.3. Reading the program from a fileB.2.2.4. Running scripts before processing dataB.2.2.5. Running scripts after processing dataB.2.3. The gawk variablesB.2.3.1. Built-in variablesB.2.3.2. Assigning variables in scriptsB.2.3.3. Assigning variables in the command lineB.2.4. The gawk program featuresB.2.4.1. Regular expressionsB.2.4.2. The matching operatorB.2.4.3. Mathematical expressionsB.2.4.4. Structured commands
C. Comparing Shells
C.1. VariablesC.1.1. Environment variablesC.1.2. User-defined variablesC.1.3. Array variablesC.2. Structured CommandsC.2.1. The if-then, while, and until statementsC.2.2. The for statementC.3. Mathematical Operations

Content preview from Linux® Command Line and Shell Scripting Bible

Chapter 19. Advanced gawk

IN THIS CHAPTER

Reexamining gawk
Use variables in gawk
Structured commands
Format your printing
Functions

Chapter 16 introduced the gawk program and demonstrated the basics of using it to produce formatted reports from raw data files. This chapter dives more deeply into customizing gawk to produce reports. The gawk program is a full-fledged programming language, providing features that allow you to write advanced programs to manipulate data. In this chapter, you'll see how to use the gawk programming language to write programs to handle just about any data formatting task you'll run into.

Using Variables

One important feature of any programming language is the ability to store and recall values using variables. The gawk programming language supports two different types of variables:

Built-in variables
User-defined variables

There are several built-in variables available for you to use in gawk. The built-in variables contain information used in handling the data fields and records in the data file. You can also create your own variables in your gawk programs. The following sections walk you through how to use variables in your gawk programs.

Built-in variables

The gawk program uses built-in variables to reference specific features within the program data. This section describes the built-in variables available for you to use in your gawk programs and demonstrates how to use them.

The field and record separator variables

Chapter 16 demonstrated one type of built-in variable ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Linux® Command Line and Shell Scripting Bible, Second Edition

Publisher Resources

ISBN: 9780470251287Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

Linux® Command Line and Shell Scripting Bible

by Richard Blum

Chapter 19. Advanced gawk

Using Variables