Shell Scripting Recipes: A Problem-Solution Approach, Second Edition

Shell Scripting Recipes: A Problem-Solution Approach, Second Edition

by Chris F. A. Johnson, Jayant Varma
Released October 2015
Publisher(s): Apress
ISBN: 9781484202203

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Shell Scripting Recipes is filled with over 150 much-needed and practical recipes that follow a problem-solution format, and help all Unix users regain some of the lost time spent creating and testing shell scripts. Most scripts in this book are POSIX (Portable Operating System Interface)-compliant, so they are supported by many of the major shell variants, including Bash, ksh, and sh, among others. This completely updated second edition takes into account the many changes to shell scripting that have occurred since its original publication nearly ten years ago, and all the recipes are now relevant for a modern audience.

Each real-world example recipe follows the same structure and easily shows you what's going on in each case. File conversion (DOS, UNIX, and Mac), system administration, and resource monitoring are just a few of the topics covered in this highly practical shell scripting reference. You will also find recipes for filename management, complex date calculations, screen control capabilities, and more.

Author Chris Johnson's code is clear, direct, and applicable, and he explains everything well in each recipe so that you understand not just how to do something, but why to do it that way. Shell Scripting Recipes is an invaluable book and one to add to your library today.

Table of contents

  1. Cover
  2. Title
  3. Copyright
  4. Dedication
  5. Contents at a Glance
  6. Contents
  7. About the Authors
  8. Acknowledgments
  9. Chapter 1 : The POSIX Shell and Command-Line Utilities
    1. Shell Commands
      1. echo
      2. printf
      3. set
      4. shift
      5. type
      6. getopts
      7. case
      8. eval
      9. local
    2. Parameters and Variables
      1. Positional Parameters
      2. Special Parameters
      3. standard-vars—A Collection of Useful Variables
    3. Patterns
      1. Pathname Expansion
      2. Regular Expressions
    4. Parameter Expansion
      1. The Bourne Shell Expansions
      2. POSIX Parameter Expansions
      3. Shell-Specific Expansions, bash2, and ksh93
    5. Shell Arithmetic
    6. Aliases
    7. Sourcing a File
    8. Functions
      1. Functions Are Fast
      2. Command Substitution Is Slow
      3. Using the Functions in This Book
    9. standard-funcs: A Collection of Useful Commands
      1. 1.1 get_key—Get a Single Keystroke from the User
      2. 1.2 getline—Prompt User to Enter a Line
      3. 1.3 press_any_key—Prompt for a Single Keypress
      4. 1.4 menu1—Print a Menu and Execute a Selected Command
      5. 1.5 arg—Prompt for Required Argument If None Supplied
      6. 1.6 die—Print Error Message and Exit with Error Status
      7. 1.7 show_date—Display Date in D[D] MMM YYYY Format
      8. 1.8 date_vars—Set Date and Time Variables
      9. 1.9 is_num—Is This a Positive Integer?
      10. 1.10 abbrev_num—Abbreviate Large Numbers
      11. 1.11 commas—Add Thousands Separators to a Number
      12. 1.12 pr1—Print Arguments, One to a Line
      13. 1.13 checkdirs—Check for Directories; Create If Necessary
      14. 1.14 checkfiles—Check That a Directory Contains Certain Files
      15. 1.15 zpad—Pad a Number with Leading Zeroes
      16. 1.16 cleanup—Remove Temporary Files and Reset Terminal on Exit
    10. The Unix Utilities
      1. cat: Concatenate Files to the Standard Output
      2. sed: A Text Stream Editor
      3. awk: Pattern Scanning and Processing Language
      4. grep: Print Lines Matching a Regular Expression
      5. date: Show or Set the System Date
      6. tr: A Character Translation Utility
      7. wc: Count Characters,Words, and Lines in a File
      8. file: Determine the File Type
      9. ls: Sort and Provide Details About Files
      10. uniq: Remove Consecutive Duplicate Lines
      11. sudo: Execute Commands as the Superuser
      12. split: Divide a File into Equal-Sized Pieces
      13. which: Show the Full Path to a Command
      14. gs, gv: Render, Convert, or View PostScript and PDF Files
    11. Summary
  10. Chapter 2 : Playing with Files: Viewing, Manipulating, and Editing Text Files
    1. End of the Line for OS Differences: Converting Text Files
      1. 2.1 dos2unix—Convert Windows Text Files to Unix
      2. 2.2 unix2dos—Convert a Unix File to Windows
      3. 2.3 mac2unix—Convert Macintosh Files to Unix
      4. 2.4 unix2mac—Convert Unix Files to Mac Format
      5. 2.5 dos2mac—Convert Windows Files to Macintosh
      6. 2.6 mac2dos—Convert Macintosh Files to Windows
    2. Displaying Files
      1. 2.7 prn—Print File with Line Numbers
      2. 2.8 prw—Print One Word per Line
      3. 2.9 wbl—Sort Words by Length
    3. Formatting File Facts
      1. 2.10 finfo—File Information Display
      2. 2.11 wfreq—Word Frequency
      3. 2.12 lfreq—Letter Frequency
      4. 2.13 fed—A Simple Batch File Editor
    4. Summary
  11. Chapter 3 : String Briefs
    1. Character Actions: The char-funcs Library
      1. 3.1 chr—Convert a Decimal Number to an ASCII Character
      2. 3.2 asc—Convert a Character to Its Decimal Equivalent
      3. 3.3 nxt—Get the Next Character in ASCII Sequence
      4. 3.4 upr—Convert Character(s) to Uppercase
      5. 3.5 lwr—Convert Character(s) to Lowercase
    2. String Cleaning: The string-funcs Library
      1. 3.6 sub—Replace First Occurrence of a Pattern
      2. 3.7 gsub—Globally Replace a Pattern in a String
      3. 3.8 repeat—Build a String of a Specified Length
      4. 3.9 index, rindex—Find Position of One String Within Another
      5. 3.10 substr—Extract a Portion of a String
      6. 3.11 insert_str—Place One String Inside Another
    3. Summary
  12. Chapter 4 : What’s in a Word?
    1. Finding and Massaging Word Lists
    2. wf-funcs: WordFinder Function Library
      1. 4.1 write_config—Write User’s Information to the Configuration File
      2. 4.2 do_config—Check For and Source Default Configuration File
      3. 4.3 set_sysdict—Select the Dictionary Directory
      4. 4.4 mkwsig—Sort Letters in a Word
      5. 4.5 wf-clean—Remove Carriage Returns and Accents
      6. 4.6 wf-compounds—Squish Compound Words and Save with Lengths
      7. 4.7 wf-setup—Prepare Word and Anagram Lists
    3. Playing with Matches
      1. 4.8 wf—Find Words That Match a Pattern
      2. 4.9 wfb—Find Words That Begin with a Given Pattern
      3. 4.10 wfe—Find Words That End with a Given Pattern
      4. 4.11 wfc—Find Words That Contain a Given Pattern
      5. 4.12 wfit—Find Words That Fit Together in a Grid
      6. 4.13 anagram—Find Words That Are Anagrams of a Given Word
      7. 4.14 aplus—Find Anagrams of a Word with a Letter Added
      8. 4.15 aminus—Remove Each Letter in Turn and Anagram What’s Left
    4. Summary
  13. Chapter 5 : Scripting by Numbers
    1. The math-funcs Library
      1. 5.1 fpmul—Multiply Decimal Fractions
      2. 5.2 int—Return the Integer Portion of a Decimal Fraction
      3. 5.3 round—Round the Argument to the Nearest Integer
      4. 5.4 pow—Raise a Number to Any Given Power
      5. 5.5 square—Raise a Number to the Second Power
      6. 5.6 cube—Raise a Number to the Third Power
      7. 5.7 calc—A Simple Command-Line Calculator
    2. Adding and Averaging
      1. 5.8 total—Add a List of Numbers
      2. 5.9 mean—Find the Arithmetic Mean of a List of Numbers
      3. 5.10 median—Find the Median of a List of Numbers
      4. 5.11 mode—Find the Number That Appears Most in a List
      5. 5.12 range—Find the Range of a Set of Numbers
      6. 5.13 stdev—Finding the Standard Deviation
    3. Converting Between Unit Systems
      1. 5.14 conversion-funcs—Converting Metric Units
      2. 5.15 conversion—A Menu System for Metric Conversion
    4. Summary
  14. Chapter 6 : Loose Names Sink Scripts: Bringing Sanity to Filenames
    1. What’s in a Name?
      1. POSIX Portable Filenames
      2. OK Filenames
    2. Functioning Filenames: The filename-funcs Library
      1. 6.1 basename—Extract the Last Element of a Pathname
      2. 6.2 dirname—Return All but the Last Element of a Pathname
      3. 6.3 is_pfname—Check for POSIX Portable Filename
      4. 6.4 is_OKfname—Check Whether a Filename Is Acceptable
      5. 6.5 pfname—Convert Nonportable Characters in Filename
      6. 6.6 OKfname—Make a Filename Acceptable
      7. 6.7 is_whitespc—Does the Filename Contain Whitespace Characters?
      8. 6.8 whitespc—Fix Filenames Containing Whitespace Characters
      9. 6.9 is_dir—Is This a Directory I See Before Me?
      10. 6.10 nodoublechar—Remove Duplicate Characters from a String
      11. 6.11 new_filename—Change Filename to Desired Character Set
      12. 6.12 fix_pwd—Fix All the Filenames in the Current Directory
      13. 6.13 fixfname—Convert Filenames to Sensible Names
    3. Summary
  15. Chapter 7 : Treading a Righteous PATH
    1. The path-funcs Library
      1. 7.1 path—Display a User’s PATH
      2. 7.2 unslash—Remove Extraneous Slashes
      3. 7.3 checkpath—Clean Up the PATH Variable
      4. 7.4 addpath—Add Directories to the PATH Variable
      5. 7.5 rmpath—Remove One or More Directories from $PATH
    2. Summary
  16. Chapter 8 : The Dating Game
    1. The date-funcs Library
      1. 8.1 split_date—Divide a Date into Day, Month, and Year
      2. 8.2 is_leap_year—Is There an Extra Day This Year?
      3. 8.3 days_in_month—How Many Days Hath September?
      4. 8.4 Julian Dates
      5. 8.5 dateshift—Add or Subtract a Number of Days
      6. 8.6 diffdate—Find the Number of Days Between Two Dates
      7. 8.7 day_of_week—Find the Day of the Week for Any Date
      8. 8.8 display_date—Show a Date in Text Format
      9. 8.9 parse_date—Decipher Various Forms of Date String
      10. 8.10 valid_date—Where Was I on November 31st?
    2. Summary
  17. Chapter 9 : Good Housekeeping: Monitoring and Tidying Up File Systems
    1. 9.1 dfcmp—Notify User of Major Changes in Disk Usage
      1. How It Works
    2. 9.2 symfix—Remove Broken Symbolic Links
      1. How It Works
    3. 9.3 sym2file—Converts Symbolic Links to Regular Files
      1. How It Works
    4. 9.4 zrm—Remove Empty Files
      1. How It Works
    5. 9.5 undup—Remove Duplicate Files
      1. How It Works
    6. 9.6 lsr—List the Most Recent (or Oldest) Files in a Directory
      1. How It Works
    7. Summary
  18. Chapter 10 : Screenplay: The screen–funcs Library
    1. 10.1 screen-vars—Variables for Screen Manipulation
      1. How It Works
    2. 10.2 set_attr—Set Screen-Printing Attributes
      1. How It Works
    3. 10.3 set_fg, set_bg, set_fgbg—Set Colors for Printing to the Screen
      1. How It Works
    4. 10.4 cls—Clear the Screen
      1. How It Works
      2. Notes
    5. 10.5 printat—Position Cursor by Row and Column
      1. How It Works
      2. Notes
    6. 10.6 put_block_at—Print Lines in a Column Anywhere on the Screen
      1. How It Works
    7. 10.7 get_size—Set LINES and COLUMNS Variables
      1. How It Works
    8. 10.8 max_length—Find the Length of the Longest Argument
      1. How It Works
    9. 10.9 print_block_at—Print a Block of Lines Anywhere on the Screen
      1. How It Works
    10. 10.10 vbar, hbar—Print a Vertical or Horizontal Bar
      1. How It Works
    11. 10.11 center—Center a String on N Columns
      1. How It Works
    12. 10.12 flush_right—Align String with the Right Margin
      1. How It Works
    13. 10.13 ruler—Draw a Ruler Across the Width and Height of the Window
      1. How It Works
    14. 10.14 box_block, box_block_at—Print Text Surrounded by a Box
      1. How It Works
    15. 10.15 clear_area, clear_area_at—Clear an Area of the Screen
      1. How It Works
    16. 10.16 box_area, box_area_at—Draw a Box Around an Area
      1. How It Works
    17. 10.17 screen-demo—Saving and Redisplaying Areas of the Screen
      1. How It Works
    18. Summary
  19. Chapter 11 : Aging, Archiving, and Deleting Files
    1. 11.1 date-file—Add a Datestamp to a Filename
      1. How It Works
      2. Notes
    2. 11.2 rmold—Remove Old Files
      1. How It Works
    3. 11.3 keepnewest—Remove All but the Newest or Oldest Files
      1. How It Works
    4. Summary
  20. Chapter 12 : Covering All Your Databases
    1. 12.1 lookup—Find the Corresponding Value for a Key
      1. How It Works
    2. shdb-funcs: Shell Database Function Library
    3. 12.2 load_db—Import Database into Shell Array
      1. How It Works
      2. Notes
    4. 12.3 split_record—Split a Record into Fields
      1. How It Works
    5. 12.4 csv_split—Extract Fields from CSV Records
      1. How It Works
    6. 12.5 put_record—Assemble a Database Record from an Array
      1. How It Works
    7. 12.6 put_csv—Assemble Fields into a CSV Record
      1. How It Works
    8. 12.7 db-demo—View and Edit a Password File
      1. How It Works
    9. PhoneBase: A Simple Phone Number Database
      1. How It Works
    10. 12.8 ph—Look Up a Phone Number
    11. 12.9 phadd—Add an Entry to PhoneBase
    12. 12.10 phdel—Delete an Entry from PhoneBase
    13. 12.11 phx—Show ph Search Results in an X Window
    14. Summary
  21. Chapter 13 : Home on the Web
    1. Playing with Hypertext: The html-funcs Library
      1. 13.1 get_element—Extract the First Occurrence of an Element
      2. 13.2 split_tags—Put Each Tag on Its Own Line
      3. 13.3 html-title—Get the Title from an HTML File
    2. HTML on the Fly: The cgi-funcs Library
      1. 13.4 x2d2—Convert a Two-Digit Hexadecimal Number to Decimal
      2. 13.5 dehex—Convert Hex Strings (%XX) to Characters
      3. 13.6 filedate—Find and Format the Modification Date of a File
    3. Creating HTML Files
      1. 13.7 mk-htmlindex—Create an HTML Index
      2. 13.8 pretext—Create a Wrapper Around a Text File
      3. 13.9 text2html—Convert a Text File to HTML
      4. 13.10 demo.cgi—A CGI Script
    4. Summary
  22. Chapter 14 : Taking Care of Business
    1. 14.1 prcalc—A Printing Calculator
      1. How It Works
    2. 14.2 gle—Keeping Records Without a Shoebox
      1. How It Works
    3. Summary
  23. Chapter 15 : Random Acts of Scripting
    1. The rand-funcs Library
    2. 15.1 random—Return One or More Random Integers in a Given Range
      1. How It Works
      2. Notes
    3. 15.2 toss—Simulate Tossing a Coin
      1. How It Works
    4. 15.3 randstr—Select a String at Random
      1. How It Works
    5. A Random Sampling of Scripts
    6. 15.4 rand-date—Generate Random Dates in ISO Format
      1. How It Works
      2. Notes
    7. 15.5 randsort—Print Lines in Random Order
      1. How It Works
    8. 15.6 randomword—Generate Random Words According to Format Specifications
      1. How It Works
    9. 15.7 dice—Roll a Set of Dice
      1. How It Works
    10. 15.8 throw—Throw a Pair of Dice
      1. How It Works
    11. Summary
  24. Chapter 16 : A Smorgasbord of Scripts
    1. 16.1 topntail—Remove Top and Bottom Lines from a File
      1. How It Works
    2. 16.2 flocate—Locate Files by Filename Alone
      1. How It Works
    3. 16.3 sus—Display a POSIX Man Page
      1. How It Works
    4. 16.4 cwbw—Count Words Beginning With
      1. How It Works
      2. Notes
    5. 16.5 cci—Configure, Compile, and Install from Tarball
      1. How It Works
      2. Notes
    6. 16.6 ipaddr—Find a Computer’s Network Address
      1. How It Works
    7. 16.7 ipaddr.cgi—Print the Remote Address of an HTTP Connection
      1. How It Works
    8. 16.8 iprev—Reverse the Order of Digits in an IP Address
      1. How It Works
      2. Notes
    9. 16.9 intersperse—Insert a String Between Doubled Characters
      1. How It Works
    10. 16.10 ll—Use a Pager for a Directory Listing Only If Necessary
      1. How It Works
    11. 16.11 name-split—Divide a Person’s Full Name into First, Last, and Middle Names
      1. How It Works
    12. 16.12 rot13—Encode or Decode Text
      1. How It Works
      2. Notes
    13. 16.13 showfstab—Show Information from /etc/fstab
      1. How It Works
      2. Notes
    14. 16.14 unique—Remove All Duplicate Lines from a File
      1. How It Works
    15. Summary
  25. Chapter 17 : Script Development Management
    1. 17.1 script-setup—Prepare the Scripting Environment
      1. How It Works
      2. Notes
    2. 17.2 cpsh—Install Script and Make Backup Copy
      1. How It Works
    3. 17.3 shgrep—Search Scripts for String or Regular Expression
      1. How It Works
    4. 17.4 shcat—Display a Shell Script
      1. How It Works
    5. Summary
  26. Appendix A: Internet Scripting Resources
    1. Introductions to Shell Scripting
    2. Intermediate and Advanced Scripting
    3. Collections of Scripts
    4. Home Pages for Shells
    5. Regular Expressions, sed, and awk
    6. Miscellaneous Pages
    7. History of the Shell
  27. Index

Product information

  • Title: Shell Scripting Recipes: A Problem-Solution Approach, Second Edition
  • Author(s): Chris F. A. Johnson, Jayant Varma
  • Release date: October 2015
  • Publisher(s): Apress
  • ISBN: 9781484202203