book

Shell Scripting Recipes: A Problem-Solution Approach, Second Edition

Name: Shell Scripting Recipes: A Problem-Solution Approach, Second Edition
ISBN: 9781484202203

by Chris F. A. Johnson, Jayant Varma

October 2015

Intermediate to advanced

380 pages

7h 5m

English

Apress

Read now

Unlock full access

Cover
Title
Copyright
Dedication
Contents at a Glance
Contents
About the Authors
Acknowledgments
Chapter 1 : The POSIX Shell and Command-Line Utilities
Shell CommandsechoprintfsetshifttypegetoptscaseevallocalParameters and VariablesPositional ParametersSpecial Parametersstandard-vars—A Collection of Useful VariablesPatternsPathname ExpansionRegular ExpressionsParameter ExpansionThe Bourne Shell ExpansionsPOSIX Parameter ExpansionsShell-Specific Expansions, bash2, and ksh93Shell ArithmeticAliasesSourcing a FileFunctionsFunctions Are FastCommand Substitution Is SlowUsing the Functions in This Bookstandard-funcs: A Collection of Useful Commands1.1 get_key—Get a Single Keystroke from the User1.2 getline—Prompt User to Enter a Line1.3 press_any_key—Prompt for a Single Keypress1.4 menu1—Print a Menu and Execute a Selected Command1.5 arg—Prompt for Required Argument If None Supplied1.6 die—Print Error Message and Exit with Error Status1.7 show_date—Display Date in D[D] MMM YYYY Format1.8 date_vars—Set Date and Time Variables1.9 is_num—Is This a Positive Integer?1.10 abbrev_num—Abbreviate Large Numbers1.11 commas—Add Thousands Separators to a Number1.12 pr1—Print Arguments, One to a Line1.13 checkdirs—Check for Directories; Create If Necessary1.14 checkfiles—Check That a Directory Contains Certain Files1.15 zpad—Pad a Number with Leading Zeroes1.16 cleanup—Remove Temporary Files and Reset Terminal on ExitThe Unix Utilitiescat: Concatenate Files to the Standard Outputsed: A Text Stream Editorawk: Pattern Scanning and Processing Languagegrep: Print Lines Matching a Regular Expressiondate: Show or Set the System Datetr: A Character Translation Utilitywc: Count Characters,Words, and Lines in a Filefile: Determine the File Typels: Sort and Provide Details About Filesuniq: Remove Consecutive Duplicate Linessudo: Execute Commands as the Superusersplit: Divide a File into Equal-Sized Pieceswhich: Show the Full Path to a Commandgs, gv: Render, Convert, or View PostScript and PDF FilesSummary
Chapter 2 : Playing with Files: Viewing, Manipulating, and Editing Text Files
End of the Line for OS Differences: Converting Text Files2.1 dos2unix—Convert Windows Text Files to Unix2.2 unix2dos—Convert a Unix File to Windows2.3 mac2unix—Convert Macintosh Files to Unix2.4 unix2mac—Convert Unix Files to Mac Format2.5 dos2mac—Convert Windows Files to Macintosh2.6 mac2dos—Convert Macintosh Files to WindowsDisplaying Files2.7 prn—Print File with Line Numbers2.8 prw—Print One Word per Line2.9 wbl—Sort Words by LengthFormatting File Facts2.10 finfo—File Information Display2.11 wfreq—Word Frequency2.12 lfreq—Letter Frequency2.13 fed—A Simple Batch File EditorSummary

Chapter 3 : String Briefs
Character Actions: The char-funcs Library3.1 chr—Convert a Decimal Number to an ASCII Character3.2 asc—Convert a Character to Its Decimal Equivalent3.3 nxt—Get the Next Character in ASCII Sequence3.4 upr—Convert Character(s) to Uppercase3.5 lwr—Convert Character(s) to LowercaseString Cleaning: The string-funcs Library3.6 sub—Replace First Occurrence of a Pattern3.7 gsub—Globally Replace a Pattern in a String3.8 repeat—Build a String of a Specified Length3.9 index, rindex—Find Position of One String Within Another3.10 substr—Extract a Portion of a String3.11 insert_str—Place One String Inside AnotherSummary
Chapter 4 : What’s in a Word?
Finding and Massaging Word Listswf-funcs: WordFinder Function Library4.1 write_config—Write User’s Information to the Configuration File4.2 do_config—Check For and Source Default Configuration File4.3 set_sysdict—Select the Dictionary Directory4.4 mkwsig—Sort Letters in a Word4.5 wf-clean—Remove Carriage Returns and Accents4.6 wf-compounds—Squish Compound Words and Save with Lengths4.7 wf-setup—Prepare Word and Anagram ListsPlaying with Matches4.8 wf—Find Words That Match a Pattern4.9 wfb—Find Words That Begin with a Given Pattern4.10 wfe—Find Words That End with a Given Pattern4.11 wfc—Find Words That Contain a Given Pattern4.12 wfit—Find Words That Fit Together in a Grid4.13 anagram—Find Words That Are Anagrams of a Given Word4.14 aplus—Find Anagrams of a Word with a Letter Added4.15 aminus—Remove Each Letter in Turn and Anagram What’s LeftSummary
Chapter 5 : Scripting by Numbers
The math-funcs Library5.1 fpmul—Multiply Decimal Fractions5.2 int—Return the Integer Portion of a Decimal Fraction5.3 round—Round the Argument to the Nearest Integer5.4 pow—Raise a Number to Any Given Power5.5 square—Raise a Number to the Second Power5.6 cube—Raise a Number to the Third Power5.7 calc—A Simple Command-Line CalculatorAdding and Averaging5.8 total—Add a List of Numbers5.9 mean—Find the Arithmetic Mean of a List of Numbers5.10 median—Find the Median of a List of Numbers5.11 mode—Find the Number That Appears Most in a List5.12 range—Find the Range of a Set of Numbers5.13 stdev—Finding the Standard DeviationConverting Between Unit Systems5.14 conversion-funcs—Converting Metric Units5.15 conversion—A Menu System for Metric ConversionSummary
Chapter 6 : Loose Names Sink Scripts: Bringing Sanity to Filenames
What’s in a Name?POSIX Portable FilenamesOK FilenamesFunctioning Filenames: The filename-funcs Library6.1 basename—Extract the Last Element of a Pathname6.2 dirname—Return All but the Last Element of a Pathname6.3 is_pfname—Check for POSIX Portable Filename6.4 is_OKfname—Check Whether a Filename Is Acceptable6.5 pfname—Convert Nonportable Characters in Filename6.6 OKfname—Make a Filename Acceptable6.7 is_whitespc—Does the Filename Contain Whitespace Characters?6.8 whitespc—Fix Filenames Containing Whitespace Characters6.9 is_dir—Is This a Directory I See Before Me?6.10 nodoublechar—Remove Duplicate Characters from a String6.11 new_filename—Change Filename to Desired Character Set6.12 fix_pwd—Fix All the Filenames in the Current Directory6.13 fixfname—Convert Filenames to Sensible NamesSummary
Chapter 7 : Treading a Righteous PATH
The path-funcs Library7.1 path—Display a User’s PATH7.2 unslash—Remove Extraneous Slashes7.3 checkpath—Clean Up the PATH Variable7.4 addpath—Add Directories to the PATH Variable7.5 rmpath—Remove One or More Directories from $PATHSummary
Chapter 8 : The Dating Game
The date-funcs Library8.1 split_date—Divide a Date into Day, Month, and Year8.2 is_leap_year—Is There an Extra Day This Year?8.3 days_in_month—How Many Days Hath September?8.4 Julian Dates8.5 dateshift—Add or Subtract a Number of Days8.6 diffdate—Find the Number of Days Between Two Dates8.7 day_of_week—Find the Day of the Week for Any Date8.8 display_date—Show a Date in Text Format8.9 parse_date—Decipher Various Forms of Date String8.10 valid_date—Where Was I on November 31st?Summary
Chapter 9 : Good Housekeeping: Monitoring and Tidying Up File Systems
9.1 dfcmp—Notify User of Major Changes in Disk UsageHow It Works9.2 symfix—Remove Broken Symbolic LinksHow It Works9.3 sym2file—Converts Symbolic Links to Regular FilesHow It Works9.4 zrm—Remove Empty FilesHow It Works9.5 undup—Remove Duplicate FilesHow It Works9.6 lsr—List the Most Recent (or Oldest) Files in a DirectoryHow It WorksSummary
Chapter 10 : Screenplay: The screen–funcs Library
10.1 screen-vars—Variables for Screen ManipulationHow It Works10.2 set_attr—Set Screen-Printing AttributesHow It Works10.3 set_fg, set_bg, set_fgbg—Set Colors for Printing to the ScreenHow It Works10.4 cls—Clear the ScreenHow It WorksNotes10.5 printat—Position Cursor by Row and ColumnHow It WorksNotes10.6 put_block_at—Print Lines in a Column Anywhere on the ScreenHow It Works10.7 get_size—Set LINES and COLUMNS VariablesHow It Works10.8 max_length—Find the Length of the Longest ArgumentHow It Works10.9 print_block_at—Print a Block of Lines Anywhere on the ScreenHow It Works10.10 vbar, hbar—Print a Vertical or Horizontal BarHow It Works10.11 center—Center a String on N ColumnsHow It Works10.12 flush_right—Align String with the Right MarginHow It Works10.13 ruler—Draw a Ruler Across the Width and Height of the WindowHow It Works10.14 box_block, box_block_at—Print Text Surrounded by a BoxHow It Works10.15 clear_area, clear_area_at—Clear an Area of the ScreenHow It Works10.16 box_area, box_area_at—Draw a Box Around an AreaHow It Works10.17 screen-demo—Saving and Redisplaying Areas of the ScreenHow It WorksSummary
Chapter 11 : Aging, Archiving, and Deleting Files
11.1 date-file—Add a Datestamp to a FilenameHow It WorksNotes11.2 rmold—Remove Old FilesHow It Works11.3 keepnewest—Remove All but the Newest or Oldest FilesHow It WorksSummary
Chapter 12 : Covering All Your Databases
12.1 lookup—Find the Corresponding Value for a KeyHow It Worksshdb-funcs: Shell Database Function Library12.2 load_db—Import Database into Shell ArrayHow It WorksNotes12.3 split_record—Split a Record into FieldsHow It Works12.4 csv_split—Extract Fields from CSV RecordsHow It Works12.5 put_record—Assemble a Database Record from an ArrayHow It Works12.6 put_csv—Assemble Fields into a CSV RecordHow It Works12.7 db-demo—View and Edit a Password FileHow It WorksPhoneBase: A Simple Phone Number DatabaseHow It Works12.8 ph—Look Up a Phone Number12.9 phadd—Add an Entry to PhoneBase12.10 phdel—Delete an Entry from PhoneBase12.11 phx—Show ph Search Results in an X WindowSummary
Chapter 13 : Home on the Web
Playing with Hypertext: The html-funcs Library13.1 get_element—Extract the First Occurrence of an Element13.2 split_tags—Put Each Tag on Its Own Line13.3 html-title—Get the Title from an HTML FileHTML on the Fly: The cgi-funcs Library13.4 x2d2—Convert a Two-Digit Hexadecimal Number to Decimal13.5 dehex—Convert Hex Strings (%XX) to Characters13.6 filedate—Find and Format the Modification Date of a FileCreating HTML Files13.7 mk-htmlindex—Create an HTML Index13.8 pretext—Create a Wrapper Around a Text File13.9 text2html—Convert a Text File to HTML13.10 demo.cgi—A CGI ScriptSummary
Chapter 14 : Taking Care of Business
14.1 prcalc—A Printing CalculatorHow It Works14.2 gle—Keeping Records Without a ShoeboxHow It WorksSummary
Chapter 15 : Random Acts of Scripting
The rand-funcs Library15.1 random—Return One or More Random Integers in a Given RangeHow It WorksNotes15.2 toss—Simulate Tossing a CoinHow It Works15.3 randstr—Select a String at RandomHow It WorksA Random Sampling of Scripts15.4 rand-date—Generate Random Dates in ISO FormatHow It WorksNotes15.5 randsort—Print Lines in Random OrderHow It Works15.6 randomword—Generate Random Words According to Format SpecificationsHow It Works15.7 dice—Roll a Set of DiceHow It Works15.8 throw—Throw a Pair of DiceHow It WorksSummary
Chapter 16 : A Smorgasbord of Scripts
16.1 topntail—Remove Top and Bottom Lines from a FileHow It Works16.2 flocate—Locate Files by Filename AloneHow It Works16.3 sus—Display a POSIX Man PageHow It Works16.4 cwbw—Count Words Beginning WithHow It WorksNotes16.5 cci—Configure, Compile, and Install from TarballHow It WorksNotes16.6 ipaddr—Find a Computer’s Network AddressHow It Works16.7 ipaddr.cgi—Print the Remote Address of an HTTP ConnectionHow It Works16.8 iprev—Reverse the Order of Digits in an IP AddressHow It WorksNotes16.9 intersperse—Insert a String Between Doubled CharactersHow It Works16.10 ll—Use a Pager for a Directory Listing Only If NecessaryHow It Works16.11 name-split—Divide a Person’s Full Name into First, Last, and Middle NamesHow It Works16.12 rot13—Encode or Decode TextHow It WorksNotes16.13 showfstab—Show Information from /etc/fstabHow It WorksNotes16.14 unique—Remove All Duplicate Lines from a FileHow It WorksSummary
Chapter 17 : Script Development Management
17.1 script-setup—Prepare the Scripting EnvironmentHow It WorksNotes17.2 cpsh—Install Script and Make Backup CopyHow It Works17.3 shgrep—Search Scripts for String or Regular ExpressionHow It Works17.4 shcat—Display a Shell ScriptHow It WorksSummary
Appendix A: Internet Scripting Resources
Introductions to Shell ScriptingIntermediate and Advanced ScriptingCollections of ScriptsHome Pages for ShellsRegular Expressions, sed, and awkMiscellaneous PagesHistory of the Shell
Index

Overview

Shell Scripting Recipes is filled with over 150 much-needed and practical recipes that follow a problem-solution format, and help all Unix users regain some of the lost time spent creating and testing shell scripts. Most scripts in this book are POSIX (Portable Operating System Interface)-compliant, so they are supported by many of the major shell variants, including Bash, ksh, and sh, among others. This completely updated second edition takes into account the many changes to shell scripting that have occurred since its original publication nearly ten years ago, and all the recipes are now relevant for a modern audience.

Each real-world example recipe follows the same structure and easily shows you what's going on in each case. File conversion (DOS, UNIX, and Mac), system administration, and resource monitoring are just a few of the topics covered in this highly practical shell scripting reference. You will also find recipes for filename management, complex date calculations, screen control capabilities, and more.

Author Chris Johnson's code is clear, direct, and applicable, and he explains everything well in each recipe so that you understand not just how to do something, but why to do it that way. Shell Scripting Recipes is an invaluable book and one to add to your library today.

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

Publisher Resources

ISBN: 9781484202203Purchase book

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills