Book description
Do amazing things with the shell
About This Book
Become an expert in creating powerful shell scripts and explore the full possibilities of the shell
Automate any administrative task you could imagine, with shell scripts
Packed with easy-to-follow recipes on new features on Linux, particularly, Debian-based, to help you accomplish even the most complex tasks with ease
Who This Book Is For
If you are a beginner or an intermediate Linux user who wants to master the skill of quickly writing scripts and automate tasks without reading the entire man pages, then this book is for you. You can start writing scripts and one-liners by simply looking at the relevant recipe and its descriptions without any working knowledge of shell scripting or Linux. Intermediate / advanced users, system administrators / developers, and programmers can use this book as a reference when they face problems while coding.
What You Will Learn
Interact with websites via scripts
Write shell scripts to mine and process data from the Web
Automate system backups and other repetitive tasks with crontab
Create, compress, and encrypt archives of your critical data.
Configure and monitor Ethernet and wireless networks
Monitor and log network and system activity
Tune your system for optimal performance
Improve your system's security
Identify resource hogs and network bottlenecks
Extract audio from video files
Create web photo albums
Use git or fossil to manage revision control and interact with FOSS projects
Create and maintain Linux containers and Virtual Machines
Run a private Cloud server
In Detail
The shell is the most powerful tool your computer provides. Despite having it at their fingertips, many users are unaware of how much the shell can accomplish.
Using the shell, you can generate databases and web pages from sets of files, automate monotonous admin tasks such as system backups, monitor your system's health and activity, identify network bottlenecks and system resource hogs, and more.
This book will show you how to do all this and much more.
This book, now in its third edition, describes the exciting new features in the newest Linux distributions to help you accomplish more than you imagine. It shows how to use simple commands to automate complex tasks, automate web interactions, download videos, set up containers and cloud servers, and even get free SSL certificates.
Starting with the basics of the shell, you will learn simple commands and how to apply them to real-world issues. From there, you'll learn text processing, web interactions, network and system monitoring, and system tuning.
Software engineers will learn how to examine system applications, how to use modern software management tools such as git and fossil for their own work, and how to submit patches to open-source projects.
Finally, you'll learn how to set up Linux Containers and Virtual machines and even run your own Cloud server with a free SSL Certificate from letsencrypt.org.
Style and approach
This book will take you through useful real-world recipes designed to make your daily life easier when working with the shell.
Table of contents
- Preface
-
Shell Something Out
- Introduction
- Displaying output in a terminal
- Using variables and environment variables
- Function to prepend to environment variables
- Math with the shell
- Playing with file descriptors and redirection
- Arrays and associative arrays
- Visiting aliases
- Grabbing information about the terminal
- Getting and setting dates and delays
- Debugging the script
- Functions and arguments
- Sending output from one command to another
- Reading n characters without pressing the return key
- Running a command until it succeeds
- Field separators and iterators
- Comparisons and tests
- Customizing bash with configuration files
-
Have a Good Command
- Introduction
- Concatenating with cat
- Recording and playing back terminal sessions
-
Finding files and file listing
- Getting ready
- How to do it...
-
There's more...
- Search based on name or regular expression match
- Negating arguments
- Searching based on the directory depth
- Searching based on file type
- Searching by file timestamp
- Searching based on file size
- Matching based on file permissions and ownership
- Performing actions on files with find
- Deleting based on file matches
- Executing a command
- Skipping specified directories when using the find command
- Playing with xargs
- Translating with tr
- Checksum and verification
- Cryptographic tools and hashes
- Sorting unique and duplicate lines
- Temporary file naming and random numbers
- Splitting files and data
- Slicing filenames based on extensions
- Renaming and moving files in bulk
- Spell–checking and dictionary manipulation
- Automating interactive input
- Making commands quicker by running parallel processes
- Examining a directory, files and subdirectories in it
-
File In, File Out
- Introduction
- Generating files of any size
- The intersection and set difference (A-B) on text files
- Finding and deleting duplicate files
- Working with file permissions, ownership, and the sticky bit
- Making files immutable
- Generating blank files in bulk
- Finding symbolic links and their targets
- Enumerating file type statistics
- Using loopback files
- Creating ISO files and hybrid ISO
- Finding the difference between files, and patching
- Using head and tail for printing the last or first 10 lines
- Listing only directories - alternative methods
- Fast command-line navigation using pushd and popd
- Counting the number of lines, words, and characters in a file
- Printing the directory tree
- Manipulating video and image files
-
Texting and Driving
- Introduction
- Using regular expressions
- Searching and mining text inside a file with grep
- Cutting a file column-wise with cut
- Using sed to perform text replacement
- Using awk for advanced text processing
- Finding the frequency of words used in a given file
- Compressing or decompressing JavaScript
- Merging multiple files as columns
- Printing the nth word or column in a file or line
- Printing text between line numbers or patterns
- Printing lines in the reverse order
- Parsing e-mail address and URLs from text
- Removing a sentence in a file containing a word
- Replacing a pattern with text in all the files in a directory
- Text slicing and parameter operations
-
Tangled Web? Not At All!
- Introduction
- Downloading from a web page
- Downloading a web page as plain text
- A primer on cURL
- Accessing unread Gmail e-mails from the command line
- Parsing data from a website
- Image crawler and downloader
- Web photo album generator
- Twitter command-line client
- Accessing word definitions via a web server
- Finding broken links in a website
- Tracking changes to a website
- Posting to a web page and reading the response
- Downloading a video from the Internet
- Summarizing text with OTS
- Translating text from the command line
-
Repository Management
- Introduction
- Creating a new git repository
- Cloning a remote git repository
- Adding and committing changes with git
- Creating and merging branches with git
- Sharing your work
- Pushing a branch to a server
- Checking the status of a git repository
- Viewing git history
- Finding bugs
- Tagging snapshots
- Committing message ethics
- Using fossil
- Creating a new fossil repository
- Cloning a remote fossil repository
- Opening a fossil project
- Adding and committing changes with fossil
- Using branches and forks with fossil
- Sharing your work with fossil
- Updating your local fossil repository
- Checking the status of a fossil repository
- Viewing fossil history
-
The Backup Plan
- Introduction
-
Archiving with tar
- Getting ready
- How to do it...
- How it works...
-
There's more...
- Appending files to an archive
- Extracting files and folders from an archive
- stdin and stdout with tar
- Concatenating two archives
- Updating files in an archive with a timestamp check
- Comparing files in the archive and filesystem
- Deleting files from the archive
- Compression with the tar archive
- Excluding a set of files from archiving
- Excluding version control directories
- Printing the total bytes
- See also
- Archiving with cpio
- Compressing data with gzip
- Archiving and compressing with zip
- Faster archiving with pbzip2
- Creating filesystems with compression
- Backing up snapshots with rsync
- Differential archives
- Creating entire disk images using fsarchiver
-
The Old-Boy Network
- Introduction
- Setting up the network
- Let us ping!
- Tracing IP routes
- Listing all available machines on a network
- Running commands on a remote host with SSH
- Running graphical commands on a remote machine
- Transferring files through the network
- Connecting to a wireless network
- Password-less auto-login with SSH
- Port forwarding using SSH
- Mounting a remote drive at a local mount point
- Network traffic and port analysis
- Measuring network bandwidth
- Creating arbitrary sockets
- Building a bridge
- Sharing an Internet connection
- Basic firewall using iptables
- Creating a Virtual Private Network
-
Put On the Monitors Cap
- Introduction
- Monitoring disk usage
- Calculating the execution time for a command
- Collecting information about logged in users, boot logs, and boot failures
- Listing the top ten CPU– consuming processes in an hour
- Monitoring command outputs with watch
- Logging access to files and directories
- Logging with syslog
- Managing log files with logrotate
- Monitoring user logins to find intruders
- Monitoring remote disk usage health
- Determining active user hours on a system
- Measuring and optimizing power usage
- Monitoring disk activity
- Checking disks and filesystems for errors
- Examining disk health
- Getting disk statistics
-
Administration Calls
- Introduction
- Gathering information about processes
- What's what – which, whereis, whatis, and file
- Killing processes, and sending and responding to signals
- Sending messages to user terminals
- The /proc filesystem
- Gathering system information
- Scheduling with a cron
- Database styles and uses
- Writing and reading SQLite databases
- Writing and reading a MySQL database from Bash
- User administration scripts
- Bulk image resizing and format conversion
- Taking screenshots from the terminal
- Managing multiple terminals from one
- Tracing the Clues
- Tuning a Linux System
- Containers, Virtual Machines, and the Cloud
Product information
- Title: Linux Shell Scripting Cookbook - Third Edition
- Author(s):
- Release date: May 2017
- Publisher(s): Packt Publishing
- ISBN: 9781785881985
You might also like
book
Linux Shell Scripting Cookbook - Second Edition
Don’t neglect the shell – this book will empower you to use simple commands to perform …
book
Linux Cookbook, 2nd Edition
This handy cookbook teaches new-to-intermediate Linux users the essential skills necessary to manage a Linux system, …
book
Linux Command Line and Shell Scripting Techniques
Practical and actionable recipes for using shell and command-line scripting on your Linux OS with confidence …
book
Mastering Linux Shell Scripting
Master the complexities of Bash shell scripting and unlock the power of shell for your enterprise …
