book

The Software Developer's Guide to Linux

Name: The Software Developer's Guide to Linux
ISBN: 9781804616925

by David Cohen, Christian Sturm

January 2024

Beginner to intermediate

300 pages

6h 42m

English

Packt Publishing

Read now

Unlock full access

Preface
Who this book is forWhat this book is notWhat this book coversTo get the most out of this bookGet in touch
How the Command Line Works
In the beginning…was the REPLCommand-line syntax (read)Command line vs. shellHow does the shell know what to run? (evaluate)A quick definition of POSIXBasic command-line skillsUnix filesystem basicsAbsolute vs. relative file pathsAbsolute vs. relative pathname reviewOpening a terminalLooking around – command-line navigationpwd - print working directoryls - listMoving aroundcd – change directoryfind – find filesReading filesless – page through a fileMaking changestouch – create an empty file, or update modification time for an existing onemkdir – create a directoryrmdir – remove empty directoriesrm – remove files and directoriesmv – move or rename files and directoriesGetting helpShell autocompletionConclusion
Working with Processes
Process basicsWhat is a Linux process made of?Process ID (PID)Effective User ID (EUID) and Effective Group ID (EGID)Environment variablesWorking directoryPractical commands for working with Linux processesAdvanced process concepts and toolsSignalsPractical uses of signalsTrappingThe kill commandlsof – show file handles that a process has openInheritanceReview – example troubleshooting sessionConclusion
Service Management with systemd
The basicsinitProcesses and servicessystemctl commandsChecking the status of a serviceStarting a serviceStopping a serviceRestarting a serviceReloading a serviceEnable and disableA note on DockerConclusion
Using Shell History
Shell historyShell configuration filesHistory filesSearching through shell historyExceptionsExecuting previous commands with !Re-running a command with the same argumentsPrepending a command to something in your historyJumping to the beginning or end of the current lineConclusion
Introducing Files
Files on Linux: the absolute basicsPlaintext filesWhat is a binary file?Line endingsThe filesystem treeBasic filesystem operationslspwdcdtouchlesstailmvMovingRenamingcpmkdirrmEditing filesFile typesSymbolic linksHard linksThe file commandAdvanced file operationsSearching file content with grepFinding files with findCopying files between local and remote hosts with rsyncCombining find, grep, and rsyncAdvanced filesystem knowledge for the real worldFUSE: Even more fun with Unix filesystemsConclusion
Editing Files on the Command Line
NanoInstalling nanoNano cheat sheetFile handlingEditingSearch and replaceVi(m)Vi/vim commandsModesCommand modeNormal modeTips for learning vi(m)Use vimtutorThink in terms of mnemonicsAvoid using arrow keysAvoid using the mouseDon’t use gvimAvoid starting with extensive configuration or pluginsVim bindings in other softwareEditing a file you don’t have permissions forSetting your preferred editorConclusion
Users and Groups
What is a user?Root versus everybody elsesudoWhat is a group?Mini project: user and group managementCreating a userCreate a groupModifying a Linux userAdding a Linux user to a groupRemoving a user from a groupRemoving a Linux userRemove a Linux groupAdvanced: what is a user, really?User metadata / attributesA note on scriptabilityConclusion
Ownership and Permissions
Deciphering a long listingFile attributesFile typePermissionsNumber of hardlinksUser ownershipGroup ownershipFile sizeModification timeFilenameOwnershipPermissionsNumeric/octalCommon permissionsChanging ownership (chown) and permissions (chmod)ChownChange ownerChange owner and groupRecursively change owner and groupChmodUsing a referenceConclusion
Managing Installed Software
Working with software packagesUpdate your local cache of repository stateSearch for a packageInstall a packageUpgrade all packages that have available updatesRemove a package (and any dependencies, provided other packages don’t need them)Query installed packagesCaution required – curl | bashCompiling third-party software from sourceExample: compiling and installing htopInstall prerequisitesDownload, verify, and unarchive the source codeConfigure and compile htopConclusion

Configuring Software
Configuration hierarchyCommand-line argumentsEnvironment variablesConfiguration filesSystem-level configuration in /etc/User-level configuration in ~/.configsystemd unitsCreate your own serviceQuick note: configuration in DockerConclusion
Pipes and Redirection
File descriptorsWhat do these file descriptors reference?Input and output redirection (or, playing with file descriptors for fun and profit)Input redirection: <Output redirection: >Use >> to append output without overwritingError redirection with 2>Connecting commands together with pipes (|)Multi-pipe commandsReading (and building) complex multi-pipe commandsThe CLI tools you need to knowcutsortuniqCountingwcheadtailteeawksedPractical pipe patterns“Top X”, with countcurl | bashSecurity considerations for curl | sudo | bashFiltering and searching with grepgrep and tail for log monitoringfind and xargs for bulk file operationssort, uniq, and reverse numerical sort for data analysisawk and sort for reformatting data and field-based processingsed and tee for editing and backupps, grep, awk, xargs, and kill for process managementtar and gzip for backup and compressionAdvanced: inspecting file descriptorsConclusion
Automating Tasks with Shell Scripts
Why you need Bash scripting basicsBasicsVariablesSettingGettingBash versus other shellsShebangs and executable text files, aka scriptsCommon Bash settings (options/arguments)/usr/bin/envSpecial characters and escapingCommand substitutionTestingTesting operators[[ file and string testing ]]Useful operators for string testingUseful operators for file testing(( arithmetic testing ))Conditionals: if/then/elseifelseLoopsC-style loopsfor…inWhileVariable exportingFunctionsPrefer local variablesInput and output redirection<: input redirection> and >>: output redirectionUse 2>&1 to redirect STDERR and STDOUTVariable interpolation syntax – ${}Limitations of shell scriptsConclusionCitations
Secure Remote Access with SSH
Public key cryptography primerMessage encryptionMessage signingSSH keysExceptions to these rulesLogging in and authenticatingPractical project: Set up a key-based login to a remote serverStep 1: Open your terminal on the SSH client (not the server)Step 2: Generate the key pairStep 3: Copy the public key to your serverStep 4: Test it out!Converting SSH2 keys to the OpenSSH formatWhat we are trying to achieveHow to convert the SSH2-formatted key to OpenSSHThe other direction: Converting SSH2 keys to the OpenSSH formatSSH-agentCommon SSH errors and the -v (verbose) argumentFile transferSFTPSCPClever examplesWithout SFTP or SCPDirectory upload and .tar.gz compressionTunnelsLocal forwardingProxyingThe configuration fileConclusion
Version Control with Git
Some background on GitWhat is a distributed version control system?Git basicsFirst-time setupInitialize a new Git repositoryMake and see changesStage and commit changesOptional: add a remote Git repositoryPushing and pullingCloning a repositoryTerms you might come acrossRepositoryBare repositoryBranchMain/master branchHEADTagShallowMergingMerge commitMerge conflictStashPull requestCherry-pickingBisectingRebasingBest practices for commit messagesGood commit messagesGUIsUseful shell aliasesPoor man’s GitHubConsiderations1. Connect to your server2. Install Git3. Initialize a repository4. Clone the repository5. Edit the project and push your changesConclusion
Containerizing Applications with Docker
How containers work as packagesPrerequisite: Docker installDocker crash courseCreating images with a DockerfileContainer commandsdocker rundocker image listdocker psdocker execdocker stopDocker project: Python/Flask application container1. Set up the application2. Create the Docker image3. Start a container from your imageContainers vs. virtual machinesA quick note on Docker image repositoriesPainfully learned container lessonsImage sizeC standard libraryProduction is not your laptop: outside dependenciesContainer theory: namespacingHow do we do Ops with containers?Conclusion
Monitoring Application Logs
Introduction to loggingLogging on Linux can get... weirdSending log messagesThe systemd journalExample journalctl commandsFollowing active logs for a unitFiltering by timeFiltering for a specific log levelInspecting logs from a previous bootKernel messagesLogging in Docker containersSyslog basicsFacilitiesSeverity levelsConfiguration and implementationsTips for loggingKeywords when using structured loggingSeverityCentralized loggingConclusion
Load Balancing and HTTP
Basic terminologyGatewayUpstreamCommon misunderstandings about HTTPHTTP statusesDon’t just check for 200 OK404 Not Found502 Bad Gateway503 Service Unavailable504 Gateway TimeoutIntroduction to curl: checking HTTP response statusHTTP headersCase-insensitive headersCustom headersViewing HTTP headers with curlHTTP versionsHTTP/0.9HTTP/1.0 and HTTP/1.1HTTP/2HTTP/3 and QUICLoad balancingSticky sessions, cookies, and deterministic hashingRound-robin load balancingOther mechanismsHigh availabilityTroubleshooting redirects with curlUsing curl as an API testing toolAccepting and displaying bad TLS certificates with curlCORSConclusion
Other Books You May Enjoy
Index

Content preview from The Software Developer's Guide to Linux

12 Automating Tasks with Shell Scripts

Sometimes, you’ll find yourself repeating the same few commands over and over, perhaps with slight variations. You reach the point of frustration, and say, “That’s it; I’m scripting this.” Being a CLI wizard, you do the following:

Run tail -n 20 ~/.bash_history > myscript.sh to create a file that contains the last 20 Bash commands you ran.
Then run bash myscript.sh to execute it.

Although this isn’t the recommended procedure (we’ll get to that in this chapter), it’s a perfectly valid way to create and run a Bash script.

This chapter is a Bash scripting crash course. Like any programming crash course, it is completely useless unless you actually follow along, type in all the code yourself, and run it in ...

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

O’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.

Julian F.

Head of Cybersecurity

I wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.

Addison B.

Field Engineer

I’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.

Amir M.

Data Platform Tech Lead

I'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.

Mark W.

Embedded Software Engineer

A Practical Guide to Linux Commands, Editors, and Shell Programming, Fourth Edition

Publisher Resources

ISBN: 9781804616925

Cloud Computing

Data Engineering

Data Science

AI & ML

Programming Languages

Software Architecture

IT/Ops

Security

Design

Business

Soft Skills

The Software Developer's Guide to Linux

by David Cohen, Christian Sturm

12

Automating Tasks with Shell Scripts

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.