book

Automate the Boring Stuff with Python, 2nd Edition

by Al Sweigart

November 2019

Beginner to intermediate

592 pages

14h 43m

English

No Starch Press

Read now

Unlock full access

Whom Is This Book For?ConventionsWhat Is Programming?About This BookDownloading and Installing PythonDownloading and Installing MuStarting MuStarting IDLEThe Interactive ShellInstalling Third-Party ModulesHow to Find HelpAsking Smart Programming QuestionsSummary

Entering Expressions into the Interactive ShellThe Integer, Floating-Point, and String Data TypesString Concatenation and ReplicationStoring Values in VariablesYour First ProgramDissecting Your ProgramSummaryPractice Questions
Boolean ValuesComparison OperatorsBoolean OperatorsMixing Boolean and Comparison OperatorsElements of Flow ControlProgram ExecutionFlow Control StatementsImporting ModulesEnding a Program Early with the sys.exit() FunctionA Short Program: Guess the NumberA Short Program: Rock, Paper, ScissorsSummaryPractice Questions
def Statements with ParametersReturn Values and return StatementsThe None ValueKeyword Arguments and the print() FunctionThe Call StackLocal and Global ScopeThe global StatementException HandlingA Short Program: ZigzagSummaryPractice QuestionsPractice Projects
The List Data TypeWorking with ListsAugmented Assignment OperatorsMethodsExample Program: Magic 8 Ball with a ListSequence Data TypesReferencesA Short Program: Conway’s Game of LifeSummaryPractice QuestionsPractice Projects
The Dictionary Data TypePretty PrintingUsing Data Structures to Model Real-World ThingsSummaryPractice QuestionsPractice Projects
Working with StringsPutting Strings Inside Other StringsUseful String MethodsNumeric Values of Characters with the ord() and chr() FunctionsCopying and Pasting Strings with the pyperclip ModuleProject: Multi-Clipboard Automatic MessagesProject: Adding Bullets to Wiki MarkupA Short Progam: Pig LatinSummaryPractice QuestionsPractice Projects
Finding Patterns of Text Without Regular ExpressionsFinding Patterns of Text with Regular ExpressionsMore Pattern Matching with Regular ExpressionsGreedy and Non-greedy MatchingThe findall() MethodCharacter ClassesMaking Your Own Character ClassesThe Caret and Dollar Sign CharactersThe Wildcard CharacterReview of Regex SymbolsCase-Insensitive MatchingSubstituting Strings with the sub() MethodManaging Complex RegexesCombining re.IGNORECASE, re.DOTALL, and re.VERBOSEProject: Phone Number and Email Address ExtractorSummaryPractice QuestionsPractice Projects
The PyInputPlus ModuleProject: How to Keep an Idiot Busy for HoursProject: Multiplication QuizSummaryPractice QuestionsPractice Projects
Files and File PathsThe File Reading/Writing ProcessSaving Variables with the shelve ModuleSaving Variables with the pprint.pformat() FunctionProject: Generating Random Quiz FilesProject: Updatable Multi-ClipboardSummaryPractice QuestionsPractice Projects
The shutil ModuleWalking a Directory TreeCompressing Files with the zipfile ModuleProject: Renaming Files with American-Style Dates to European-Style DatesProject: Backing Up a Folder into a ZIP FileSummaryPractice QuestionsPractice Projects
Raising ExceptionsGetting the Traceback as a StringAssertionsLoggingMu’s DebuggerSummaryPractice QuestionsPractice Project
Project: mapIt.py with the webbrowser ModuleDownloading Files from the Web with the requests ModuleSaving Downloaded Files to the Hard DriveHTMLParsing HTML with the bs4 ModuleProject: Opening All Search ResultsProject: Downloading All XKCD ComicsControlling the Browser with the selenium ModuleSummaryPractice QuestionsPractice Projects
Excel DocumentsInstalling the openpyxl ModuleReading Excel DocumentsProject: Reading Data from a SpreadsheetWriting Excel DocumentsProject: Updating a SpreadsheetSetting the Font Style of CellsFont ObjectsFormulasAdjusting Rows and ColumnsChartsSummaryPractice QuestionsPractice Projects
Installing and Setting Up EZSheetsSpreadsheet ObjectsSheet ObjectsWorking with Google Sheets QuotasSummaryPractice QuestionsPractice Projects
PDF DocumentsProject: Combining Select Pages from Many PDFsWord DocumentsCreating PDFs from Word DocumentsSummaryPractice QuestionsPractice Projects
The csv ModuleProject: Removing the Header from CSV FilesJSON and APIsThe json ModuleProject: Fetching Current Weather DataSummaryPractice QuestionsPractice Project
The time ModuleRounding NumbersProject: Super StopwatchThe datetime ModuleReview of Python’s Time FunctionsMultithreadingProject: Multithreaded XKCD DownloaderLaunching Other Programs from PythonProject: Simple Countdown ProgramSummaryPractice QuestionsPractice Projects
Sending and Receiving Email with the Gmail APISMTPSending EmailIMAPRetrieving and Deleting Emails with IMAPProject: Sending Member Dues Reminder EmailsSending Text Messages with SMS Email GatewaysSending Text Messages with TwilioProject: “Just Text Me” ModuleSummaryPractice QuestionsPractice Projects
Computer Image FundamentalsManipulating Images with PillowProject: Adding a LogoDrawing on ImagesSummaryPractice QuestionsPractice Projects
Installing the pyautogui ModuleSetting Up Accessibility Apps on macOSStaying on TrackControlling Mouse MovementControlling Mouse InteractionPlanning Your Mouse MovementsWorking with the ScreenImage RecognitionGetting Window InformationControlling the KeyboardSetting Up Your GUI Automation ScriptsReview of the PyAutoGUI FunctionsProject: Automatic Form FillerDisplaying Message BoxesSummaryPractice QuestionsPractice Projects
The pip ToolInstalling Third-Party ModulesInstalling Modules for the Mu Editor
Running Programs from the Terminal WindowRunning Python Programs on WindowsRunning Python Programs on macOSRunning Python Programs on Ubuntu LinuxRunning Python Programs with Assertions Disabled
Chapter 1Chapter 2Chapter 3Chapter 4Chapter 5Chapter 6Chapter 7Chapter 8Chapter 9Chapter 10Chapter 11Chapter 12Chapter 13Chapter 14Chapter 15Chapter 16Chapter 17Chapter 18Chapter 19Chapter 20

Content preview from Automate the Boring Stuff with Python, 2nd Edition

12WEB SCRAPING

In those rare, terrifying moments when I’m without Wi-Fi, I realize just how much of what I do on the computer is really what I do on the internet. Out of sheer habit I’ll find myself trying to check email, read friends’ Twitter feeds, or answer the question, “Did Kurtwood Smith have any major roles before he was in the original 1987 RoboCop?”¹

Since so much work on a computer involves going on the internet, it’d be great if your programs could get online. Web scraping is the term for using a program to download and process content from the web. For example, Google runs many web scraping programs to index web pages for its search ...