O'Reilly logo
live online training icon Live Online training

Programming with Python: Beyond the Basics

How to Write a Web Scraper in Python

Topic: Software Development
Arianne Dee

This training is meant as a next step after Arianne’s Introduction to Python Programming Live Training course. In it, you will go beyond the basics and learn how to write useful programs in Python.

This is a hands-on course where you will build a web scraper in Python. It covers concepts like reading and writing to files, making HTTP requests, installing external libraries, and managing projects.

What you'll learn-and how you can apply it

  • How to read and write to CSV files
  • How to install libraries and make HTTP requests using the requests library
  • The basics of HTML and how to scrape a website

This training course is for you because...

  • You’re a novice Python developer who is ready to learn some intermediate concepts
  • You’re an intermediate/experienced developer who is new to Python
  • You’ve attended "Introduction to Python Programming" (live online training course) by Arianne Dee and want to learn more


  • Some experience coding in Python 2 or 3
  • Basic understanding of the following concepts in Python: types, functions, conditionals, lists, while loops, and for loops

Course Set-up

Recommended Preparation

Recommended Follow-up

About your instructor

  • Arianne is a freelance software developer and educator from the University of British Columbia and an instructor for Canada Learning Code and MakerLabs in Vancouver, BC.


The timeframes are only estimates and may vary according to how the class is progressing

Segment 1: Introduction, set-up, and review Length (45 min)

  • Installing Python and PyCharm
  • Reviewing Python Basics: conditionals, lists, and for loops
  • Break/Q&A (15 mins)

Segment 2: Intermediate concepts Length (25 min)

  • Dictionaries
  • Exceptions
  • Testing

Segment 3: Reading and Writing to Files Length (30 min)

  • Reading from files
  • Writing to files
  • CSV files
  • Q&A / Break (15 min)

Segment 4: Scraper overview Length (40 min)

  • Installing external libraries
  • Using the Requests library to make HTTP requests
  • Introduction to HTML
  • Using the Beautiful Soup library to parse HTML documents
  • Q&A / Break (10 min)

Segment 5: Scraping websites Length (50 min)

  • Scraping data
  • Saving to a CSV file
  • Refactoring

Segment 6: Further discussion Length (10 min)

  • How to deal with logins, JavaScript,
  • Course wrap-up and next steps