urllib

The urllib library is a standard Python package that collects several modules to work with HTTP-related communication models. Modules inside urllib are specially designed and contain functions and classes that deal with various types of client-server communication.

Similarly named packages also exist, like urllib2, an extensible library, and urllib3, a powerful HTTP client that addresses missing features from Python standard libraries.

Two of the most important urllib modules that deal with URL requests and responses are as follows. We will be using these modules in this and upcoming chapters:

  • urllib.request: Used for opening and reading URLs and requesting or accessing network resources (cookies, authentication, and so on)
  • urllib.response ...

Get Hands-On Web Scraping with Python now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.