Learning Python Networking - Second Edition

Consuming web services in Python with urllib

In this section, we will learn how to use urllib and how we can build HTTP clients with this module.

The urllib module allows access to any resource published on the network (web page, files, directories, images, and so on) through various protocols (HTTP, FTP, SFTP). To start consuming a web service, we have to import the following libraries:

#! /usr/bin/env python3import urllib.requestimport urllib.parse

There are four functions in urllib:

• request: Opens and reads the request's URL
• error: Contains the errors generated by the request
• parse: A tool to convert the URL
• robotparse: Converts the robots.txt files

The urllib.request module allows access to a resource published on the internet through ...