Mastering Python for Networking and Security

Customizing requests with urllib2

We can customize a request to retrieve a specific version of a website. For this task, we can use the Accept-Language header, which tells the server our preferred language for the resource it returns.

In this section, we are going to see how to add our own headers using the User-Agent header. User-Agent is a header used to identify the browser and operating system that we are using to connect to that URL. By default, urllib2 is identified as "Python-urllib / 2.5"; if we wanted to identify ourselves, for example, as a Chrome browser, we could redifine the headers parameter.

In this example, we create the same GET request using the Request class by passing as parameter a custom HTTP User-Agent header:

You can ...