Advanced Python Programming

HTTP requests

In a typical communication process on the web, HTML texts are the data that is to be saved and/or further processed. This data needs to be first collected from web pages, but how can we go about doing that? Most of the communication is done via the internet—more specifically, the World Wide Web—and this utilizes the Hypertext Transfer Protocol (HTTP). In HTTP, request methods are used to convey the information of what data is being requested and should be sent back from a server.

For example, when you type packtpub.com in your browser, the browser sends a request method via HTTP to the Packt website's main server asking for data from the website. Now, if both your internet connection and Packt's server are working well, then ...