Not every website is as welcoming to scraping as Wikipedia. There are many sites that will check to see if you're actually a web browser (or if you say that you are, at least) before sending you the site data. In this recipe, we will learn how to subvert this check (while making sure to comply with the Terms of Service) in order to get the desired data from a website.
Web servers can check which browser you are using by checking the HTTP header information you are sending with every request you make for a web page.
HTTP header information looks like this:
Host: www.google.com Connection: keep-alive User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.65 ...