Scraping information from websites

Governments or jurisdictions around the world are increasingly embracing the importance of open data, which aims to increase citizen involvement and informed decision-making, and also aims to make policies more open to public scrutiny. Some examples of open data initiatives around the world include https://www.data.gov/ (United States of America), https://data.gov.uk/ (United Kingdom), and https://data.gov.hk/en/ (Hong Kong).

These data portals often provide an API for programmatic access of data. However, an API is not available for some datasets, hence we need to rely on good old web scraping techniques to extract information from websites.

Beautiful Soup (https://www.crummy.com/software/BeautifulSoup/ ...

Get Matplotlib for Python Developers now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.