Mastering Python Scripting for System Administrators

Extracting information from Wikipedia

In this section, we are going to see an example of a list of dance forms from Wikipedia. We are going to list all classical Indian dances. For that, create a extract_from_wikipedia.py script and write the following content in it:

import requestsfrom bs4 import BeautifulSouppage_result = requests.get('https://en.wikipedia.org/wiki/Portal:History')parse_obj = BeautifulSoup(page_result.content, 'html.parser')h_obj = parse_obj.find(class_='hlist noprint')h_obj_a_content = h_obj.find_all('a')print(h_obj)print(h_obj_a_content)

Run the script and you will get the following output:

student@ubuntu:~/work$ python3 extract_from_wikipedia.pyOutput:<div class="hlist noprint" id="portals-browsebar" style="text-align: ...