Python Web Scraping Cookbook

Getting ready

The first thing that we will do is add a scraping function to the sojobs module. This function will be named get_job_listing_skills. The following is the code for this function:

def get_job_listing_skills(job_listing_id):    print("Got a request for a job listing skills with id: " + job_listing_id)    req = requests.get("https://stackoverflow.com/jobs/" + job_listing_id)    content = req.text    bs = BeautifulSoup(content, "lxml")    script_tag = bs.find("script", {"type": "application/ld+json"})    job_listing_contents = json.loads(script_tag.contents[0])    skills = job_listing_contents['skills']    return json.dumps(skills)

This function retrieves the job listing, extracts the JSON provided by StackOverflow, and then only returns the skills property ...