Get full access to Hands-On Web Scraping with Python and 60K+ other titles, with a free 10-day trial of O'Reilly.

There are also live events, courses curated by job role, and more.

HTML <form> processing

In this section, we will be handling form processing or form submission in order to search for an activity from http://toscrape.com (ViewState). ViewState is an AJAX-based filter form.

This particular form submission is performed in multiple steps with the help of AJAX (https://www.w3schools.com/js/js_ajax_intro.asp). For more information on AJAX, please visit W3Schools AJAX:

http://toscrape.com with various endpoints in the Quotes section

Let's set up the code. The pyquery and requests libraries need to be imported and the required URLs need to be collected so that they can be used. The processRequests() function, ...

Get Hands-On Web Scraping with Python now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Don’t leave empty-handed

Get Mark Richards’s Software Architecture Patterns ebook to better understand how to design components—and how they should interact.

It’s yours, free.

Get it now

Check it out now on O’Reilly

Dive in for free with a 10-day trial of the O’Reilly learning platform—then explore all the other resources our members count on to build skills and solve problems every day.

Start your free trial Become a member now