The Login form

The first form we'll automate is the Login form, which is available at http://example.webscraping.com/user/login. To understand the form, we can use our browser development tools. With the full version of Firebug or Chrome Developer Tools, it is possible to simply submit the form and check what data was transmitted in the Network tab (similar to how we did in Chapter 5, Dynamic Content). However, we can also see information about the form if we use "Inspect Element" features:

The important parts regarding how to send the form are the action, enctype, and method attributes of the form tag, and the two input fields (in the above ...

Get Python Web Scraping - Second Edition now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.