In this chapter, we dived a bit deeper into the internals of Scrapy spiders. We used
FormRequest to log in, passed variables around with
Response, used relevant XPaths and
.csv files as sources, and much more.
We are now ready to see how we can deploy our spiders in the Scrapinghub cloud in the brief Chapter 6, Deploying to Scrapinghub before we move onto reviewing the wealth of Scrapy settings in Chapter 7, Configuration and Management.