O'Reilly logo

Learning Scrapy by Dimitrios Kouzis-Loukas

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Summary

The quality of markup continuously improves, and it's now much easier to create robust XPath expressions that extract data from HTML documents. In this chapter, you learned the basics of HTML documents and XPath expressions. You saw how to use Google Chrome to automatically get some XPath expressions as a starting point that we can later optimize. You also learned how to create such expressions directly by inspecting the HTML document, and how to tell a robust XPath expression from a less robust one. We are now ready to use all this knowledge to write our first few spiders with Scrapy in Chapter 3, Basic Crawling.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required