More reasons to love Scrapy

Scrapy has been around for more than half a decade, and is mature and stable. Beyond the performance benefits that we mentioned in the previous section, there are several other reasons to love Scrapy:

  • Scrapy understands broken HTML

    You can use Beautiful Soup or lxml directly from Scrapy, but Scrapy provides selectors—a higher level XPath (mainly) interface on top of lxml. It is able to efficiently handle broken HTML code and confusing encodings.

  • Community

    Scrapy has a vibrant community. Just have a look at the mailing list at!forum/scrapy-users and the thousands of questions in Stack Overflow at Most questions get answered within minutes. ...

Get Learning Scrapy now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.