O'Reilly logo

Learning Scrapy by Dimitrios Kouzis-Loukas

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Deploying our spiders and scheduling runs

We will deploy directly from our dev machine. In order to do so, we just have to copy the lines from the Scrapy Deploy page (3) and put them on scrapy.cfg of our project, replacing the default [deploy] section. You will note that we don't need to set a password. As an example, we will use the properties project from Chapter 4, From Scrapy to a Mobile App. The reason that we use that spider is that we need our target data to be accessible from the Web exactly as we used to back in that chapter. Before we use it, we restore the original settings.py file by removing any references to the Appery.io pipeline:

Note

The code from this chapter is in the ch06 directory. This example in particular is in the ch06/properties ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required