March 2019
Beginner
490 pages
12h 40m
English
Before starting Scrapy, is recommended that you modify the settings and limit the speed at which the data is accessed, so as not to create a DOS attack. For doing this task, we need to configure settings.py with the DOWNLOAD_DELAY property:
# Scrapy settings for scrapy project## For simplicity, this file contains only the most important settings by# default. All the other settings are documented here:## http://doc.scrapy.org/en/latest/topics/settings.html#BOT_NAME = 'hacker_news'SPIDER_MODULES = ['hacker_news.spiders']NEWSPIDER_MODULE = 'hacker_news.spiders'# Configure a delay for requests for the same website (default: 0)# See http://scrapy.readthedocs.org/en/latest/topics/settings.html#download-delayDOWNLOAD_DELAY = 60
Read now
Unlock full access