Scrapy has so many settings that categorizing them becomes an urgent necessity. We start with the most essential settings that we summarize in the following diagram. They give you awareness of the important system features and you may find yourself adjusting them somewhat frequently.
By using those settings you can configure the way Scrapy provides performance and debugging information via logs, statistics, and the telnet facility.
Scrapy has different levels of logs based on their severity:
DEBUG (lowest level),
CRITICAL (highest level). Beyond that, there's a