The following diagram summarizes Scrapy's architecture:
You may notice three familiar types of objects upon which this architecture operates;
Items. Our spiders lie right at the core of the architecture. They create
Responses, and generate
Items and more
Item generated by a spider is postprocessed by a sequence of Item Pipelines using their
process_item() method. Typically,
Items and passes them to the subsequent pipelines by returning them. Occasionally (for example, in the case of a duplicate or invalid data), we ...