The following code snippet represents the constructor of the crawler class. It initializes all the relevant instance variables. logger is one of the custom classes written to log debug messages, so that if any error occurs during the execution of the crawler, which will have been spawned as a subprocess and will be running in the background, it can be debugged:
Let's now take a look at the start() method of the crawler, from where the crawling actually begins:
It can be seen in section (1), which will be true ...
