Mastering Functional Programming

Collecting information from the graph

Collecting information from a graph can be depicted as follows:

In the preceding illustration, the blue nodes are the ones currently being processed, the green ones are the ones that have already been processed, and the white ones are the ones not yet processed. This preceding task can be implemented as follows:

1. Specify which URL you would like to start from, that is, the starting node from which the traversal will start.

2. The crawler will visit the URL by issuing a GET request to it, and receive some HTML:

3. After receiving this HTML text, the crawler will extract the edges of the current node—the ...