Hands-On Software Engineering with Golang

Inserting discovered outgoing links to the graph

The next crawler pipeline stage that we will be examining is the graph updater. Its main purpose is to insert newly discovered links into the link graph and create edges connecting them to the web page they were retrieved from. Let's take a look at the definition of the graphUpdater type and its constructor:

type graphUpdater struct {
    updater Graph
}
func newGraphUpdater(updater Graph) *graphUpdater {
    return &graphUpdater{
        updater: updater,
    }
}

The constructor expects an argument of the Graph type, which is nothing more than an interface describing the methods needed for the graph updater to communicate with a link graph component:

type Graph interface {
    UpsertLink(link *graph.Link) error ...