Hands-On Software Engineering with Golang

Deploying the web crawler

The crawler service deployment will use the same partition detection logic that the monolithic implementation from the previous chapter did. Consequently, we will be deploying two instances of the crawler service as a Kubernetes Stateful set, which guarantees that each pod will be assigned a predictable hostname that includes the pod's ordinal in the set.

In addition, we will be creating a headless Kubernetes service that will populate the SRV records for the crawler pods and allow the partition detection code to query the total number of available pods.