Learn Kubernetes in a Month of Lunches

13 Centralizing logs with Fluentd and Elasticsearch

Applications generate lots of logs, which often aren’t very useful. As you scale up your apps across multiple Pods running in a cluster, it’s difficult to manage those logs using standard Kubernetes tooling. Organizations usually deploy their own logging framework, which uses a collect-and-forward model to read container logs and send them to a central store where they can be indexed, filtered, and searched. You’ll learn how to do that in this chapter using the most popular technologies in this space: Fluentd and Elasticsearch. Fluentd is the collector component, and it has some nice integrations with Kubernetes; Elasticsearch is the storage component and can run either as Pods in the cluster ...