Cilium: Up and Running

Chapter 11. Cluster Egress

Now that you know how to direct traffic inside your cluster, let’s focus on how to control packets as they leave.

The nature of containerized applications and Kubernetes networking can lead to a lack of visibility for engineers and operators needing to understand the origins of a packet. Indeed, by default, when a packet leaves the cluster, Cilium replaces the source IP address of the originating pod with the IP address of the node it runs on.

This feature, known as masquerading, is where we’ll start the chapter. We’ll explain why masquerading is needed, the two modes that are available (eBPF and iptables), and how to customize it. We’ll spend most of the chapter on Egress Gateway, a feature that provides a deterministic way to assign egress IP addresses and interfaces to traffic leaving the cluster. Finally, we’ll look at Cilium’s Bandwidth Manager, a lesser-known egress traffic control feature that allows you to rate-limit traffic from bandwidth-hungry pods.

You can find all the YAML manifests you will use in this chapter in the chapter11 directory of the book’s GitHub repository.

Let’s dig in!

Masquerading

One of the design principles of Kubernetes is to provide isolation for workloads across a shared infrastructure. But applications within a cluster don’t use the underlying network for this; instead, they use the internal pod network.

PodCIDR networks are typically allocated from RFC 1918 private address blocks and thus not routable (as you know ...