The best virtual networking is inspired on the Internet

A decent MPLS background should be part of the culture of any network engineer.

By Antonio Sánchez-Monge and Krzysztof Szarkowicz
February 9, 2016
Radar Dome Radar Dome (source: Pixabay)

If you are an IT professional or amateur, you have almost certainly experienced virtualized computing in one flavor or another (let it be virtual machines or containers). These virtual computing units provide an abstraction level over the underlying hardware. For example, you can easily deploy several VMs on a cluster of servers, and treat the servers as a pool of computing resources (memory, processing, storage, and so on). Is a similar approach possible in networking? Let’s see.

Overlays and Underlays

Interconnecting VMs in different network segments has traditionally relied on an explicit configuration of the underlying network devices (firewalls, switches, and routers); for example, by creating virtual LANs (VLANs) in legacy L2 underlays. Fortunately, this is changing. With Software Defined Networking (SDN), you can treat the underlay as a pool of network resources. VMs are interconnected through an overlay, which is basically an encapsulation over the underlying IP network.

Learn faster. Dig deeper. See farther.

Join the O'Reilly online learning platform. Get a free trial today and find answers on the fly, or master something new and useful.

Learn more

This model leverages L3 underlays with robust IP and IP/MPLS fabrics. So you can forget about L2 loops. Finally! As for the overlay, it can be either L2 or L3. Indeed, inside the overlay encapsulation you find the original user data, which may be a bare L3 packet or, why not, a full L2 frame.

Provisioning a new virtual network does not require any changes on the underlay configuration, at all: you just need to configure the overlay. This is a major breakthrough, especially taking into account that state-of-the-art SDN solutions offer an API to provision the overlay in an orchestrated manner.

How new is Network Virtualization?

Let’s step back for a moment. Is virtual networking a new concept? Not really. Virtual Private Networks (VPNs) have existed for decades in the WAN. Some VPN flavors are visible to end users (for example, certain IPSEC VPNs and SSL VPN), while other VPNs are invisible to the end users because the overlay encapsulation only lives deep inside the network. You might find surprising that “invisible VPNs” like Multiprotocol Label Switching (MPLS) VPNs are actually very widely deployed. Indeed, typical enterprise intranet sites are interconnected via MPLS VPNs, which are basically an overlay over the ISP’s backbone network. Thanks to a service identifier (a so-called MPLS service label) in the overlay encapsulation, ISPs can use the same underlay (backbone) to transport traffic of different customer intranets while also providing other services like residential or mobile Internet access. MPLS VPNs are invisible to the end user because the overlay encapsulation only takes place among ISP-owned devices called PEs (Provider Edge).

Now, moving from the WAN to a cloud or data center scenario, SDN overlays are also “invisible” to the VMs for the very same reasons: the encapsulation takes place among hypervisors (or between hypervisors and network devices), but it does not reach the VMs. With this powerful analogy in mind, does it make sense to reinvent the wheel? Or wouldn’t it be better to take advantage of an architecture that ISPs and networking vendors have been optimizing for decades? This is what our new book, MPLS in the SDN era, is mainly about. Paraphrasing Yakov Rekhter, one of the inventors of modern Internet:

Do not assume that being innovative, or being a technology leader, requires inventing new technologies. To the contrary, one can be quite innovative by simply combining creative use and packaging with high-quality implementation of existing technologies.

The MPLS Paradigm

In the SDN era, MPLS brings a paradigm which, combined with powerful orchestration techniques, results in best-in-breed solutions.

But, what is MPLS? In its original conception, MPLS is a technology that allows encoding one or more instructions as a set of packet headers inserted between the original packet and the underlay’s external header(s). It has many applications and VPNs are just one of them. Actually, ISPs typically use MPLS labels to tunnel Internet packets through their backbone, too. Yes, MPLS brought this blog post to your terminal (computer, tablet, or smartphone). Every piece of text, image, video, or sound that you get from the Internet undergoes a series of transformations in one or more ISPs before the packets reach their destination. These transformations include adding, swapping, and removing labels at the original IP packets. Most likely, you won’t see these labels if you take a packet capture at your terminal device. Indeed, packets are typically unlabeled when they exit an ISP, and this “invisibility” is one of the reasons why MPLS has been relatively unknown for nearly two decades.

However, what makes MPLS so powerful and flexible is its control plane, particularly when combined with the Border Gateway Protocol (BGP). This is how MPLS becomes an architectural model, rather than just an encapsulation. Even if the MPLS encapsulation is quite advantageous (stackable headers, each of four bytes only), the MPLS architecture also supports other encapsulations, like Virtual Extensible LAN (VxLAN).

In a nutshell, the MPLS architectural framework is based on:

  • Decoupling control plane from forwarding plane.
  • Decoupling service from transport.
  • Decoupling overlay from underlay.
  • Layered architecture with a feature-rich edge and a fast transport fabric (core).
  • Building overlay networks at the edge in order to support multitenancy and multiservice.
  • Providing mechanisms to aggregate (minimize) the forwarding state on the core.
  • Optionally, advanced packet steering by either signaling forwarding paths and/or by encoding a sequence of instructions on packet headers. This is not mandatory in single data centers, whose physical topologies are quite homogeneous and optimized (fabric Clos model) as compared to the geographically sparse ISP core networks.

It is hard to imagine a scalable network that does not follow most of the principles of the MPLS paradigm. Indeed, the first six bullet points are precisely some key characteristics of the industry-leading SDN solutions for data center and cloud. If on top of that you add flexible programmability and orchestration through an API, what you get is a modern SDN solution.

Despite its secondary role, it is only natural for networkers to pay attention to the encapsulation details. MPLS is becoming more and more popular-albeit often in disguise. For example, our book considers Ethernet VPN (EVPN) with Virtual eXtensible LAN (VXLAN) transport as a genuine MPLS technology. Even if it does not make use of MPLS encapsulation, this solution truly relies on fundamental aspects of the MPLS paradigm. Indeed, it uses the Border Gateway Protocol (BGP) and a label-like object (the VXLAN Network Identifier or VNI).

There are also emerging implementations of native MPLS in servers’ operating systems, a fast-growing trend in large-scale cloud providers, which leverage BGP for the underlay too. You can find these and many more details in the new book. Today, a decent MPLS background should be part of the culture of any network engineer. Our book aims at covering this gap from the very basics to the state of the art. There is a fast growing variety of MPLS features that networking vendors (and open-source communities) are developing to meet the requirements of a fast-changing market, and the new book tries to reflect this reality by also including technologies and use cases that are in their earliest life stage. The book also provides interoperable scenarios featuring Junos, IOS XR, and OpenContrail.

MPLS in the SDN Era

Looking at SDN and MPLS as competing technologies is fundamentally wrong. MPLS is a key SDN enabler. MPLS is a flexible technology that is not complex in itself. As any modular technology, it can become as complex as you want (or rather, as complex as the requirements are). MPLS was born for the Internet. It started small and continues to grow. Its initial goal was to solve a very specific challenge, but now it keeps evolving to meet many other requirements in terms of service, transport, flexibility, performance, resilience, resource management, prioritization, and so on. There is no more excuse to treat MPLS as a stranger. Time to close the gap!


Of course, a big thank you to all the contributors to MPLS in the SDN era, for making the book possible! This particular blog post has been greatly inspired by conversations with Michael Henkel, Javier Antich, and Raghu Subramanian.

Post topics: Infrastructure