Arista Warrior, 2nd Edition

Chapter 1. Why Arista?

If you’re reading this book, you have an interest in Arista products for any number of reasons. My goal is for you to understand why Arista is here, why it should be taken seriously, and why its products are selling like crazy. So let’s get started by explaining how it all began.

A Brief History of Arista

Arista Networks is a successful networking equipment company that’s been around only since 2004. It takes something special to succeed in an industry dominated by well-entrenched companies, many of which have been on top for decades. Certainly, a good product is needed, but that product and everything it takes to produce it comes from people. The people are what make Arista great.

Key Players

Please indulge me while I give you a quick tour of some of the key players at Arista, because having met many of them, I firmly believe that these people infect everyone around them with the same attitudes, excitement, and belief in what they’re doing. There are three people responsible for the creation of Arista Networks: Andy Bechtolsheim, David Cheriton, and Ken Duda.

Andy Bechtolsheim

Andy Bechtolsheim cofounded a company called Sun Microsystems in 1982. You might have heard of it. In 1995, he left Sun to found a company called Granite Systems. This new company made its mark by developing (then) state-of-the-art high-speed network switches. In 1995, Cisco acquired Granite Systems for a cool $220 million. With the sale, Andy became vice president and general manager of the Gigabit Systems Business Unit, where he stayed until 2003. He left Cisco in December of that year to found Kealia, Inc., with a Stanford professor named David Cheriton. Kealia was later acquired by Sun Microsystems, where Andy returned to the role of senior vice president and chief architect. In 2005, Andy cofounded Arastra, which later changed its name to Arista Networks.

Andy has an MS in computer engineering from Carnegie Mellon University and a PhD from Stanford University.

Andy Bechtolsheim is a multibillionaire Silicon Valley visionary. He has either designed or had a hand in the creation of some of the most significant computing and networking devices of the past 30 years. Andy and David Cheriton were the two initial investors in Google. Each of their $100,000 investments are now worth…well, let’s just say they made their money back and then some. I’ve had the pleasure of talking with Andy on multiple occasions, and I can tell you that he’s one of the smartest and most humble people I’ve ever met. I also stepped on his foot once before a speaking engagement, so I’ve got that going for me.

David Cheriton

David Cheriton is a Stanford University computer science professor who has an amazing knack for spotting and investing in successful startups. David cofounded Granite Systems with Andy Bechtolsheim, and the two have started other successful companies, including the aforementioned Kealia. David served as a technical advisor for Cisco for seven years and was the chief architect for the ASICs used in the Catalyst 4000s and 4500s. He has also served as a technical advisor for companies such as Sun, VMware, and Google. David is one of the original founders of Arastra, later renamed Arista Networks. He served as chief scientist for Arista until the IPO in 2014, at which point he left the company for reasons that are outside the scope of this book.

David has multiple inventions and patents to his name, has a PhD in computer science from the University of Waterloo, and has been at Stanford since 1981.

Given the track record of Andy and David and the fact that these two men funded the new company without any other investors, it would seem that Arista is destined for greatness, but the story doesn’t end here.

Ken Duda

Ken Duda is a founder, chief technology officer, and senior vice president of software engineering at Arista. Prior to founding Arastra (now Arista), Ken was CTO of There.com, where he designed a real-time 3-D distributed system that scaled to thousands of simultaneous users. I have no idea what that means, but it sure sounds cool.

Ken was the first employee of Granite Systems and, while working at Cisco, led the development of the Catalyst 4000 product line.

Ken has three simultaneous engineering degrees from MIT and a PhD in computer science from Stanford University.

Much of what you will read in this book about EOS is a result of Ken Duda’s vision. I met Ken while visiting Arista (along with many of the other people mentioned in this chapter), and within minutes, I realized that he was living the dream. Well, to be fair, maybe it was my dream, but what I saw was a seriously smart guy who knew the right way to do it and who had the freedom to do just that. I might be a hack writer now, but I went to school for programming (COBOL on punch cards, thank you very much) and loved being a programmer (we weren’t called developers back then). I gave up programming because I grew tired of having to fix other people’s crappy code. I wanted to write amazing new systems, but companies weren’t looking for that—they wanted grunts to fix their crappy code.

Ken not only gets to write the kind of code he likes, but he gets to design an entire networking equipment operating system from the ground up. When I first visited Arista, I drilled him with questions. Wouldn’t that delay delivery? Wouldn’t investors complain? Didn’t you ever get rushed into finishing something early to be first to market? As he answered my questions, it all started to become clear to me. There were no crazy investors demanding artificial deadlines. These guys had decided to do it the right way and not to deviate from that course. I also realized that everyone at Arista felt the same way. It was my meeting with Ken Duda that started the idea in my mind to write this book. Someone had to tell the world that companies like this could thrive, because in my almost 30 years in this industry, I can tell you that Arista is the first company I’ve seen that does it the right way.

Jayshree Ullal

The three founders certainly set the direction for Arista as a whole, but Jayshree keeps the place running. Jayshree Ullal is the president and CEO of Arista Networks. She was senior vice president at Cisco, where she was responsible for Data Center Switching and Services, including the Cisco Nexus 7000, the Catalyst 4500, and the Catalyst 6500 product lines. She was responsible for $10 billion in revenue, and reported directly to John Chambers, CEO of Cisco.

Jayshree has a BS in electrical engineering from San Francisco State University and an MS in engineering management from Santa Clara University.

Jayshree was named one of the “50 Most Powerful People” in 2005 by Network World Magazine and one of the “Top Ten Executives” at VMWorld in 2011. She has garnered many awards, including one of the 20 “Women to Watch in 2001” by Newsweek magazine. According to NewsIndiaTimes, she is the world’s first female Indian-American billionaire.

I can hear you now saying, “blah blah blah, I could read this on Wikipedia.” But consider this: Arista is a company peopled by mad scientists who just happen to work in legitimate jobs doing good work. Jayshree keeps them all in line and keeps the business not only humming but also prospering. Having managed teams and departments of both developers and engineers, I know what a challenge it can be. She makes it look easy.

All of these people are powerful forces in the networking and IT worlds, and all of them manage to make time to meet with prospective customers and even speak during classes held onsite at Arista. I’ve been in both situations and have seen this for myself.

You can read more about Arista and the management team at Arista’s website.

The Needs of a Data Center

So, what’s the big deal about data centers? Why do they need special switches anyway? Can’t we just use the same switches we use in the office? Hell, can’t we just go to Staples and buy some switches from Linksys, Netgear, or D-Link or something?

Believe it or not, I’ve had this very conversation on more than one occasion with executives looking to save some money on their data center builds. While it might be obvious to me, I quickly learned that it’s not apparent to everyone why data centers are unique.

Data centers are usually designed for critical systems that require high availability. That means redundant power, efficient cooling, secure access, and a pile of other things. But most of all, it means no single points of failure.

Every device in a data center should have dual power supplies, and each one of those power supplies should be fed from discrete power feeds. All devices in a data center should have front-to-back airflow, or ideally, airflow that can be configured front to back or back to front. All devices in a data center should support the means to upgrade, replace, or shut down any single chassis at any time without interruption to the often-extreme Service-Level Agreements (SLAs). In-Service Software Upgrades (ISSU) should also be available, but this can be circumvented by properly distributing load to allow meeting the prior requirement. Data center devices should offer robust hardware, even Network Equipment Building System (NEBS) compliance where required, and robust software to match.

Although data center switches should be able to deliver all of those features, they should also not be loaded down with features that are not desired in the data center. Examples of superfluous features might include Power Over Ethernet (PoE), backplane stacking, VoIP gateway features, wireless LAN controller functions, and other generally office-specific features.

Note

Note that this last paragraph greatly depends on what’s being housed in the data center. If the data center is designed to house all the IT equipment for a large office, then PoE and WAN Controllers might be desirable. Really, though, in a proper data center, those functions should be housed in proper dual-power-supply devices dedicated to the desired tasks.

Even though stacked switches seem like a great way to lower management points and increase port density, you might find that switches that support such features often don’t have the fabric speed or feature set to adequately support a data center environment. I’ve made a lot of money swapping out closet switches for data center–class switches in data centers. Data centers are always more resilient when using real data center equipment. If you don’t pay to put them in from the start, you’ll pay even more to swap them in later.

Data Center Networking

VMware really shook up the data center world with the introduction of vMotion. With vMotion, virtual machines (VMs) can be migrated from one physical box to another, without changing IP addresses and without bringing the server offline. I have to admit, that’s pretty cool.

The problem is that in order to accomplish this, the source and destination servers must reside in the same VLANs. That usually means having VLANs spanning across physical locations, which is just about the polar opposite of what we’ve spent the past 20 years trying to move away from!

Data center switches should support, or have the ability to support, at least a subset of data center–specific technologies. If your executive comes in and says that you need to support some new whizbang data center technology because he read about it in CIO magazine on the john that morning, having a data center full of closet switches will mean a rough conversation about how he bought the wrong gear.

The Case for Low Latency

Low latency might seem like a solution in need of a problem if you’re used to dealing with email and web servers, but in some fields, microseconds mean millions: millions of dollars, that is.

I talk about trading floors later on in this book, and some of Arista’s biggest customers use Arista switches in order to execute trades faster than their competitors. But think about other environments in which microseconds translate into tangible benefits—environments such as computer animation studios that can spend 80 to 90 hours rendering a single frame for a blockbuster movie, or scientific compute farms that might involve tens of thousands of compute cores. If the network is the bottleneck within those massive computer arrays, the overall performance is affected. And imagine the impact that an oversubscribed network might have on such farms. I’ve never had the pleasure of working in such environments, but I’ve taught people who do, and I can tell you that dropping packets is frowned upon.

Sure, those systems require some serious networking, but you might be surprised how much latency can affect more common applications. iSCSI doesn’t tolerate dropped packets well, nor does it tolerate a lot of buffering. Heck, even Network Attached Storage (NAS), which can tolerate dropped packets, is often used for systems and applications that do not tolerate latency well. Couple that with the way that most NAS are designed (many hosts to one filer), and things like buffering become a huge issue. Not only have I seen closet switches fail miserably in such environments, I’ve seen many data center–class switches fail, too.

In 2018, Arista acquired the company Metamako, which, according to the press release, is a leader in low-latency, field-programmable gate array (FPGA)-enabled network solutions. As we approach 2020, Arista still maintains its edge in the industry with respect to low-latency switching.

What About the Enterprise?

Arista is absolutely used in the enterprise, but before about 2018 Arista’s focus was mostly in the data center and before that in places like Wall Street. That is rapidly changing with things like the acquisition of Mojo Networks and the introduction of concepts like the Cognitive Campus, which I’ll cover in a bit.

Network-Based Storage

NAS was developed in the early 1980s as a means for university students to share porn between systems. OK, I totally made that up, but I’d be willing to bet that it was one of the first widespread uses of the technology. NAS protocols like Network File System (NFS) really were developed in the early 1980s, though, and although they’ve come a long way, they were not originally designed to be a solution for low-latency, high-throughput storage. Compared with more low-level solutions such as Fibre Channel, NAS tends to be slow and inefficient.

Still, NAS is comparatively inexpensive, it doesn’t require special hardware on the server side, and many vendors offer specialized NAS solutions aimed at centralizing storage needs for scores, if not hundreds, of servers. NAS is a reality in the modern data center, and the networks that NAS rides on must be robust, offer low latency, and, whenever possible, not drop packets. Even with nonblocking 10 Gb architectures, it can be easy to oversubscribe the 10 Gbps links to the NAS devices if many servers make simultaneous 10 Gbps reads or writes. Hell, in today’s networks, that can even be 100 Gbps, or 400 Gbps!

Automation

As I put the finishing touches on this edition, I find myself in the middle of 2019. For the past six years or so, I’ve been ranting to anyone who will listen that network engineers need to learn to code or I will script them out of a job. Read Chapter 30 to see why, but while you consider that, think about the fact that in my humble opinion, no other networking vendor supports automation better than Arista does. With an Arista switch out of the box, I can write Python scripts, Bash scripts, eAPI scripts, command-line interface (CLI) scripts (by using the interpreter directive [shebang], #!/usr/bin/Cli in Bash), and I can combine them in clever ways if need be. I can automate Arista builds using tools like Ansible, Chef, Puppet, and Arista’s own CloudVision.

Arista Delivers

So how does Arista deal with the requirements outlined in this chapter? Here’s a short list to whet your appetite. Each one of these topics is covered in detail within this book, so here I’ll just supply a list with a brief explanation of each feature and a reference to the chapter in which the topic is covered in more detail.

Hardware

Arista switches all have dual power supplies, hot-swappable and (usually) reversible airflow fans, completely nonblocking fabrics (even the eight- and twelve-slot chassis switches!), and merchant silicon. In almost every case, they are physically smaller, weigh less, consume less power, and often cost less than comparable switches from other manufacturers; although as you’ll come to learn, there really are no other switches that compare, though as the industry has moved to adopt merchant silicon, that is beginning to change. Sure, Arista makes great hardware, but the real difference is in the operating systems. See Chapter 6 for more information.

EOS

The Extensible Operating System (EOS) offers an industry-standard CLI while offering the power, flexibility, and expandability of Linux. Man, what a mouthful of marketing buzzwords that is. Let’s cut the BS and tell it like it is: EOS is Linux, with an industry-standard CLI. Actually, even that barely tells the whole story. Arista switches run Linux. They don’t run some stripped-down version of Linux that’s been altered beyond recognition—they run Linux. Some other vendors say that their operating system (OS) is based on Linux, and I guess it is, but on an Arista switch, you can drop down into the Bash shell and kill processes if you’re so inclined. Hell, you can even spawn another CLI session from Bash, write scripts that contain CLI commands, send email from CLI, pipe Bash commands through the CLI, and do a host of other exciting things, all because the switch runs Linux and because the developers care about one thing above all else: doing things the right way.

Arista hardware is amazing, but EOS makes these devices profoundly different than any other vendor’s offerings. See Chapters 8, 9, 10, 16, and many others for more information.

Bash

OK, so I blew the surprise with my EOS fan-boy ravings, but yes, you can issue the bash command from the CLI and enter the world of Linux. It’s not a Linux simulator either—it’s Bash, in Fedora Core Linux. You can even execute the sudo shutdown –r now command if you want, and you know you want to. All your other favorite Linux commands are there too: ps, top, grep, more, less, vi, cat, tar, gunzip, and python just to name a few. But not perl—unless you want to add it, in which case you can, because it’s Linux.

The fact that these switches run Linux is such a big deal that I recommend learning Linux to my clients when they’re considering Arista switches. Of course, the beauty of EOS is that you don’t need to know Linux, thanks to the CLI, but trust me when I say you’ll be able to get much more out of your Arista switches with some good Linux experience. The best part? Since writing the first edition, I now write training material for Arista, and we will be happy to teach you Linux in a class that’s been specially tailored for networking engineers. Actually, I’d probably have Adam Levin teach you that class given that he wrote it and he’s just that much better at it than I am. Linux is so important in the world of modern networking that I wouldn’t be surprised if Linux knowledge became a requirement in networking certifications moving forward. See Chapter 9 for more information.

SysDB

SysDB is one of the main features that makes EOS and Arista switches great. Simply put, SysDB is a database on the switch that holds all of the critical counters, status, and state information necessary for processes to run. The processes read and write this information to and/or from SysDB instead of storing it locally. If another process needs the information, it gets it from SysDB. Thus, processes never need to talk to each other; they communicate through SysDB. This dramatically lowers the possibility of one process negatively affecting another. Additionally, if a process dies, it can restart quickly without having to reinitialize all values, because it can read them all from SysDB. See Chapter 7 for more information.

Automation

In my humble opinion (some would say I’m incapable of a humble opinion), Arista leads the industry in automation capabilities. Because EOS is actually Linux, you can use tools like Chef, Puppet, and Ansible. Arista embraces the principles of DevOps and NetOps and has many ways of automating just about anything you can think of in EOS. You’ll see examples of me doing this in this book using one of my favorite Arista features, eAPI. You don’t need to learn about eAPI to automate EOS, though, because you can write Python and Bash scripts natively in Linux, and because there’s a Bash command called Cli that we cover later on, you can even script CLI commands in those programming languages. If you’re a hardcore developer, you can even write your own EOS agents using the EOS SDK. If you want to automate but you don’t like to code, you can use CloudVision, as well. See Chapters 15, 28, and 30 for more information.

CloudVision

CloudVision is Arista’s software solution for managing and maintaining your network. With its ability to centralize reporting, management, and even telemetry, not to mention doing things like rolling back the entire network configuration (as in hundreds of devices) to a previous point in time, CloudVision is a powerful tool that allows you to automate a litany of tasks without knowing how to code, all through the use of a web-based GUI. Even better, if you do know how to code, you can make CloudVision even more powerful by automating in a more customized fashion. With the addition of features like Bug Alerts, Macro-Segmentation, Topology Views, and a pile of other nice features, CloudVision can be a powerful solution for increased visibility and control of your network. See Chapter 15 for more information.

Cognitive Campus

In 2018, Arista acquired the WiFi company Mojo Networks, Inc., which was a big step toward Arista expanding into the enterprise space. With the addition of Cognative Wifi, PoE switches, and the cloud-controlled infrastructure, Mojo has changed the campus WiFi landscape by moving away from proprietary controllers and protocols.

In addition to the Mojo acquisition, CloudVision allows something Arista calls the Cognitive Management Plane, which promises to deliver many of the lessons learned from the data center into enterprise networks.

Although released too late in the publishing process to be included in this book, the Cognative Campus initiative looks like a game-changer to me. If Arista can change the way that enterprise networks operate the way it did with the data center, and I see no reason why it can’t, Arista once again stands to be the leader of that space, too. There are no chapters on this topic yet because, as I just mentioned, it was just coming into existence as this book was being finished.

My Personal Take

Before I worked at Arista, I was an independent consultant who moonlighted as a writer for no other reason than I liked to write. We made the trip out to Arista to get the executive briefing treatment, just like we had with the other big players at the time (around 2011, if I recall). I’d been in the networking world for a long time, and all those big players had pretty much annoyed me with their stories, “secret sauce,” and marketing double-speak. Arista was different, though.

When I asked another vendor about how a feature worked, I got a speech about how secret their sauce was (that’s not a metaphor—that actually happened). When I asked Ken Duda about how a feature worked, he said, “I didn’t write that code, let me get the person who did.” That person then came in, opened their laptop, and showed me the code. I was floored.

As someone with a background in programming, Linux, and networking, seeing all of those worlds coming together in a single place got me very excited in a way that the CCIE-level people sitting with me didn’t understand. This was game-changing! It was those experiences that made me want to work there and made me want to write the first edition of this book. I was a successful self-employed consultant. I hadn’t wanted to work for anyone but myself for years, maybe even decades; I’d been to Arista’s headquarters in California multiple times, and each time I left, I felt like I should have gone back and begged for a job. Finally, I did just that and had a whirlwind interview with Jayshree after which I was offered a job. There’s something special happening there, and the people I listed in the beginning of the chapter are at the heart of it.

Some seven or eight years later, I still see the same level of “wow” almost daily. Arista was a much smaller company then, and when I was hired in 2012, I was just shy of Arista’s 600th employee. Arista now has thousands of employees and continues to grow like crazy, and though all companies change as they grow, I’m still floored by some of the things I can do on an Arista switch. In fact, I joke with my students all the time that after they get used to the power of EOS, they’ll be frustrated and annoyed using anything else.

Now that I’ve been an employee for six years, I can still say that the executives at Arista constantly surprise me because they do things like respond well to logic. As someone who’s dealt with a lot of executives in my career, it’s my impression that most of them find politics and game-playing more important than technical or business acumen. I have never felt that from the Arista executive team. Not only that, but I routinely see C-level executives respond in technical threads internally with not only respectful grace and aplomb, but with a terrific application of deep and accurate technical knowledge. In a world in which I am convinced that most companies survive despite their own best efforts to the contrary, Arista manages to continue to impress me year after year. No company is perfect, and Arista isn’t, either, but it is easily the best place I’ve ever worked. It is my fervent hope that Arista can continue to impress because having been spoiled by this experience, I don’t think I could go back.