If you’re reading this book, you’ve got an interest in Arista products for any number of reasons. My goal is for you to understand why Arista is here, why they should be taken seriously, and why their switches are selling like crazy. So let’s get started by explaining how it all began.
Arista Networks is a successful networking equipment company that’s only been around since 2005. It takes something special to succeed in an industry dominated by well-entrenched companies, many of which have been on top for decades. Certainly a good product is needed, but that product and everything it takes to produce it comes from people. The people are what make Arista great. Please indulge me while I give you a quick tour of some of the key players at Arista, because having met many of them, I firmly believe that these people infect everyone around them with the same attitudes, excitement, and belief in what they’re doing.
There are three people responsible for the creation of Arista Networks: Andy Bechtolsheim, David Cheriton, and Ken Duda. Allow me to explain who these people are, so that you might get an idea of what sort of company Arista is.
Andy Bechtolsheim co-founded a company called Sun Microsystems in 1982. You may have heard of them. In 1995, he left Sun to found a company called Granite Systems. This new company made its mark by developing (then) state-of-the art high-speed network switches. In 1995, Cisco acquired Granite Systems for a cool $220 million. With the sale, Andy became Vice President and General Manager of the Gigabit Systems Business Unit, where he stayed until 2003. He left Cisco in December of that year to found Kealia, Inc., with a Stanford professor named David Cheriton. Kealia was later acquired by Sun Microsystems, where Andy returned to the role of Senior Vice President and Chief Architect. In 2005, Andy co-founded Arastra, which later changed its name to Arista Networks.
Andy has an M.S. in Computer Engineering from Carnegie Mellon University, and a Ph.D. from Stanford University.
Andy Bechtolsheim is a multibillionaire Silicon Valley visionary. He has either designed or had a hand in the creation of some of the most significant computing and networking devices of the past 30 years. Andy and David Cheriton were the two initial investors in Google. Each of their $100,000 investments are now worth, well, let’s just say they made their money back and then some.
David Cheriton is a Stanford University computer science professor who has an amazing knack for spotting and investing in successful startups. David co-founded Granite Systems with Andy Bechtolsheim, and the two have started other successful companies including the aforementioned Kealia. David served as a technical advisor for Cisco for seven years, and was the Chief Architect for the ASICs used in the Catalyst 4000s and 4500s. He has also served as a technical advisor for companies such as Sun, VMware, and Google. David is one of the original founders of Arastra, later renamed Arista Networks. He is now the Chief Scientist for Arista.
David has multiple inventions and patents to his name, has a Ph.D. in Computer Science from the University of Waterloo, and has been at Stanford since 1981.
Given the track record of Andy and David, and the fact that these two men funded the new company without any other investors, it would seem that Arista is destined for greatness, but the story doesn’t stop there.
Ken Duda is a founder, Chief Technology Officer, and Senior Vice President of Software Engineering at Arista. Prior to founding Arastra (now Arista), Ken was CTO of There.com, where he designed a real-time 3-D distributed system that scaled to thousands of simultaneous users. I have no idea what that means, but it sure sounds cool.
Ken was the first employee of Granite Systems, and while working at Cisco, led the development of the Catalyst 4000 product line.
Ken has three simultaneous engineering degrees from MIT, and a Ph.D. in Computer Science from Stanford University.
Much of what you will read in this book about EOS is a result of Ken Duda’s vision. I met Ken while visiting Arista (along with many of the other people mentioned in this chapter), and within minutes, I realized that he was living the dream. Well, to be fair, maybe it was my dream, but what I saw was a seriously smart guy, who knew the right way to do it, and who had the freedom to do just that. I may be a hack writer now, but I went to school for programming (COBOL on punch cards, thank you very much), and loved being a programmer (we weren’t called developers back then). I gave up programming because I got tired of having to fix other people’s crappy code. I wanted to write amazing new systems, but companies weren’t looking for that—they wanted grunts to fix their crappy code.
Ken not only gets to write the kind of code he likes, but he gets to design an entire networking equipment operating system from the ground up. When I was there, I drilled him with questions. Wouldn’t that delay delivery? Wouldn’t investors complain? Didn’t you ever get rushed into finishing something early to be first to market? As he answered my questions, it all started to become clear to me. There were no crazy investors demanding artificial deadlines. These guys had decided to do it the right way, and not to deviate from that course. I also realized that everyone at Arista felt the same way. It was my meeting with Ken Duda that started the idea in my mind to write this book. Someone had to tell the world that companies like this could thrive, because in my almost 30 years in this industry, I can tell you that Arista is the first company I’ve seen that does it the right way.
The three founders certainly set the direction for Arista as a whole, but Jayshree keeps the place running. Jayshree Ullal is the President and CEO of Arista Networks. She was Senior Vice President at Cisco, where she was responsible for Data Center Switching and Services, including the Cisco Nexus 7000, the Catalyst 4500, and the Catalyst 6500 product lines. She was responsible for $10 billion in revenue, and reported directly to John Chambers, CEO of Cisco.
Jayshree has a B.S. in electrical engineering from San Francisco State University, and an M.S. in engineering management from Santa Clara University.
Jayshree was named one of the “50 Most Powerful People” in 2005 by Network World Magazine, and one of the “Top Ten Executives” at VMWorld in 2011. She has garnered many awards, including one of the 20 “Women to Watch in 2001” by Newsweek magazine.
I can hear you now saying, “blah blah blah, I could read this on Wikipedia.” But consider this: Arista is a company peopled by mad scientists who just happen to work in legitimate jobs doing good work. Jayshree keeps them all in line, and keeps the business not only humming, but also prospering. Having managed teams and departments of both developers and engineers, I know what a challenge it can be. She makes it look easy.
All of these people are powerful forces in the networking and IT worlds, and all of them manage to make time to meet with prospective customers and even speak during classes held onsite at Arista. I’ve been in both situations, and have seen this for myself.
I’m a successful, self-employed consultant who moonlights as a writer for no other reason than I like to write. I haven’t wanted to work for anyone but myself for years, maybe even decades; I’ve been to Arista’s headquarters in California multiple times, and each time I left, I felt like I should have gone back and begged for a job. There’s something special happening there, and these people are all at the heart of it.
You can read more about Arista and the management team at Arista’s website.
So what’s the big deal about data centers? Why do they need special switches anyway? Can’t we just use the same switches we use in the office? Hell, can’t we just go to Staples and buy some Linksys or Netgears, or D-Links or something?
Believe it or not, I’ve had this very conversation on more than one occasion with executives looking to save some money on their data center builds. While it may be obvious to me, I quickly learned that it’s not apparent to everyone why data centers are unique.
Data centers are usually designed for critical systems that require high availability. That means redundant power, efficient cooling, secure access, and a pile of other things, but most of all, it means no single points of failure.
Every device in a data center should have dual power supplies, and each one of those power supplies should be fed from discrete power feeds. All devices in a data center should have front-to-back airflow, or ideally, airflow that can be configured front to back or back to front. All devices in a data center should support the means to upgrade, replace, or shut down any single chassis at any time without interruption to the often-extreme Service Level Agreements (SLAs). In-Service Software Upgrades (ISSU) should also be available, but this can be circumvented by properly distributing load to allow meeting the prior requirement. Data center devices should offer robust hardware, even NEBS compliance where required, and robust software to match.
While data center switches should be able to deliver all of those features, they should also not be loaded down with features that are not desired in the data center. Examples of superfluous features might include Power Over Ethernet, backplane stacking, VoIP Gateway features, Wireless LAN Controller functions, and other generally office-specific features.
Note
Note that this last paragraph greatly depends on what’s being housed in the data center. If the data center is designed to house all the IT equipment for a large office, then PoE and WAN Controllers might be desirable. Really though, in a proper data center, those functions should be housed in proper dual power supply devices dedicated to the desired tasks.
While stacked switches seem like a great way to lower management points and increase port density, you may find that switches that support such features often don’t have the fabric speed or feature set to adequately support a data center environment. I’ve made a lot of money swapping out closet switches for Cisco Nexus and Arista 7000 switches in data centers. Data centers are always more resilient when using real data center equipment. If you don’t pay to put them in from the start, you’ll pay even more to swap them in later.
VMware really shook up the data center world with the introduction of Vmotion. With Vmotion, virtual machines can be migrated from one physical box to another, without changing IP addresses and without bringing the server offline. I have to admit, that’s pretty cool.
The problem is that in order to accomplish this, the source and destination servers must reside in the same VLANs. That usually means having VLANs spanning across physical locations, which is just about the polar opposite of what we’ve spent the last 20 years trying to move away from!
In the past few years, a pile of technologies have surfaced to try to address this issue, from the open standard TRILL, to 802.1aq (Shortest Path Bridging), to Cisco’s OTV, and even VXLAN. They all have their benefits, and they all have their (often severe) drawbacks. During that time, some standards have developed around something called Data Center Bridging, which aims to (among other things) make the Vmotion issue a little bit easier to cope with. Features such as priority-based flow control, Fiber Channel over Ethernet (FCoE), and others are also a consideration with data center bridging. Though there is no widely accepted standard as of mid-2012, data center switches should support, or have the ability to support, at least a subset of these technologies. If your executive comes in and says that you need to support some new whizbang data center technology because he read about it in CIO magazine on the john that morning, having a data center full of closet switches will mean a rough conversation about how he bought the wrong gear.
Low latency may seem like a solution in need of a problem if you’re used to dealing with email and web servers, but in some fields, microseconds mean millions: millions of dollars, that is.
I talk about trading floors later on in this book, and some of Arista’s biggest customers use Arista switches in order to execute trades faster than their competitors. But think about other environments where microseconds translate into tangible benefits. Environments such as computer animation studios that may spend 80 to 90 hours rendering a single frame for a blockbuster movie, or scientific compute farms that might involve tens of thousands of compute cores. If the network is the bottleneck within those massive computer arrays, the overall performance is affected. And imagine the impact that an oversubscribed network might have on such farms. I’ve never had the pleasure of working in such environments, but I can imagine that dropping packets would be frowned upon.
Sure, those systems require some serious networking, but you might be surprised how much latency can affect more common applications. iSCSI doesn’t tolerate dropped packets well, nor does it tolerate a lot of buffering. Heck, even NAS, which can tolerate dropped packets, is often used for systems and applications that do not tolerate latency well. Couple that with the way that most NAS are designed (many hosts to one filer), and things like buffering become a huge issue. Not only have I seen closet switches fail miserably in such environments, I’ve seen many data center class switches fail too.
The NAS protocol was developed in the early 1980s as a means for university students to share porn between systems. OK, I totally made that up, but I’d be willing to bet that it was one of the first widespread uses of the technology. NAS really was developed in the early 1980s though, and although it’s come a long way, it was not designed to be a solution for low-latency, high-throughput storage. NAS was designed to be used over IP, and often uses TCP for reliability. Compared with more low-level solutions such as FibreChannel, NAS is slow and inefficient.
Still, NAS is comparatively inexpensive, doesn’t require special hardware on the server side, and many vendors offer specialized NAS solutions aimed at centralizing storage needs for scores, if not hundreds of servers. NAS is a reality in the modern data center, and the networks that NAS rides on must be robust, offer low latency, and whenever possible, not drop packets. Even with non-blocking 10 Gb architectures, it can be easy to oversubscribe the 10 Gbps links to the NAS devices if many servers make simultaneous 10 Gbps reads or writes.
So how does Arista deal with the requirements outlined in this chapter? Here’s a short list to whet your appetite. Each one of these topics is covered in detail within this book, so here I’ll just supply a list with a brief explanation of each feature and a reference to the chapter in which the topic is covered in more detail.
Arista switches all have dual power supplies, hot swappable and reversible airflow fans, completely non-blocking fabrics (even the eight-slot chassis switches!), and merchant silicon. In almost every case, they are physically smaller, weigh less, consume less power, and often cost less than comparable switches from other manufacturers; although as you’ll come to learn, there really are no other switches that compare. See Chapter 5 for details on the Arista product offerings. Sure they may make great hardware, but the real difference is in the operating systems.
The Extensible Operating System (EOS) offers an industry standard CLI while offering the power, flexibility, and expandability of Linux. Man, what a mouthful of marketing buzzwords that is. Let’s cut the BS and tell it like it is: EOS is Linux, with a Cisco-like CLI. Actually, even that barely tells the whole story. Arista switches run Linux. They don’t run some stripped down version of Linux that’s been altered beyond recognition—they run Linux. Some other vendors say that their OS is based on Linux, and I guess it is, but on an Arista switch, you can drop down into the bash shell and kill processes if you’re so inclined. Hell, you can even spawn another CLI session from bash, write scripts that contain CLI commands, send email from CLI, pipe bash commands through CLI, and a host of other exciting things, all because the switch runs Linux and because the programmers care about one thing above all else: doing things the right way.
Arista hardware is amazing, but EOS makes these devices profoundly different than any other vendor’s offerings.
OK, so I blew the surprise with my EOS fan-boy ravings, but
yes, you can issue the bash command from CLI and enter the world of
Linux. It’s not a Linux simulator either – it’s bash, in Linux. You can
even execute the sudo shutdown –r now
command if you want, and you know you want to. All your other favorite
Linux commands are there too: ps,
top, grep, more,
less, vi, cat,
tar, gunzip, and python just to name a few. But not perl. Unless you want to add it, in which case
you can, because it’s Linux.
The fact that these switches run Linux is such a big deal that I recommend learning Linux to my clients when they’re considering Arista switches. Of course the beauty of EOS is that you don’t have to know Linux thanks to the CLI, but trust me when I say you’ll be able to get much more out of your Arista switches with some good Linux experience.
SysDB is one of the main features that makes EOS and Arista switches great. Simply put, SysDB is a database on the switch that holds all of the critical counters, status, and state information necessary for processes to run. The processes read and write this information to and/or from SysDB instead of storing it locally. If another process needs the information, it gets it from SysDB. Thus, processes never need to talk to each other; they communicate through SysDB. This dramatically lowers the possibility of one process negatively affecting another. Additionally, if a process dies, it can restart quickly without having to reinitialize all values, since it can read them all from SysDB. See Chapter 10 for more information on SysDB.
Multichassis Link Aggregation (MLAG) allows port-channels to exist to multiple switches at the same time. Similar to Cisco’s VPC, Arista’s MLAG is easier to configure and, in my experience, less likely to induce colorful profanity from me during use. Of course your mileage may vary. See Chapter 12 for more detail about MLAG.
Virtual ARP (VARP) is an amazingly simple idea that allows multiple switches to respond to ARP requests for the same IP. That might sounds like a bad idea, but delve into Chapter 14 to see why it’s a pretty cool feature.
Data center switches sometimes suffer from a problem known as microbursting, wherein the buffers become overrun and drop packets. The problem is that these microbursts happen often at microsecond intervals, so the switches never report them. These problems can be horrific to diagnose, and even worse to try and explain to executives. That is, unless you have an Arista switch with latency analyzer (LANZ). Check out Chapter 20 to see LANZ in action.
VM Tracer allows an Arista switch to have visibility into the VMware virtual machines connected to it. It also allows the switch to dynamically create and delete VLANs when they are created on the ESX host, thus rendering you, the network admin, completely obsolete. Well, not really obsolete; I mean, someone has to configure VM Tracer, right? To see the truth about the feature that you may never tell the server guys about, check out Chapter 22.
Zero Touch Provisioning (ZTP) allows your Arista switch to not only load its configuration from the network, but also from its operating system. What’s more, it can download scripts that tell it to do both of those things and more, all without human interaction. To see it in action, take a look at Chapter 25.
Did you know that Arista switches could be configured to send emails? Not only can they send emails, but they can do it from bash, from EOS, and from within scripts. Any command can be piped directly to your inbox on a properly configured Arista switch. Check out Chapter 19 to see how.
Yeah, email is cool, but with an Arista switch, you can schedule a job that will email the status of an interface to you every five minutes. Hell, you could configure your Arista switch to email a message with the subject of “I love Arista switches!” to John Chambers every hour if you’d like, but I don’t recommend it. Seriously, don’t do that. But check out Chapter 23 to see how; you know, for research.
You can run tcpdump from bash or EOS, and it captures every packet on an interface that is destined for, or sourced, from the CPU of the switch. You could probably pipe the output to email, but I wouldn’t recommend that either. See Chapter 24 for details on how to use tcpdump.
Event handler lets you configure triggers on your switch that will
execute a command when activated. You could trigger an email to your
phone every time the switch boots, or you
could configure the switch to send you the output of show log last 2 minutes to your email
when a specified interface goes up or down. Take a look at Chapter 26 for details.
Event Monitor records every add, change, and/or deletion of ARP, MAC, and route entries on your switch to a database. You can access the database to produce reports, which can come in very handy when you need to find out what happened, say, yesterday at 6 p.m. when some server you don’t care about stopped working. Imagine having a view into what happened on the switch in the past. Now you don’t have to imagine! Go read Chapter 27 to see how to make the most of this unique feature.
Did I mention that Arista switches run Linux? Just like a Linux machine, you can add additional packages that have been written for EOS. These extensions are easy to install, manage, and remove, and in Chapter 28, I’ll show you how to do just that.
CloudVision is a feature set that allows for centralized monitoring and management of your network. Currently, this lets you configure your switch (or group of switches) from an instant messenger (IM) client running XMPP, but there will be features available in future releases as well. Configure 100 switches with an IM? Yeah, I’ll show you how to do that in Chapter 29.
As you can see, Arista switches can do some pretty interesting things that aren’t available on any other switches. Features aside, the OS is written so well and with such attention to detail that even without all the cool features, I think you’ll find Arista switches to be a cut above the other vendors’ offerings. But enough hype, let’s dig in and learn the inner workings of Arista’s switches.
Get Arista Warrior now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
