DevOps Tools for Java Developers

Chapter 1. DevOps for (or Possibly Against) Developers

Baruch Sadogursky

While you here do snoring lie,
Open-eyed conspiracy His time doth take.
If of life you keep a care,
Shake off slumber, and beware:
Awake, awake!

William Shakespeare, The Tempest

Some might ask if the DevOps movement is simply an ops-inspired plot against developers. Most (if not all) who would do so wouldn’t expect a serious response, not least because they intend the question as tongue-in-cheek teasing. It’s also because—and regardless of whether your origins are on the development or the operations side of the equation—when anyone strikes up a conversation about DevOps, it will take approximately 60 seconds before someone inquires, “But what is DevOps really?”

And you’d think, 11 years after the coining of the term (a decade within which industry professionals have spoken, debated, and shouted about it), that we’d all have arrived at a standard, no-nonsense, commonly understood definition. But this simply isn’t the case. In fact, despite an exponentially increasing corporate demand for DevOps personnel, it’s highly doubtful that any five DevOps-titled employees, chosen at random, could tell you precisely what DevOps is.

So, don’t be embarrassed if you still find yourself scratching your head when the subject comes up. Conceptually, DevOps may not be easy to grok, but it’s not impossible either.

But regardless of how we discuss the term or what definition(s) we might agree upon, there’s one thing, above all else, that’s critical to bear in mind: DevOps is an entirely invented concept, and the inventors came from the ops side of the equation.

DevOps Is a Concept Invented by the Ops Side

My premise about DevOps may be provocative, but it’s provable too. Let’s start with the facts.

Exhibit 1: The Phoenix Project

The Phoenix Project by Gene Kim et al. (IT Revolution) became a classic since published almost a decade ago. It’s not a how-to manual (not in the traditional sense, anyway). It’s a novel that tells the story of a highly problematic company and its IT manager who is suddenly assigned the task of implementing a make-or-break corporate initiative that’s already way over budget and months behind schedule.

If you dwell in the realms of software, the rest of the book’s central characters will be familiar to you. For the moment, though, let’s have a look at their professional titles:

• Director, IT service support

• Director, distributed technology

• Manager, retail sales

• Lead systems administrator

• Chief information security officer

• Chief financial officer

• Chief executive officer

Notice the connective tissue between them? They’re the protagonists of one of the most important books about DevOps ever written and not one of them is a developer. Even when developers do figure into the plotline, well…let’s just say they’re not spoken of in particularly glowing terms.

When victory comes, it’s the hero of the story (together with a supportive board member) who invents DevOps, pulls the project’s fat out of the fire, turns his company’s fortunes around, and gets rewarded with a promotion to chief information officer (CIO) of the enterprise. And everyone lives happily—if not ever after, then for at least the two or three years such successes tend to buy you in this business before it’s time to prove your worth all over again.

Exhibit 2: The DevOps Handbook

It’s better to read The Phoenix Project before The DevOps Handbook by Gene Kim et al. (IT Revolution) because the former places you within a highly believable, human scenario. It’s not difficult to immerse yourself in the personality types, the professional predicaments, and the interpersonal relationships of the characters. The hows and whys of DevOps unfold as inevitable and rational responses to a set of circumstances, which could have just as easily led to business collapse. The stakes, the characters, and the choices they make all seem quite plausible. Parallels to your own experience may not be too hard to draw.

The DevOps Handbook allows you to explore the component conceptual parts of DevOps principles and practices in greater depth. As its subtitle suggests, the book goes a long way toward explaining How to Create World-Class Agility, Reliability, and Security in Technology Organizations. But shouldn’t that be about development? Whether it should or shouldn’t may be open to debate. What’s incontrovertible is that the book’s authors are bright, super-talented professionals who are, arguably, the fathers of DevOps. However, Exhibit 2 isn’t included here to praise them so much as to take a close look at their backgrounds.

Let’s start with Gene Kim. He founded the software security and data integrity firm Tripwire, serving as its chief technology officer (CTO) for over a decade. As a researcher, he’s devoted his professional attention to examining and understanding the technological transformations that have and are occurring within large, complex businesses and institutions. In addition to coauthoring The Phoenix Project, in 2019 he coauthored The Unicorn Project (which I’ll have more to say about later). Everything about his career is steeped in ops. Even when Unicorn says it’s “about Developers,” it’s still developers as seen through the eyes of an ops guy!

As for the other three authors of the Handbook:

• Jez Humble has held positions including site reliability engineer (SRE), CTO, deputy director of Delivery Architecture and Infrastructure Services, and Developer Relations. An ops guy! Even though the last of his titles references development, the job isn’t about that. It’s about relations with developers. It’s about narrowing the divide between dev and ops, about which he has written, taught, and lectured extensively.

• Patrick Debois has served as a CTO, director of Market Strategy, and director of Dev♥Ops Relations (the heart is his addition). He describes himself as a professional who is “bridging the gap between projects and operations by using Agile techniques in development, project management, and system administration.” That sure sounds like an ops guy.

• John Willis, as of this writing, holds the title of VP of DevOps and Digital Practices. Previously, he’s been a director of Ecosystem Development, a VP of Solutions, and notably a VP of Training and Services at Opscode (now known as Progress Chef). And while John’s career has been a bit more deeply involved with development, most of his work has been about ops, particularly to the degree that he has focused his attention on tearing down the walls that had once kept developers and operations personnel in distinct and separate camps.

As you can see, all the authors have an ops background. Coincidence? I think not.

Still not convinced that DevOps is ops driven? How about we have a look at the leaders trying to sell us on DevOps today.

Google It

As of this writing, if you typed “What is DevOps?” in a Google search just to see what would come up, your first page of results would likely include the following:

• Agile Admin, a system administration company

• Atlassian, whose products include project and issue tracking, list making, and team collaboration platforms

• Amazon Web Services (AWS), Microsoft Azure, and Rackspace Technology, all of which sell cloud ops infrastructure

• Logz.io, which sells log management and analysis services

• New Relic, whose specialty is application monitoring

All of these are very ops-focused. Yes, that first page contained one firm that was a bit more on the development side and one other that really wasn’t directly related to the search. The point is that when you try to look for DevOps, most of what you’ll find tends to skew toward ops.

What Does It Do?

DevOps is a thing! It’s in big demand. And with this, many will want to know, concretely, what DevOps does, what it substantively produces. Rather than go down that route, let’s look at it structurally, conceptualizing it as you would the sideways, figure-eight-shaped infinity symbol. In this light, we see a loop of processes that goes from coding to building to testing to release to deployment to operations to monitoring, and then back again to begin planning for new features, as shown in Figure 1-1.

Figure 1-1. DevOps infinity loop

And if this looks familiar to some readers, it should because it bears a conceptual similarity to the Agile development cycle (Figure 1-2).

Figure 1-2. Agile development cycle

There’s no profound difference between these two never-ending stories other than the fact that ops folks grafted themselves onto the old world of the Agile circle, essentially stretching it into two circles and shoehorning their concerns and pains into the domain that was once considered solely the province of developers.

State of the Industry

Since 2014, further proof that DevOps is an ops-driven phenomenon has come packaged in the form of an easy-to-read annual digest of data collected, analyzed, and summarized from tens of thousands of industry professionals and organizations worldwide. The “Accelerate: State of DevOps” report was primarily the work of DevOps Research and Assessment (DORA) and is the most important document in the software industry to gauge where DevOps is and where it’s likely going. In the 2018 edition, for example, we can see a serious focus on questions such as these:

• How often do organizations deploy code?

• How long does it typically take to go from code commits to successfully running in production?

• When impairments or outages occur, how long does it generally take to restore service?

• What percentage of deployed changes result in degraded service or require remediation?

Notice that all of those are very ops-centered concerns.

What Constitutes Work?

Now let’s examine how work is defined by the “Accelerate: State of DevOps” report and The Phoenix Project. Well, first, planned work focuses on business projects and new features, which span both ops and dev. Internal projects, including server migrations, software updates, and changes driven by feedback on deployed projects can be broad-based and may or may not weight more to one side of the DevOps equation than the other.

But what about unplanned activities, such as support escalations and emergency outages? Those are ops heavy, as is the coding of new features, bug fixes, and refactoring—which are all about how the life of ops can be made easier by including developers in the DevOps story.

If We’re Not About Deployment and Operations, Then Just What Is Our Job?

Clearly, DevOps isn’t anything developers were (or are) demanding. It’s an ops invention to make everyone else work harder. And assuming this truth, let’s ponder what would happen if developers were to rise up as one and say, “Your ops concerns are yours, not ours.” Fine. But in that case, it would be only right and proper to ask the rebelling developers for their definition of “done.” What standards do they believe they need to achieve to say, “We did our job well and our part is now complete”?

This isn’t a flippant question, and there are sources we can look to for answers. One, although imperfect and not infrequently criticized, is the Manifesto for Software Craftsmanship, which puts forth four fundamental values that should motivate developers. Let’s consider them:

Well-crafted software

Yes, indeed, quality is important.

Steadily adding value

No argument there. Of course we want to provide services and features that people need, want, or would like.

Community of professionals

In broad strokes, who could oppose this? Cordiality among industry peers is just being professionally neighborly.

Productive partnerships

Collaboration is certainly the name of the game. Developers aren’t against quality assurance (QA), ops, or the products themselves. So, in context, this is just about being friendly with everybody (so long as other teams don’t start dictating what their jobs are supposed to be).

Just What Constitutes “Done”?

With all that we’ve established so far, we can safely say that we need to produce code that’s simple, readable, understandable, and easy to deploy. We must be certain that nonfunctional requirements (e.g., performance, throughput, memory footprint, security, privacy, etc.) are satisfied. We should work diligently to avoid incurring any technical baggage and, if we’re lucky, shed some along the way. We have to be sure that all tests pass. And we’re obligated to maintain fruitful relations with QA teams (when they’re happy, we’re happy).

With good-quality code, plus positive team leader and peer reviews, everything should be fine out of the gate. With a product team that defines standards for value and added value, benchmarks can be firmly established. Through their feedback, product owners help to determine whether those benchmarks are (or aren’t) being met and by how much. That’s a pretty good thumbnail definition of a good software developer having “done” what they needed to do. It also demonstrates that “well done” can never adequately be measured (or even known) without the involvement of and clear communications with ops personnel.

Rivalry?

So yes, although it can be proven that DevOps really wasn’t anything for which developers were clamoring, it can equally be shown that its infinite practice benefits everyone. And still there are holdouts; those who imagine a rivalry, even an antagonism, between developers and, for example, QA testers. Developers work hard on their creations and then feel like the QA teams are almost like hackers, out to prove something, just digging and digging until they find a problem.

This is where DevOps counseling comes in. Every conscientious developer wants to be proud of what they’re producing. Finding flaws can seem like criticism, when it’s really just conscientious work coming from another direction. Good, clear, open, and continuous communication between developers and QA personnel helps reinforce the benefits of DevOps, but it also makes plain that everyone is ultimately working toward the same goal. When QA folks identify bugs, all they’re doing is helping their developer colleagues to write better code, to be better developers. And this example of interplay between those on the ops side and others on the dev side demonstrates the useful blurring of the distinctions and separations between the two worlds. Their relationship is necessarily symbiotic and, once again, works along an endless continuum of activities, with one informing the other for the mutual benefit of all.

More Than Ever Before

The increasing demand for DevOps is coming as much from external forces as it is from within software companies themselves. And this is because our expectations, all of our expectations, as people living in a 21st century world, continue to change rapidly. The more reliant we become on ever-improving software solutions, the less time we have to waste on information and communication gaps, and delays between developers and operations personnel.

Take, for example, banking. A decade ago, most major banks had reasonably adequate websites. You could log in to have a look at your accounts, your statements, and recent transactions. Maybe you were even beginning to make payments electronically through the e-services your bank was offering. And while those services were nice and offered a certain amount of convenience, you likely still needed to go (or, at least, felt more comfortable going) to your local branch to handle your banking affairs.

What didn’t exist is today’s fully digital experience—complete with mobile apps, automated account monitoring and alerts, and enough services that make it more and more common for average account holders to do everything online. You may even be among those who not only don’t care if you ever see the inside of your bricks-and-mortar branch ever again, but also don’t even know where that branch is! What’s more, banks are reacting to these rapidly changing banking habits by consolidating and closing physical branches, and offering incentives for their customers to shift their banking to the online sphere. This accelerated even more during the COVID-19 crisis, when access to branches was restricted to appointment-only services, limited walk-in access, and shorter hours.

So, 10 years ago, if your bank’s website went down for 12 hours of maintenance while the bank was deploying a better, more secure site, you’d probably have taken that in stride. What’s a dozen hours if it’s going to result in higher-quality services? You didn’t need 24/7, online banking—and, besides, the local branch was there for you. Today, that’s simply not the case. Half a day’s worth of downtime is unacceptable. In essence, you expect your bank to always be open and available. This is because your (and the world’s) definition of quality has changed. And that change requires DevOps more than ever before.

Volume and Velocity

Another pressure impelling the growth of DevOps is the amount of data that’s being stored and handled. And that’s only logical. If more and more of our daily lives are reliant on software, a tremendous rise in the amount of data it generates will obviously occur. In 2020, the entire global datasphere amounted to nearly 10 zettabytes. A decade prior, it was 0.5 zettabytes. By 2025, it’s reasonably estimated that it will balloon exponentially to over 50 zettabytes!

This isn’t only about behemoths like Google, Netflix, Facebook, Microsoft, Amazon, Twitter, and others getting bigger and better, and therefore needing to process larger amounts of data. This projection affirms that more and more companies will be graduating into the world of big data. With that comes the demands of vastly increased data loads, as well as the shift away from traditional staging-server environments, which offered exact replicas of given production environments. And this shift is predicated on the fact that maintaining such pair-to-pair schemes is no longer feasible in terms of size or speed.

Happy were the days of yore when everything could be tested before going into production, but this isn’t possible anymore. Things are and will increasingly be released into production about which software firms don’t have 100% confidence. Should this cause us to panic? No. The necessity to release fast and remain competitive should inspire innovation and creativity in the ways required to best execute controlled rollovers, test procedures, and more in-production testing—what’s now referred to as progressive delivery. This comes along with feature flags and observability tools, such as distributed tracing.

Some equate progressive delivery with the blast radius of an explosive device. The idea is that when we deploy to an in-production environment, explosions should be expected. Therefore, to optimize such rollouts, the best we can hope for is to minimize casualties, to keep the size of the blast radius as small as is possible. This is consistently accomplished through improvements in the quality of servers, services, and products. And if we agree that quality is a concern of developers and its achievement is part of a developer’s definition of “done,” then it means there can be no pause or disconnect between that dev moment of done and the next, ops moment of production. Because no sooner does this happen than we’re looping back into development, as bugs are fixed, services restored due to outages, and so on.

Done and Done

Maybe it’s becoming clear that the expectations and demands that were and continue to be generated from the ops environment, of necessity, drove the push to DevOps. And as a result, the increased expectations and demands on developers aren’t coming from some festering hatred that ops folks have for their developer colleagues, nor is it part of a plot to deprive them of sleep. Rather, all of this, all of what DevOps represents, is a realpolitik business response to our changing world and the changes they’ve forced on the software industry across the board.

The fact is that everyone has new responsibilities, some of which require professionals (certainly many departments) to be at the ready to respond whenever duty calls because ours is a nonstop world. Here’s another way of putting this: our old definitions of “done” are done!

Our new definition is site reliability engineering (SRE). This Google-coined term forever weds dev to ops by bridging any lingering perceived gaps between the two. And while SRE areas of focus may be taken up by personnel on either or both sides of the DevOps equation, these days companies often have dedicated SRE teams that are specifically responsible for examining issues related to performance, efficiency, emergency responsiveness, monitoring, capacity planning, and more. SRE professionals think like software engineers in devising strategies and solutions for system administration issues. They’re the folks who are increasingly making automated deployments work.

When SRE staff are happy, it means builds are becoming ever more reliable, repeatable, and fast, particularly because the landscape is one of scalable, backward- and forward-compatible code operating in stateless environments that are relying on an exploding universe of servers and emitting event streams to allow for real-time observability and alerts when something goes wrong. When new builds occur, they need to launch rapidly (with the full expectation that some will die equally quickly). Services need to return to full functionality as rapidly as possible. When features don’t work, we must have an immediate ability to turn them off programmatically through an API. When new software is released and users update their clients, but then encounter bugs, we must have the ability to execute swift and seamless rollbacks. Old clients and old servers need to be able to communicate with new clients.

And while SRE is assessing and monitoring these activities and laying out strategic responses, the work in all of these areas is completely that of developers. Therefore, while Dev staff are doing, SRE is today’s definition of done.

Float Like a Butterfly…

In addition to all of the considerations already mentioned, a fundamental characteristic must define code in our modern DevOps (and related SRE) era: lean. And by this we’re talking about saving money. “But what,” you may ask, “does code have to do with saving money?”

Well, one illustration might be cloud providers who charge companies for a plethora of discrete services. Some of these costs are directly affected by the code being output by those corporate cloud service subscribers. Cost reductions can therefore come from the creation and use of innovative developer tools, as well as writing and deploying better code.

The very nature of a global, we-never-close, software-driven society with a constant desire for newer, better software features and services means that DevOps can’t be concerned with only production and deployments. It must also be attentive to the bottom line of business itself. And although this may seem to be yet another burden thrown into the mix, think about it the next time the boss says that costs must be cut. Instead of negative, knee-jerk solutions, such as employee layoffs or reductions in salaries and benefits, needed savings may be found in positive, business-profile-enhancing moves like going serverless and moving to the cloud. Then, no one gets fired, and the coffee and donuts in the breakroom are still free!

Being lean not only saves money, but also gives companies the opportunity to improve their marketplace impact. When firms can achieve efficiencies without staff reductions, they can retain optimal levels of team strength. When teams continue to be well compensated and cared for, they’ll be more motivated to produce their best possible work. When that output is achieving success, it means customers are grinning. So long as customers continue to get well-functioning new features quickly as a result of faster deployments, well…they’ll keep coming back for more and spreading the word to others. And that’s a virtuous cycle that means money in the bank.

Integrity, Authentication, and Availability

Hand in hand, and shoulder to shoulder with any and all DevOps activities is the perpetual question of security. Of course, some will choose to address this concern by hiring a chief information security officer. And that’s great because there’ll always be a go-to person to blame when something goes wrong. A better option might be to actually analyze, within a DevOps framework, how individual employees, work teams, and companies as a whole think about security and how it can be strengthened.

We talk much more about this in Chapter 10, but for now consider that: breaches, bugs, Structured Query Language (SQL) injections, buffer overflows, and more aren’t new. What’s different is the increasing speed of their appearance, their rising quantity, and the cleverness with which malicious individuals and entities are able to act. It’s not surprising. With more and more code being released, more and more problems will follow, with each type demanding different solutions.

With faster deployments, it becomes ever more critical to be more reactive to risks and threats. The 2018 discovery of the Meltdown and Spectre security vulnerabilities made it clear that some threats are impossible to prevent. We are in a race, and the only thing to do is to deploy fixes as quickly as possible.

Fierce Urgency

It should be clear by now that DevOps is not a plot, but a response to evolutionary pressures. It’s a means to an ends that does the following:

• Delivers better quality

• Produces savings

• Deploys features faster

• Strengthens security

And it doesn’t matter who likes it or not, or who came up with the idea first, or even its original intent. What matters is covered in the next section.

The Software Industry Has Fully Embraced DevOps

Bу now, every company is a DevOps company. So, get on board…because you don’t have any other choice.

The DevOps of today, what DevOps has evolved into, is, as stated earlier, an infinity loop. It doesn’t mean that groups and departments no longer exist. It doesn’t mean that everyone is responsible for their areas of concern along with those of everyone else along this continuum.

It does mean that everyone should be working together. It does mean that software professionals within a given enterprise must be aware of and taking into reasonable consideration the work that all of their other colleagues are doing. They need to care about the issues their peers are confronting, how those matters can and do impact the work they do, the products and services their company offers, and how the totality of this affects their firm’s marketplace reputation.

This is why DevOps engineer is a term that makes no sense because it implies the existence of someone who can comprehensively and competently do (or is, at least, completely versed in) everything that occurs within the DevOps infinity loop. No such person exists. They never will. In fact, even trying to be a DevOps engineer is a mistake because it runs entirely counter to what DevOps is, which is eliminating silos where code developers are walled off from QA testers, who are walled off from release personnel, and so on.

DevOps is a coming together of efforts, interests, and feedback in a continuous effort to create, secure, deploy, and perfect code. DevOps is about collaboration. And as collaborations are organic, communicative endeavors, well…just as collaboration engineering isn’t a thing, neither is DevOps engineering (no matter what any institute or university might promise).

Making It Manifest

Knowing what DevOps is (and isn’t) only establishes a concept. The question is, how can it be sensibly and effectively implemented and sustained in software firms far and wide? Best advice? Here goes.

First, you can have DevOps enablers, DevOps evangelists, DevOps consultants and coaches (and I know how Scrum spoiled all those terms for all of us, but there aren’t any better ones). That’s OK. But DevOps is not an engineering discipline. We want site/service reliability engineers, production engineers, infrastructure engineers, QA engineers, and so on. But once a company has a DevOps engineer, the next thing it’s almost guaranteed to have is a DevOps department, which will just be another silo that’s likely to be nothing more than an existing department that’s been rebranded, so it looks like the firm is on the DevOps bandwagon.

A DevOps office isn’t a sign of progress. Rather, it’s simply back to the future. Then, the next thing that will be needed is a way to foster collaborations between Dev and DevOps, which will necessitate the coining of yet another term. How about DevDevOps?

Second, DevOps is about nuances and small things. Like cultures (especially corporate cultures), it’s about attitudes and relationships. You may not be able to clearly define those cultures, but they exist all the same. DevOps is also not about code, engineering practices, or technical prowess. No tool you can buy, no step-by-step manual, and no home edition board game can help you create DevOps in your organization.

It’s about behaviors that are encouraged and nurtured within companies. And much of this is simply about how rank-and-file staff are treated, the structure of the firm, and the titles people hold. It’s about how often people have an opportunity to get together (especially in nonmeeting settings), where they sit and eat, talk shop and nonshop, tell jokes, etc. It’s in these spaces, not within data centers, where cultures form, grow, and change.

Finally, companies should actively seek and invest in T-shaped people (Ж-shaped is even better, as my Russian-speaking readers might suggest). As opposed to I-shaped individuals (who are absolute experts in one area) or generalists (who know a good bit about a lot, but have no mastery of any particular discipline), a T-shaped person has world-class expertise in at least one thing. This is the long vertical line of the “T” that firmly roots their depth of knowledge and experience. The “T” is crossed above by a breadth of accumulated capabilities, know-how, and wisdom in other arenas.

The total package is someone who demonstrates a clear and keen propensity to adapt to circumstances, learn new skills, and meet the challenges of today. In fact, this is a nearly perfect definition of an ideal DevOps staffer. T-shaped personnel allow businesses to work effectively on prioritized workloads, instead of only what companies think their in-house capacities can bear. T-folks can see and are intrigued by the big picture. This makes them terrific collaborators, which, as a result, leads to the construction of empowered teams.

We All Got the Message

The good news is that a decade after ops invented DevOps, they completely understand that it’s not only about them. It’s about everybody. We can see the change with our own eyes. For example, the 2019 “Accelerate: State of DevOps” report got more developers to participate in the study than ops or SRE personnel! To find more profound evidence that things have changed, we return full circle to Gene Kim. Also in 2019, the man who helped novelize the ops end of the equation with The Phoenix Project released The Unicorn Project (IT Revolution). If the earlier book gave short shrift to developers, here our hero is Maxine, her company’s lead developer (and ultimate savior).

DevOps began with ops, no question about it. But the motivation wasn’t the subjugation of developers, nor the supremacy of operations professionals. It was and remains predicated on everyone seeing everyone else, appreciating their value and contributions to an enterprise—not simply out of respect or courtesy, but as personal self-interest and business survival, competitiveness, and growth.

And if you’re scared that DevOps is going to overwhelm you in a sea of ops concepts, it’s actually most likely to be the other way around. Just look at SRE’s definition by Google (the company that invented the discipline):

SRE is what you get when you treat operations as if it’s a software problem.

So, the ops folks want to be developers now? Welcome. Software problems belong to all software professionals all the time. We’re in the problem-solving business—which means that everyone is a little bit of an SRE, a little bit of a developer, a little bit into operations…because it’s all the same thing. They’re all just intertwined facets that allow us to devise solutions for the software of today, as well as the individual and societal problems of tomorrow.