Grand principles that generate no action are mere vapor. Conversely, specific practices in the absence of guiding principles are often inappropriately used.
The board game Othello has the slogan, “A minute to learn, a lifetime to master.” This applies really well to a team that’s learning Scrum. The basic practices and mechanics of Scrum are straightforward, and not difficult to adopt. But understanding how Scrum’s values can make those practices and mechanics result in better software can be more challenging.
There are three main roles on a Scrum project: Product Owner, Scrum Master, and team member. (We capitalize “Product Owner” and “Scrum Master” when talking about the Scrum roles.)
The software is built using timeboxed iterations called sprints. At the start of each sprint, the team does sprint planning to determine which features from the backlog they will build. This is called the sprint backlog, and the team works throughout the sprint to build all of the features in it.
Every day, the team holds a short face-to-face meeting called the Daily Scrum to update each other on the progress they’ve made, and to discuss the roadblocks ahead. Each person answers three questions: What have I done since the last Daily Scrum? What will I do until the next Daily Scrum? What roadblocks are in my way?
One person, the Scrum Master, keeps the project rolling by working with the team to get past roadblocks that they’ve identified and asked for help with. At the end of the sprint, working software is demonstrated to the product owner and stakeholders in the sprint review, and the team holds a retrospective to figure out lessons they’ve learned, so they can improve the way they run their sprints and build software in the future.
But for a Scrum team to become effective, they need to do more than just follow the basic Scrum pattern. Effective Scrum teams are self-organizing, as Ken Schwaber explains in Agile Project Management with Scrum (note the words that we emphasized in italics):
For Scrum to work, the team has to deeply and viscerally understand collective commitment and self-organization. Scrum’s theory, practices, and rules are easy to grasp intellectually. But until a group of individuals has made a collective commitment to deliver something tangible in a fixed amount of time, those individuals probably don’t get Scrum. When the team members stop acting as many and adopt and commit to a common purpose, the team becomes capable of self-organization and can quickly cut through complexity and produce actionable plans.
The goal of this chapter is to help you “get Scrum” by building on the ideas from Chapters 2 and 3 to teach you the practices and patterns of Scrum. We’ll use those practices to show you the ideas behind the principles of collective commitment and self-organization.
In Agile Project Management with Scrum, Ken Schwaber lays out the rules of Scrum that describe the basic pattern for a Scrum project. In fact, you can download the definitive rules of Scrum from www.scrum.org as a free PDF—it’s an ebook called The Scrum Guide, by Ken Schwaber and Jeff Sutherland, the people who created Scrum and have helped spread it to the rest of the industry. These rules should be pretty familiar to you by now, because many of them were already described in Chapters 2 and 3. A typical Scrum project follows these rules:
Each sprint starts with sprint planning done by the Scrum Master, Product Owner, and the rest of the team, consisting of a meeting divided into two parts, each timeboxed to four hours. The Product Owner’s homework prior to sprint planning is to come up with a prioritized backlog for the product that consists of a set of items that the users and stakeholders have bought into. In the first part of the meeting, the Product Owner works with the team to select items that will be delivered at the end of the sprint based on their value and on the team’s estimate of how much work they will be. The team agrees to give a demo of working software that includes those items at the end of the sprint. This first part is timeboxed (for a 30-day sprint it’s timeboxed to four hours; for shorter sprints, it’s proportionately shorter) so at the end the team takes whatever they’ve come up with so far and uses that as the sprint backlog. In the second part of the meeting, the team members (with the Product Owner’s help) figure out the individual tasks they’ll use to actually implement those items. Again, this part is timeboxed based on the length of the sprint (but often takes less time than that). At the end of sprint planning, the items they’ve selected become the sprint backlog.
The team holds a Daily Scrum meeting every day. All team members (including the Scrum Master and Product Owner) must attend,19 and interested stakeholders may attend as well (but must remain silent observers). The meeting is timeboxed to 15 minutes, so all team members must show up on time. Every team member answers three questions: What have I done since the last Daily Scrum? What will I do between now and the next Daily Scrum? What obstacles and roadblocks are in my way? Each team member must be brief; if an answer requires discussion, the relevant team members schedule a follow-up for immediately after the meeting.
Each sprint is timeboxed to a specific length decided during sprint planning: many teams use 30 calendar days, but this length can vary—some teams choose two-week sprints, some choose one month (and again, the planning timebox should vary accordingly). During the sprint, the team builds the items in the sprint backlog into working software. They can get help from people who are not on the team, but people who are not on the team cannot tell the team how to do their jobs, and must trust the team to deliver. If anyone on the team discovers partway through the sprint that they overcommitted or that they can add extra items, they need to make sure the Product Owner knows as soon as they realize the sprint is in danger. He’s the team member who can work with the users and stakeholders to reset their expectations, and use that information to adjust the sprint backlog to match the team’s actual capacity. And if they find that they’ll run out of work before the sprint ends, they can add more items to the sprint backlog. The team must keep the sprint backlog up to date and visible to everyone. In very abnormal cases and extreme circumstances, the Product Owner can terminate the sprint early and initiate new sprint planning if the team discovers that they cannot deliver working software (e.g., a serious technology, organizational, or staffing issue comes up). But everyone needs to know that terminating the sprint is rare, and has an extremely negative cost in terms of their ability to produce and deliver software, and seriously impacts the trust that they’ve built up with their users and stakeholders.
At the end of the sprint, the team holds a sprint review meeting where they demonstrate working software to users and stakeholders. The demo may only include items that are actually done done20 (which, in this case, means that the team has finished all work on it and tested it, and that it’s been accepted by the Product Owner as complete). The team can only present functional, working software, not intermediate items like architecture diagrams, database schemas, functional specs, etc. Stakeholders can ask questions, which the team can answer. At the end of the demo, the stakeholders are asked for their opinions and feedback, and are given the opportunity to share their thoughts, feelings, ideas, and opinions. If changes are needed, that’s taken into account when the next sprint is planned. The Product Owner can add the changes to the product backlog, and if they need to be made immediately they’ll end up in the sprint backlog for the next sprint.
After the sprint, the team holds a sprint retrospective meeting to find specific ways to improve how they work. The team and Scrum Master (and optionally the Product Owner) attend. Each person answers two questions: What went well during the sprint? What can improve in the future? The Scrum Master takes note of any improvements, and specific items (such as setting up a new build server, adopting a new programming practice, or changing the office layout) are added to the product backlog as nonfunctional items.
And that’s it. Simple!
Well, maybe not. If it’s so simple, then why aren’t we all doing Scrum? For those of us who are doing Scrum and following all of the rules, why do so many of our team members feel like we’re only getting better-than-not-doing-it results? What’s missing?
Hover Puppy Software is a small company that builds websites and mobile phone apps, and it’s had a very good few years. Six months ago, their latest mobile phone app had enormous sales, and the CEO decided to invest the earnings back into the company by starting up a new project, a website called Lolleaderz.com that lets users create leaderboards and achievements for pet videos.
When the project first started up, the team lead, Roger, wanted to go agile. He was really happy to discover that his team was just as excited as he was at the prospect. Another team in the same company, the mobile phone team, had used Scrum to develop their wildly successful app, and the idea had spread around the company. One of the team members started calling Roger the Scrum Master, and the role stuck.
The first thing Roger did was look for a Product Owner, but it wasn’t immediately obvious who that should be. Luckily for him, Hover Puppy is the sort of small company where everyone calls the CEO by his first name. Roger went to him and explained the situation, describing the Product Owner as the “king of the stakeholders” for the project. As it happened, one of the account managers, Avi, had just finished a project. The CEO introduced him to Roger, let them know that he was fully behind Scrum, and left it to them to figure out how to make it work.
The project seemed to go pretty well at first. Roger set up one-month sprints, and at the start of each sprint, Avi came up with a backlog of features to build. Roger set up a Daily Scrum meeting, and Avi blocked it out in his calendar so he could show up every day. The first sprint was great—everyone was getting along, and the team was making progress. At the end of that sprint, the team gave Avi a demo of a simple version of the website with a few features they’d all been planning. It seemed like this Scrum experiment was working.
Over the next few weeks, cracks started to show in the project, but it still looked like it was mostly on track. One account manager was working with one of her clients, a movie production company, to put ads for their upcoming summer blockbuster into each of the Hover Puppy websites. The team had promised Avi that they’d be able to give him a demo of this feature at the end of the sprint. But they ran into technical problems early on, and it ended up being pushed to the next sprint. Roger explained that Scrum teams always deliver working software, so if a feature’s not done at the end of the sprint, it just gets pushed to the next one. But they weren’t really sure if this was how Scrum was supposed to work.
Things seemed to get slower and slower with each sprint. By the end of the third sprint, Avi felt like he was spending more and more of his time working with the team, and had less time each day to do his real job of working with customers. At the beginning of the project, he thought he’d have control over what they did. Now he was starting to feel like he drew the short straw when he was chosen to be the Product Owner for the Lolleaderz.com project. Roger got upset when he heard that Avi had started to complain to the other account managers that the team was hard to work with.
Worse, the team was really getting fed up with all of this Scrum stuff. Before Roger and Avi even got a chance to work though that problem, another minor crisis erupted when three of the developers pulled Roger aside and started to complain about the new Daily Scrum they had to go to. One of them said, “I’ve got a lot of work to do already to get this out the door, and these meetings are just wasting our time. Why do I need to sit through your daily status updates, and wait for you to give each person the next day’s tasks? Can’t you just email them to us?” Roger didn’t have a great answer for that. He said that those were the rules of Scrum, and that was enough for the developers to keep going to the meetings. But none of them were particularly satisfied with that answer, and Roger was starting to wonder if the developers had a point.
One big blowup happened when Roger was planning the fourth sprint. Avi insisted that the feature that let users give “paws up” ratings to videos had to go in; otherwise they’d start to lose advertisers. But the database work for that feature was intensive, and needed their star database administrator (DBA) to make major changes to the data model, which meant that now he wouldn’t have time to write the stored procedures. It looked like they were going to be a week late with the feature, but pushing it to the next sprint wasn’t an option.
It was now six months and five sprints into the project. Roger felt like Avi was making increasingly larger demands of the team. Avi was frustrated that the project wasn’t delivering a website that he could sell to his customers. The team was supposed to be done with the video tagging page and the social media feeds two sprints earlier, but they still hadn’t delivered them. At the latest account managers meeting, Avi blamed the team for being late. Roger knew the project was at risk of getting cancelled.
Roger had had enough. The project that had started so well had turned into a monster. He felt like he was in over his head, but didn’t know how to fix things. He went back to the books and websites where he’d learned about Scrum, and from everything he read it looked he was doing everything right—at least on paper. He had sprints, Daily Scrum meetings, and retrospectives. He was working with the Product Owner to prioritize the backlog, and he pulled the most valuable features from the backlog to work on during each sprint, working with the team to estimate them so they’d fit, and assigning them to developers.
Roger had just given the team the best pep talk he could: “I was just pulled into the CEO’s office. He’s not happy that we’re behind, and Avi isn’t happy either. Look, I’ll protect you from this and take the heat for the problems. But we need to work on our estimates, because they were way off—and I really need you to make up for it by working the weekend to get this right.”
This was the third time he’d asked them to work nights and weekends in the last two months. The project felt just like past projects that had gone off the rails.
So what went wrong? Can you spot the problem? What would you do if you were asked to take over as Scrum Master for this project? Think about the agile values and principles. Is there a way to apply them so the project might run better?
Every Scrum project has a Product Owner, a Scrum Master, and a team. But not every project that has those roles assigned to people is an effective Scrum project. A Product Owner in Scrum acts differently than a Product Owner on a typical “big requirements up front” waterfall project. A Scrum Master does not do the same things as a command-and-control project manager, or technical team lead. When the Scrum Master, Product Owner, and team start to work together instead of separately, the project begins to look more like Scrum.
On a command-and-control project, the project manager is the owner and maintainer of the schedule and plan. He talks to the stakeholders, gets the requirements, breaks down the work, obtains estimates from the team, assigns tasks, and builds out a schedule. This is the approach that Roger took with the Lolleaderz.com project—he got the requirements from Avi, got estimates from the team, and assigned each person his tasks.
Team members on a command-and-control project have a natural CYA urge to wash their hands of project problems when they’re caused by someone else’s plan. If someone has exclusive ownership of the schedule and the plan, the rest of the team and the Product Owner are happy to let that person make those decisions.
That’s one reason why Scrum doesn’t have a separate role for a person who owns the plan. If a team has separate roles for owning the plan and following it, it’s too easy for individual team members to say, “It’s that guy’s problem” when they run into trouble. And almost every project runs into trouble at some point, because almost all project problems either cause or are caused by planning problems. So the Scrum Master doesn’t own the plan. He may help the team to create the plan. But more importantly, he guides the team’s use of Scrum and its practices, and helps everyone feel like they own the plan together. The practices and values of Scrum help them all feel that sense of ownership.
Imagine a conversation between the CEO and the Scrum Master about a project. The CEO asks what he’s going to get with a $2 million investment in software over the next year. The Scrum Master tells the CEO that he’s not sure yet; they’ll have an update every month, and at the end of the project they’ll have at least $2 million worth of software. No CEO in his right mind would ever approve that project. Why not?
The reason the CEO would never approve that project is that it’s not based on a real commitment. Commitments are promises made by people to get certain things done, usually by a certain time. And a real commitment comes with an additional responsibility to alert everyone and come up with a solution when things change in a way that could mean that the promise won’t be kept.
Most of us have been in meetings where a team member says that they didn’t sign off on the deadlines they were committed to, so the fallout for not meeting the deadline isn’t that person’s fault. This happens because many inexperienced project managers make the mistake of thinking that once a task is written into a plan, the team is automatically committed to it.
Plans don’t commit us. Our commitments commit us: the plan is just a convenient place to write those commitments down. Commitments are made by people, and documented in project plans. When someone points to a project plan to show that a commitment was made, it’s not really the piece of paper that the person is pointing to. It’s the promise that’s written down on it that everyone is thinking about.
On a Scrum team, the Product Owner is the person who made the commitment to the company. He’s the person who has to stand up and promise something specific that will be delivered at the end of the project. The Product Owner is keyed into the real business goals that the project is meant to fulfill. The more effectively he can get the team to understand those goals and feel committed to meeting them, the better the project will go. And when the project inevitably runs into trouble like technical problems, a change in the business, or people leaving the team, the Product Owner has to find a way to keep the team’s understanding of goals current, and keep up their feeling of commitment. He makes day-to-day decisions based on business changes, and meets with the team every day to make sure they understand how the backlog and the goals of the project are changing.
The Product Owner doesn’t just sit around and wait for the sprint to finish. The Product Owner’s job is to own and prioritize the backlog, to be the voice of the business to the team, to help them understand what stories and backlog items are most important and most valuable, and to make sure that everyone understands what it means to be done done with a backlog item. During sprint planning, the team works collectively to choose the items to move from the product backlog into the sprint backlog based on their value and how much work they’ll require, but the Product Owner is the one who guides them through this. And his job doesn’t end there.
The Product Owner has a very active day-to-day role in the project. Like all good agile teams, Scrum teams rely heavily on face-to-face communication to understand exactly what they’re building. The sprint planning that happens at the beginning of the sprint gives everyone enough information to get started, but that’s not nearly enough time for the Product Owner to communicate to the entire team all of the details about what they’ll be building. So during the sprint, the Product Owner works with all of the team members every single day, answering lots of detailed questions and giving them specific answers about what they’ll be building and how users will use the features they’re working on, as well as making lots of small decisions about how the product will work.
The Product Owner has the authority to make these decisions. (If he doesn’t, he’s the wrong person for the job! Scrum depends on the ability of the Product Owner to make decisions on behalf of the business, including accepting completed work.) But he doesn’t have all of the information—there are usually many users and stakeholders, and they have valuable information and opinions as well, so the Product Owner also spends a lot of his time talking to them to get answers that the developers need. He also uses this as an opportunity to stay on top of any changes that happen, and to keep the product backlog up to date so it reflects the latest changes to what the company needs. That way, if there’s a change to the relative value of different product backlog items, he can keep them prioritized so the team is ready for the next sprint planning session.
A Pig and a Chicken are walking down the road.
The Chicken says: “Hey Pig, I was thinking we should open a restaurant!”
Pig replies: “Hm, maybe; what would we call it?”
The Chicken responds: “How about ‘ham-n-eggs’?”
The Pig thinks for a moment and says: “No, thanks. I’d be committed, but you’d only be involved!”21
So on a Scrum project, who’s an involved “chicken” and who’s a committed “pig”? How is that different than an ineffective waterfall project? It all boils down to how the team members, project manager, and Product Owner act.22
Scrum teams often talk about roles in terms of pigs and chickens. That’s shorthand for whether an individual person in that role is simply assigned to the project (a chicken), or if he or she is truly committed to its success (a pig). Think back to your own projects: did you always consider your own success or failure truly dependent on that project’s success?
The fact is that the majority of people in their careers consider themselves chickens. They want to contribute to the project, but it’s risky to have their actual success at work—reviews, raises, future career goals, continued employment—dependent on that project. It’s also easy to forget that people work for money. Why do people do their jobs? What really motivates them? It’s not necessarily success at the current project.
People on an effective agile team—all pigs—genuinely feel that in order for them to succeed, their project needs to succeed. But it’s very easy for even an experienced Scrum team to fall back into a chicken mentality.
How many times have you, as a developer, chosen a technology for the project just because you want to learn it? As a project manager, have you chosen to work on agile projects because it makes you more marketable? Probably. We’ve all done this, at least a little bit. Everyone’s self-motivated, and it’s important to recognize that. But one thing that Scrum is asking you to do is: while you’re working on a sprint, making the project successful is more important than any other professional goal23 that you have. In other words, keep being a pig when you’re on a Scrum team.
When all team members are pigs, it means that everyone is committed. And it also means that everyone will do any job, if that’s what the project needs.
Here’s an example to help you understand commitment. Let’s say you’re the CEO of a company, but you’re also genuinely committed to the project. And let’s also say that the project team is getting a little tired, and it’s clear that they need coffee. If everyone else is working on something that just can’t stop right now, then you’ll go get coffee—and because you’re genuinely committed, even though you’re the CEO you feel like that’s actually the best use of your time, because it’s what the team needs right now.
On the other hand, every software project needs chickens. For example, every user is a potential chicken. How many times have you been frustrated by a feature of a browser, or word processor, or email client? Have you ever posted your feedback to a website or emailed it to a support team? That’s one way to make yourself into a chicken. If the team listens to you and fixes the problem, you’ve helped add value to the product. The more involved a chicken gets, the more value he or she can potentially add.
When you’re a chicken on a Scrum project, your opinion matters. You care about the outcome, and the team wants to hear what you have to say. But your job isn’t tied to the project: you have other goals (like, say, selling the product, or supporting the product, or running the company). They’re important goals, but they’re not the same as the specific things that the project needs to produce.
This is why Scrum teams—and especially their Product Owners—cultivate relationships with their chickens. One of the most effective ways that they have to do this is to release working software to their users on a regular and predictable schedule. This keeps the chickens involved, and helps them see the influence that they have over the project.
When a team member just does the tasks that are assigned to him by a project manager, but doesn’t feel like it’s really his problem when the plan runs into trouble, he’s acting like a chicken. He doesn’t feel a real commitment to the project, and more importantly, he doesn’t feel commitment to the other people on his team. On the other hand, when that team member genuinely feels a sense of responsibility for getting the plan right and building the most valuable software possible for the company and for his team, he’s acting like a pig.
The sad truth is that many companies expect team members to act like chickens, and not pigs. In fact, programmers often find that when they try to get involved with planning the project they’re pushed away from the process, because planning and decision making is a privilege reserved for managers, not rank-and-file programmers. (“Who do you think you are, some sort of manager? Get back to the cubicle farm, lowly programmer!”) When a company has that value built into their culture, it makes it very hard for a team to effectively adopt Scrum.
It’s far too easy for a Scrum Master to inadvertently encourage the team to be chickens by becoming the owner—or “keeper”—of the plan. This is a very difficult habit to break for a command-and-control project manager who’s learning to become a Scrum Master. It doesn’t help that many project managers have found a lot of success in their careers being the “keeper” of the plan. It’s easy for upper management to have one person to go to who’s accountable. (Some people like to refer to that person as a “single, wringable neck.”) It feels really good to be that person, because you feel useful and important, and that you’ve brought order to the chaos.
Planning is necessary. Taking planning out of the hands of the team isn’t. When the Scrum Master breaks down the work for a sprint by himself, goes and gets estimates from the team, assigns work to them, and checks up on their status, he’s acting like the “keeper” of the plan. He’s also encouraging them to be chickens instead of pigs.
Correcting a chicken attitude among the team is one of the most difficult barriers to Scrum adoption that Scrum Masters face. Teams often embrace the role of the command-and-control project manager because it takes the burden of having to think about the whole project off of their shoulders. But it’s central to getting the project done quickly and well, because development teams that take a short-sighted approach to code build poorly designed, fragile systems. When the team takes shortcuts and cuts corners, they can get something that looks like it works out the door today, but they end up with a fragile system that’s a nightmare to maintain down the road.
A Scrum Master can help encourage team members to be pigs by treating estimates as facts that have yet to be uncovered, not commitments to be wrung from the team. An estimate for a task is a fact because the team will spend a certain amount of time doing the work. The best estimates are real attempts to tell the truth about what will happen, not pie-in-the-sky, hopeful guesses that are used to please or placate a manager, or get out of the planning meeting faster.
A good plan is like a history that hasn’t been written yet. At the end of the sprint, the team will be able to look back and say with 100% certainty exactly what they worked on and how long they spent on each task. The sprint will be behind them, and whatever happened during that sprint will be established information—facts on the record. If they’ve done a good job, then the plan at the beginning of the sprint will be very similar to the facts at the end. The more the plan looks like what will eventually happen, the more accurate it was.
This is a real change in mindset for many teams. People on a team that’s used to treating a plan as a set of optimistic goals often find themselves struggling and failing to meet those goals. After all, it only takes one overly optimistic team member or manager to ruin everyone’s plan (and the next four weekends), even if he thinks he’s helping at the time. On the other hand, if the team treats the plan as their most realistic attempt at writing down what they think will actually happen over the next 30 days, they’re much less likely to bite off more than they can chew—and much more likely to come in on time without cutting corners, taking shortcuts, and ending up with fragile code.
To many project managers, this is a new way of thinking about planning. Traditional project managers often see the plan as a way to motivate the team, and commit them to a deadline. They often see the plan as a way to get the team to do something, as if the team will just sit around doing nothing if not for the plan. A project manager that wrings aggressive estimates from the team and “ratifies” them at the start of the sprint will often feel comfortable using the plan later to bully the team. This is why an attitude of “plan the work, work the plan” can create a fracture between the project manager and the team.
On an effective Scrum team, instead of demanding estimates and then holding each person accountable for them, the Scrum Master works with the whole team in order to discover each estimate—not just the individual person assigned to do each task. A Scrum Master and team who work together to do the best job they can of predicting what will happen, and who keep working together throughout the project to keep that picture as accurate as possible, will give themselves the time to do the best work they can—and will end up delivering more valuable software as a result.
Team members also feel more committed when they participate in assigning the work during a sprint planning session, rather than having the work assigned to them. In fact, if the team members really feel that sense of commitment, the assignments don’t even need to be made at the start of the sprint. Effective Scrum teams can decide during the sprint who will do which tasks, based on which people are available and the skills that they can bring to the task. This is one of the keys to understanding how self-organizing teams work.
This means that on effective Scrum teams, the team members don’t just commit to their individual tasks. Every single person feels a genuine commitment to delivering the most valuable software they can to their users and stakeholders. When everyone on the team shares this feeling, and agrees to deliver working software at the end of each sprint, we say that the team has made a collective commitment. Instead of committing to individual, micro-level tasks in a plan, the team is committed to delivering valuable items in the backlog. That gives the team freedom to adjust the way they’ll get the work done as they discover new facts about the project. For example, if partway through the project Roger and the developers realize that they need to change the way something is built, Avi will trust them as long as everything in the sprint backlog is delivered. And if it turns out that they were wrong (which happens!) and they won’t get everything in the sprint backlog built, they can talk to Avi about the things he cares about—the backlog items—without having to justify every little detail.
This is one of the fundamental reasons that Scrum works so well, and makes the difference between a highly productive Scrum team and one that’s only gotten better-than-not-doing-it results.
There is no room for a chicken on a Scrum team. Product Owners are part of the team, which means they need to be pigs, too. This does not always come easy to Product Owners, especially if they feel like they “drew the short straw” to end up assigned to a Scrum team. Most stakeholders naturally want to be chickens, because more distance is more comfortable. (Sometimes they do want to be pigs, even when it’s not appropriate, because it will give them more authority to try to exert influence over the team. This is something that the Product Owner needs to sort out.)
There are ways that the Scrum Master and team can help encourage a genuine feeling of commitment in the Product Owner. The most important way is to actually listen to the Product Owner’s opinions and ideas, and recognize that he or she brings real expertise to the project that the team needs.
Many programmers take the attitude that programming is the only important thing in a project, and that every other aspect of the project should take a backseat to the technical details. What’s worse, many companies encourage this when they structure their teams around a core set of technical people. Separating project managers and Product Owners from the technical team in a company’s hierarchy encourages the developers to see everyone else as “outsiders,” and to value their opinions less.
The Product Owner doesn’t necessarily understand the technical details, but that’s OK. On a Scrum team, everyone brings the specific skills and knowledge that they have, and does the tasks that are most appropriate for them. A Product Owner brings a real, deep understanding of the goals of the project to the table. The more that the Scrum Master and team engage the Product Owner, ask about those goals, and try to understand them from the Product Owner’s point of view, the more committed the Product Owner will be to the project and the team.
Every company has its own culture that includes specific values. For example, some companies value separation of duties, where each person has a specific role to play, and is protected from having to be accountable for things that he or she can’t easily influence or control. Other companies value transparency, where information is shared freely and even low-level employees can influence management decisions. Neither of these is the “right” way to run a company. Every individual company has a culture that evolves over time, based on the way it’s managed and the decisions that are made.
Every methodology has values built into it. We learned in Chapter 3 that specific agile principles are often tied to (or implemented by) individual practices, and that those practices are an effective way for a team to bring each principle to the project. We’ve already seen that people on a team in a company that reserves decision making exclusively for managers will find it difficult to feel genuinely committed. The same goes for any value or principle: if it clashes with the company’s values, it presents a barrier to adoption.
But in a company where the culture matches the agile values and principles, an agile team will be much more successful than a command-and-control team. (This is one of the sources of the “astonishing results” that some agile teams report.)
A good first step in introducing agile to your company is to talk about the values, and how they might impact your company’s culture. If you find that your agile adoption runs into trouble, finding the mismatch between agile values and company culture can help you smooth out the transition (or at least help you feel better by understanding why things went wrong).
Self-organizing teams work differently than command-and-control teams because they have different values. In Agile Project Management with Scrum, Ken Schwaber discusses the five Scrum values: courage, commitment, respect, focus, and openness. Understanding self-organization starts with learning how these values are practical things that can be incorporated into your projects:
That level of commitment can be achieved when the team has the authority to make decisions in order to meet those goals, and everyone has a say in how the project is planned and executed. The ebook reader team in Chapter 3 was originally given a requirement to build an Internet storefront. To make the product successful, they had to ignore that requirement in order to deliver a project that was much more valuable. This was only possible because they were allowed to make that decision with just the team, the Scrum Master, and the Product Owner. They didn’t need to go through a bureaucracy to get it done.
When team members have mutual respect, they’re able to trust each other to do a good job with the work they’ve taken on. But that respect doesn’t always come easily to programmers and other technical people. Many programmers, especially highly skilled ones, often base their respect purely on technical ability. This can be a barrier to effective Scrum adoption. If a programmer doesn’t respect the Product Owner, he won’t listen to that Product Owner when they talk about the goals of the project.
A good Scrum Master finds ways to increase the team members’ mutual respect for each other. For example, he may show the programmers that the Product Owner has a deep understanding of how the users think and what the company needs. As programmers start to understand how that knowledge is useful for the project to be successful, they start to value and respect the Product Owner’s opinions.
When a Scrum team member is working on a sprint, that’s his only job for the duration of the sprint. He is free to do whatever work is needed to complete the sprint backlog, and handle any changes that are made to that backlog during the sprint. When every team member is focused on the sprint goals and given the freedom to do whatever work is needed to meet those goals, the whole team is able to organize itself and easily redirect whenever a change is needed.
On the other hand, a distracted team is a less effective team. There’s a myth in the modern workplace that people—especially programmers—work best when multitasking, because they can move onto tasks for a second project when blocked on the first one. This is not how people work in real life! Switching between projects, or even between unrelated tasks on the same project, adds unexpected delays and effort, because context switching requires a lot of cognitive overhead. It takes a surprising amount of mental effort to put down your current work and pick up where you left off on another project. You end up having to review what you did last time just to remind yourself what problems you were trying to solve. Telling a team member to switch to a task on another project requires not just the time it takes to do that new task, but also the time it takes to switch to the task and back—which can be almost as much time as it takes to actually do it.
Don’t buy it? Try this thought experiment. Say you’ve got two one-week tasks. And pretend that through some amazing bending of the laws of physics, multitasking doesn’t add any overhead at all. You can switch between those tasks seamlessly, without adding a single second of overhead or delay, so those two tasks will take exactly two weeks. Even under these perfect (and impossible) circumstances, it doesn’t make sense to multitask. If you don’t multitask, you’ll get the first task done at the end of the first week, and the second one done at the end of the second week. If you multitask, however, then you must have spent at least some time in the first week doing the second task, so it won’t get done until sometime the next week. That’s one reason why even if humans were good at multitasking (which we’re not), it doesn’t make sense to do it.
Multitasking isn’t the only way that team members are distracted. They will often be required to attend useless meetings and extraneous committees, to perform activities that are irrelevant to the project, and to do support work for other projects. A good Scrum team is given permission to ignore these distractions without risking their careers or promotions.24 (Support work for the current project that must be done now can be added to the sprint backlog—but only if something else is taken out to make it fit the timebox.)
When you’re working on a Scrum team, everyone else on the team should always be aware of what you’re working on and how it moves the project toward its current goals. That’s why the practices in the basic Scrum pattern are aimed at encouraging openness among the team members. Task boards, for example, allow everyone to see all of the work being done by each team member, and how much work is left to do. Burndown charts let each person gauge for him- or herself how quickly the sprint is meeting its sprint goals. The Daily Scrum, when done effectively, is an almost pure exercise in openness, because each person shares his or her tasks, challenges, and progress for the whole team to see. All of these things can help the team to create an atmosphere of mutual support and encouragement.
Creating a culture of openness for a Scrum team sounds wonderful and positive, and it is. But it can often be one of the most difficult things for a Scrum team to do, because it’s one of the most common ways for the values of Scrum to clash with the pre-existing culture of the company.
Many companies have a culture that discourages transparency, and replaces it with a rigid hierarchy that depends on opaqueness. Managers that establish a culture like this benefit from it in several ways. It’s much easier in an opaque organization to tell a team to meet an unrealistic goal (“I don’t care how you do it, just get it done!”), forcing the team to work overtime to meet it. And it gives the manager plausible deniability to CYA when the team inevitably fails to meet that goal (“It’s not my fault they screwed up!”).
This is why openness and self-organization can often be the untouchable “third rail” of a Scrum adoption. It’s a central concept to getting a Scrum adoption right, but it also requires the company to treat the team differently than in the past. Being exposed to the details of development denies the opaque manager the cover to CYA. Many fledgling Scrum teams have found their adoption efforts undermined from above once opaque managers started to see how the sausage was made.
Openness is threatening to a cartoonish, pointy-haired, opaque manager. But realistically, it can also be a difficult thing for even a good team to adopt. Think about openness from the perspective of a developer who is seen as an expert in one part of the code, or a project manager who is the “keeper” of the plan, or a Product Owner who is the sole contact for many of the users and primary decision maker about what goes into the software. Each of these team members has every right to see those things as their contribution to the project. It can be very difficult to open those things up to the team, and encourage the other team members to share ownership and make changes without first getting permission.
This is a very natural way for individual team members to resist openness. But when they get past this and share ownership—including accountability when things go wrong—of all of those things with the whole team, everyone benefits, because it’s the only way to trust each other and quickly deliver more valuable software.
When you choose openness over opaqueness, you make the team stronger, rather than making yourself stronger at the expense of the team. It takes courage to do that, but when you do, you end up with a better product and a better work environment.
Scrum teams have the courage to live by values and principles that benefit the project. It takes courage to ward off the constant pushback from a company whose values clash with the Scrum and agile values. This requires vigilance on the part of every team member, especially the Scrum Master. But it also requires each person to be willing to trust that delivering valuable software will help him or her overcome resistance to these values. This requires courage too, especially when it comes time to sit down for a review with the boss. It takes courage to say to yourself, “Helping this team produce valuable software is more important to me than bragging rights from my own personal contribution.”
So how would you build courage on a team? How would you get a team to believe in themselves, and believe that Scrum will not only help them build more valuable software, but that their company will see the value in their new methodology?
Back on the Lolleaderz.com team, Roger and Avi needed help—and they knew it. They also knew that there was another team at Hover Puppy that had a great experience with Scrum. Roger started talking to people on that team to see what the secret was, but he ended up more confused than before. It seemed like they were doing exactly the same things that the Lolleaderz.com team was doing: they also had sprints, Daily Scrums, retrospectives, a backlog, a Product Owner, and a Scrum Master. It seemed like both teams were doing exactly the same thing, but one team was getting great results while the other was slowly sinking. Roger and Avi sat down with Eric. Eric was Scrum Master on another team. He’d been having a lot of success with Scrum, and he was happy to help Avi and Roger to try to figure out what was giving them trouble.
The first thing Eric asked was, “Do you have a coach?” Roger wasn’t sure what that meant. “You know, a mentor? Someone to help you get Scrum right? There’s no way my team would have done as well as we did without one.” This was the first time Roger really appreciated how good a salesman Avi was, because by the end of the discussion he’d convinced Eric to sign on as Scrum coach for the Lolleaderz.com team.
The following Monday, Roger and Avi wanted to make a big production of introducing Eric to the team at the Daily Scrum. Eric asked everyone to just go ahead the way they normally did—he’d watch the team work, and then try to come up with a few small suggestions. It was a good thing that he said that, because only half of the team was there for the introduction, anyway—they all knew that the lead developer always went first, and because he usually had the longest update, the rest of the team mostly arrived halfway through his status report.
For the rest of the meeting, team members took turns talking to Roger about their tasks. Everyone told Roger the progress they’d made on the tasks they were assigned. They asked Roger for their next assignments. During an update, one person pointed out that they were still waiting for the sysadmins to fix the configuration on one of the web servers, and asked Roger what he’d do to get it fixed. Roger added it to his list of roadblocks to clear for the team. Eric watched the whole thing without saying anything.
The next day, Eric watched through another Daily Scrum, which went the same way. He noticed that one of the team members reported that he was 95% done with his task. The same team member was 95% done with the same task previously. Afterward, he asked Roger about it. “Yes, it looks like that’s going to slip. Don’t worry, I’m on top of it. I’ve already updated the schedule. He’s always late with tasks, so I added enough contingency to take care of it, and if it slips enough I’ll make sure that gets reported to Avi and the CEO.”
Eric set up a meeting later that day with Roger and Avi. He started out by explaining what he thought was the biggest problem. “Roger, you’re just using the Daily Scrum as a way for you to manage your schedule. What happens if that team member keeps slipping? You just update your schedule, and now you’ve done your job, right? Except that updating a Gantt chart in a folder somewhere doesn’t make the project any less late.”
Roger did not like hearing this. Neither did Avi, because he used that schedule to update the rest of the project’s stakeholders. Eric continued, telling Roger that he was using the Daily Scrum to get updates from the team. This made things even worse. “Of course I’m using it to get updates! That’s what it’s for, right?” Roger was beginning to have second thoughts about bringing Eric on as a coach.
Can you spot why Eric had a problem with Roger and Avi using the Daily Scrum to get status updates from the team, or maintaining the schedule based on those updates? If that’s not what Daily Scrums are for, then why do Scrum teams hold them?
The Daily Scrum is one of the most effective tools that a Scrum team has at its disposal. This is because it does two very important things for the team. It functions as an inspection of the work that the team is doing, so they can adapt that work to deliver the most value. And it gives the team the opportunity to make decisions at the last responsible moment, giving them the flexibility to have the right person do the right work at the right time. When everyone on the team uses the Daily Scrum to limit the amount of planning they do to include only what’s necessary to build the next increment of the software, the whole team starts to recognize it as a valuable tool and use it effectively.
A lot of developers who are new to agile are used to an entire world that revolves around programming. They come into work to a set of things that need to be built, and go home having built them. This is a very comfortable place for developers to be, because they can focus their jobs mainly on solving technical problems.
But every programmer knows what it’s like to put a lot of time and effort into building out a solution, only to find at the end that there was an important part of the problem that they didn’t understand—usually because it was never communicated to them.
This can be especially problematic on traditional, “big requirements up front” (BRUF) projects. Think about all of the steps a requirement has to go through on one of these projects before it gets to the developer. On typical BRUF waterfall project, planning works something like this:
The project manager needs to scope out the work, often in some sort of business requirements or scope and objectives document.
Managers need to sign off on the scope.
A business analyst needs to review the scope, then talk to the users and other stakeholders to understand their jobs.
The business analyst comes up with use cases, functional requirements, etc.
The programmer takes the requirements and generates an estimate.
The project manager takes the requirements and estimates, builds out a schedule, and reviews it with stakeholders and managers.
That’s quite a chain of events that has to happen before the development even starts. It’s no wonder that it’s so rare for requirements to make it perfectly through that “game of telephone” into a developer’s brain.
This isn’t just a problem for BRUF waterfall teams. Even a team that is able to rely almost entirely on face-to-face communication will have problems with misunderstandings and incomplete communication. After all, face-to-face communication is very efficient, but also less precise (although often more accurate) than written communication. It’s very easy for three people to have a great discussion and think they reached consensus, only to have three different impressions of what was discussed.
There’s an old saying: “sunlight is the best disinfectant.” That may be questionable medical advice, but it’s very good advice for project teams. The best way we have for users to judge whether or not the team is building valuable software is to put working software in their hands as frequently as possible. That’s called visibility (or transparency), and it applies to communication as well.
The Daily Scrum is an effective tool for visibility because it helps with these communication problems. When those problems happen, they come out in the work. Take the example of those three people who have three different impressions of the same discussion. What if all three of them now had to go off and do three separate tasks that get integrated at the end of a sprint? These issues, the small misunderstandings that cause incremental problems and friction along the way, pile up if they’re not caught. And as they occur throughout a project, they lead to a steady erosion of the quality of the code that the team produces. A small misunderstanding here or there causes a defect or two, which has to be ripped out when it’s discovered later, which requires a quick patch or a larger change. As this happens over the course of a long project, the quality of the codebase deteriorates over time.
On the other hand, think about the effect it would have if those three people took 15 minutes out of every day to sit down and ask each other three questions:
What have I done since our last meeting?
What am I planning on doing between now and our next meeting?
What roadblocks are in my way?
When the team members describe their work to each other in the same way every day, a lot of the problems that are caused by inevitable miscommunication get caught before they become costly (and caustic!). When everyone on the team inspects what everyone else is doing, they come to conclusions together and maintain a common understanding of the project’s goals and how they’re meeting them.
During the Daily Scrum, when one person describes what he’s working on, one of his teammates might have a suggestion for a way he can improve. If her suggestion is a good one, the work he does the next day will be better. She might also discover that he’s working on the wrong task entirely, a problem that’s most likely caused by miscommunication, which would also cause him to change his plans for the next day’s work. These changes are called adaptation. The daily cycle of visibility, inspection, and adaptation allows teams to continuously use feedback from their projects to improve how they build software. This is one of the most important features of Scrum. Scrum teams make decisions based on experience and on actual, known facts from their projects.25
This kind of feedback is the way that Scrum teams cut out the “middle man” of a disconnected project manager who may be too distant from the work, reduce the damage caused by “game of telephone” communication, and save time while improving quality. This cycle is an important feedback loop that the team can use to keep the project on track, and to make sure that everyone is communicating and on the same page.
It’s not always comfortable for programmers to play the role of “inspector” for all of their teammates’ work. However, even the most introverted team members typically get used to it, and many eventually look forward to the Daily Scrum. That’s because it’s the most efficient way that Scrum teams have of managing their communication and building a shared understanding of their work.
This is about the point where a smart, skeptical command-and-control project manager might be thinking something like this: “OK, we get it. Teams don’t claim to know everything about their projects up front in Scrum. Communication and shared understanding are important for a team. But work actually needs to get done, and that work needs to be assigned to people. How does that actually happen? What are the mechanics of an actual programming task, DBA task, testing task, or other task ending up on a programmer’s to-do list?”
There’s a common technique used by Scrum trainers and agile coaches teaching their teams to run better Daily Scrum sessions by letting them fail.26 When a team that’s used to command-and-control management arrives for the Daily Scrum, they typically look to the project manager or Scrum Master for guidance. That’s when he asks each of them for their status updates, and gives them their next assignments. This feels natural to a team used to having their assignments handed to them. An agile coach looking to help this team run more effective Daily Scrums might advise the Scrum Master, just this once, to stay silent. This will often lead to a minute or two of painful, awkward silence. Finally, someone will speak up and describe the work they did since the last meeting. If the team is lucky, the next thing that team member does is ask, “So what’s my next task?”
This is the moment when the Scrum Master really distinguishes himself from a project manager. A project manager would proceed to hand the team member a pre-parceled task. The Scrum Master recognizes this as the chance for the team to have a collective “a ha!” moment, when they finally figure out exactly where task assignments come from. He might do this by asking a question like, “What do you think you should be doing next?” Or he might simply remain silent, depending on the team.
The point is that the team members themselves are the source of the task assignments. Each person self-assigns her next task as soon as she’s done with the current one. This is done during the Daily Scrum because it gives the rest of the team the opportunity to give input, and help course-correct. For example, if a developer assigns himself a complex database optimization task, the DBA might step in and suggest that he leave that task for later, and promise to do it next.
Command-and-control project management starts with the assumption that tasks on a development project must be done by a specific team member, typically because that person has specialized knowledge or skills (like a DBA with database optimization skills). This seems like an argument for planning up front: that person’s tasks form a bottleneck in the schedule, and a team working on a deadline must plan around that bottleneck in order to avoid it.
Ironically, this actually turns out to be a very common source of project management problems. Very complex tasks are more difficult to estimate than simple ones. And tasks that require one specific person or resource carry more risk than tasks that can be done by any of several team members. So it shouldn’t be a surprise that complex tasks that must be done by one specific, skilled team member—like that complex database optimization task—are the ones whose estimates are most likely to be incorrect. What’s worse, project managers typically depend on that one skilled person for the estimate as well as the work.
We can’t know everything in advance. There are some decisions that you need to make at the beginning of the project (Java or C#? Windows or Linux? Mac or PC?). And there are even some tasks that absolutely must be done by a specific person. But in general, Scrum teams do not make task assignments at the beginning of the project, or even at the beginning of a sprint.
In fact, they don’t try to come up with a “final” sequence of tasks at all. The reason is that for most project tasks, and especially programming tasks, the team doesn’t really know exactly how long they will take until they start working on them, and they often only discover dependencies when they crop up. It’s very common for teams to discover missing tasks throughout a project. And it’s also common for a task that seemed small to end up large, and vice versa. Of course, while it’s common for teams to discover tasks that they missed during a sprint, that doesn’t absolve them of the responsibility of developing as complete or final a task list as possible during sprint planning.
So instead of decomposing the work into tasks at the beginning of the sprint, sequencing the tasks, assigning each of them to team members before any work is started, and tracking to that plan, agile teams follow a simple rule for scheduling. They make all decisions at the last responsible moment.
Flip back to the Lolleaderz.com story. Roger and Avi ran into their problem in the fourth sprint because the DBA ended up taking longer to do one of those specialized tasks that seem like they absolutely must be fully and formally scheduled from day one. This is a common project failure mode caused by overplanning. The project manager assumes that a set of tasks will be done by a single person, usually an expert with specialized skills. Those tasks tend to be the ones with the greatest risk of slipping. When they inevitably slip, nobody else can handle those tasks, so it causes cascading delays—or, more commonly, overtime that leads the experts to do poor work (and possibly start looking for a new job!).
A Scrum team making decisions at the last responsible moment might handle this differently. Instead of assuming at the beginning of the sprint that the DBA will do all of those tasks, the tasks are written on index cards (or some electronic equivalent) and go up on the task board in the “to do” column. During the Daily Scrum, everyone can see them, and someone will typically point to them and ask if those are going to cause problems later in the sprint.
Open source teams have a saying: “given enough eyeballs, all bugs are shallow” (Linus’s Law). The same thing applies to the plan, and the Daily Scrum is how the team gets eyeballs on it. When the team looks as carefully at the project work as they do at the source code, those eyeballs are much more likely to find the “bugs” in the schedule as well. It’s much easier for a team to discover bottlenecks on the fly during a Daily Scrum than it is for a command-and-control project manager to figure them out up front. And when the team can see them coming, they have time to find a way around them. Because everyone is vigilant, they often discover that the last responsible moment for some tasks is much earlier in the sprint than it is for other tasks. This is the value of the visibility-inspection-adaptation cycle: it gives the group the opportunity to identify the potential problems and come up with a solution together.
So what would have happened if Roger and Avi had been holding more effective Daily Scrums? Instead of assigning work to the team, they could have used the time to get everyone on the team working together to identify schedule problems and self-assign tasks. They all might have realized early on—by working together and talking about it as a team—that they needed someone other than the DBA to start working on stored procedures in order to make it to the end of the sprint. Or, if that wasn’t an option, they could at least have figured out that they’d bitten off more than they could chew, and set everyone’s expectations earlier in the sprint about what working software would be delivered at the end of that sprint.
During this meeting, every team member is accountable to his or her teammates, and should explain what happened if a commitment that he or she made in the previous meeting wasn’t met. Imagine yourself on a self-organizing team, where you truly feel accountable for the project. What do you need to know in order to do your job every day? The most important thing you need to know is what you’re working on. But if your team is truly self-organizing, you can’t rely on someone like a command-and-control project manager to decide for you what that is. So you need some other mechanism to get your current assignment. That’s the first thing that you get from the Daily Scrum: your next task. When it’s your turn to answer the question about what you’ll be doing between now and the next Daily Scrum, if you’ve finished all the work assigned to you so far, your job is to look at the tasks in the “to do” column and choose the one that makes the most sense for you and for the project. If it’s not the right choice, someone else who’s also acting like a genuinely committed “pig” will speak up.
The goal of the Daily Scrum is to identify problems, not solve them. If a problem hasn’t been resolved after a minute or two of discussion, schedule a follow-up meeting with anyone who feels they need to be involved. A lot of these follow-up meetings will be about who does what task. This is how teams self-organize: most tasks can be self-assigned, but some of them need discussion. It’s only through inspection during the Daily Scrum that you can tell which is which.
There’s no single person who’s the “keeper” of the schedule, and there’s no single person who’s more important to the project than anyone else. Obviously, some developers have more expertise than others. But good ideas can come from anyone, and if a junior person on the team has a good idea, don’t dismiss it just because it didn’t come from your top programmer. He or she may have spotted a serious problem with the tasks that the whole team needs to deal with. One way to keep everyone listening to each other for those good ideas is to have a different team member start the Daily Scrum each day.
Even though we hold these meetings every day—some Scrum teams even refer to them as “ceremonies”—everyone needs to be fully present and engaged every step of the way. It’s easy to treat the three questions that each person answers—What have I done since the last Daily Scrum? What will I do until the next one? What’s blocking my progress?—as a ritual that just needs to be adhered to for some reason everyone’s forgotten. Rituals tend to fade over time, because people see them as perfunctory. These questions are the core of the Daily Scrum because those are exactly the things that the team needs to inspect every day in order to find the project problems early. For example, an effective way to spot a bottleneck caused by too many tasks that have to be done by a single person is to have each person answer the question about blocking progress, because the first person whose progress will eventually be blocked by the bottleneck will spot it earlier than anyone else.
This includes testers, business analysts, and anyone else on the team—and the Product Owner, too. They’re all genuinely committed. The Product Owner has an especially important job to do because he or she keeps everyone up to date on what tasks in the backlog are most valuable to the users and the company. The better the team understands the value they’re delivering, the more accurately they can aim the software at the users. The Product Owner should also answer the three questions along with the rest of the team, because it’s easy for the team to forget that he or she has an important job to do for the project too, and the answers help the rest of the team understand that job. (It turns out that talking to users, understanding their business, and managing that backlog really is a full-time job, and the developers respect that job more when they see that firsthand!)
A typical status meeting is a great example of a “ritual” that we have to go through every week. Because we’re all familiar with the ritualistic aspect of it, it’s something we rarely question, or even think about. The status meeting is supposed to serve two purposes: keeping everyone on the team informed, and keeping management informed. But for most teams, this is really just a one-way communication, a bunch of individual two-person conversations between a single team member and a project manager. You can avoid this by making sure that each person in the Daily Scrum is really listening. (This means not checking email or cell phones, or doing work!) As the team members start to see the Daily Scrum as a way to catch problems early and avoid wasting developer time by going down the wrong path, it feels less like bureaucratic red tape, and more like a genuine developer-centric tool that they can use to make the code better.
When you’re looking for roadblocks, don’t just look at what you’re currently doing. Look several moves ahead by examining every item in the “to do” column to figure out if any of those items could have an impact. If there’s a potential problem, it’s better to bring it up with the team now and have it dismissed than to keep quiet and get burned by it later. That’s why inspecting the tasks requires trust between everyone on the team. If someone—intentionally or unintentionally—doesn’t accurately describe what they’re working on and planning to do, the rest of the team could miss out on a potential roadblock, which will cause bigger problems if it’s not caught early.
This is the “adapting” part of the visibility-inspection-adaptation cycle, and it’s what puts the teeth in self-organization. Let’s say the team identifies a roadblock during the Daily Scrum, and has a follow-up meeting where they realize that they made a serious miscalculation and won’t be able to deliver a major feature that they’d promised. Does it make sense to keep working a plan that they know won’t work? Of course not. The backlog and task board need to reflect the reality of the project, and if a problem is discovered, then the entire team must work together to fix the backlog and task board. This is where having a Product Owner who’s a pig can come in very handy, because he or she can immediately start setting everyone else’s expectations. Just remember, no matter how badly people will react to the change when they find out about it today, they’ll react even worse if you don’t tell them now, and they find out later.
After Eric, Roger, and Avi took a long lunch to really talk about how to use the Daily Scrum, Roger had an idea. At the next Daily Scrum meeting, he asked one of the junior developers to start the meeting by answering the three questions. When she asked Roger what her next task was, he just stayed silent. Nobody else said anything for about half a minute, and things got a little uncomfortable. Just as Roger started to wonder if this was really a good idea, one of the more senior developers spoke up. After a brief discussion that involved several other people on the team, the junior developer knew exactly what to do next. She pulled the index card for her task off of the “to do” column of the task board, wrote her name on the card, and taped it back up in the “in progress” column.
The rest of the Daily Scrum went really well. It seemed like all it took was that one discussion, and the team suddenly “got” it—they all started talking about each other’s tasks, and only had to schedule two follow-up discussions to determine who would do what. Roger was pleasantly surprised that he didn’t even have to go to one of them. The follow-up he did have to lead was with a developer who was 95% done with a task that had been 95% done for the last week. It turned out that there was a serious roadblock, and the developer needed help from someone but was afraid to ask for it because he didn’t want to waste his teammates’ time (and probably was a bit embarrassed, too).
A few Daily Scrums later, Roger was really happy when Eric pointed out that the team was starting to work together. And over the next week, he really started to feel like they were finally getting the real idea behind self-organization—and they were doing it together, as a team. Every day, the whole team worked together to choose the next day’s work, and helped each other stay on track and solve problems. They were using the Daily Scrum to course-correct as a team, based on a daily review of the work they were actually doing.
Things seemed to be going great, all the way up to the end of the sprint. The team rolled out a new version of the working software, just like they had after the previous six sprints.
It was a disaster.
Avi came back from the next stakeholder meeting utterly crestfallen. He’d expected all of the account managers to be really excited about the new version of the Lolleaderz.com achievement editor that let users create their own achievements and share them on social networking sites. Plus, the team had updated the banner ad feature to allow each account manager to give customized pages to each of his or her accounts that showed up-to-date information on their page views and advertising costs.
Instead, most of the account managers were confused and caught off-guard. They felt like nobody had told them that so many things would change. Suddenly, each of them had dozens of voicemails from clients who were asking about these new features. In the past, they could rely on Roger’s schedules to give them plenty of advance notice to start selling the new features. Now the world was changing quickly, and they felt like they didn’t have time to keep up.
Avi had even worse news. Some of the stakeholders were asking for the old schedules back, and wondered if the team could hold off on releasing anything until the next quarter. It sounded like the company wanted them to stop using Scrum entirely and go back to their waterfall process. This was terrible!
Or was it? Eric heard the same news as Roger did, but instead of seeming despondent he seemed oddly optimistic. Why do you think he was?
For some projects, sprint planning is easy—like when it’s mainly a matter of finally getting around to building things in the backlog that people have been asking you to do for months. If there’s a feature that your users keep demanding, and you make that a high priority, it’s an easy win. That’s when sprint planning is just common sense.
But sometimes sprint planning is more nuanced than that. Often, sprint planning requires you—and your whole team—to think about what your users need, and what they value, in a way that you never have before. When people talk about Scrum being hard, this is generally what they’re referring to.
Luckily, effective Scrum teams have a not-so-secret weapon to attack this problem: the Product Owner. When the Product Owner really takes the time to understand what the stakeholders need and what will give them value, he can help the team aim each sprint at the problems that the company needs solved first. By giving the team visibility into that value, and helping them come up with a new plan for each sprint to deliver that value, he can help them turn a merely incremental process into a truly iterative one. And when the team is holding effective retrospectives at the end of each sprint, the Product Owner can help take the lessons the team learned back to the company so that everyone’s expectations can stay in line with what the team can actually deliver.
If you plan out timeboxed sprints, stick to them so that when the time runs out the team stops all work, and have a team that delivers working software at the end of each sprint, you’ll get a lot of benefit from that adoption. You’ll get a routine checkpoint that helps you stay on top of the quality of the software. It allows your Product Owner, users, and stakeholders to see functionally complete versions, so you can all see how the features being built can fit together. This significantly lowers the risk of integration because it keeps the team from waiting until the end of the project to integrate features built by different people, only to discover that those features didn’t quite work together.
Try this thought experiment to help understand integration problems. Let’s say that two of your team members are working on different features of a program that have to save the user’s current work to a file, but they do it differently. How many ways can you think of for those features to clash with each other? Here are a few to get you started: one feature might use a save icon, while another uses a file menu; or the two features could have entirely incompatible ways of accessing shared resources; or they could save files in incompatible formats; or they could overwrite shared data that the application manages; or there could be any number of other integration problems. Can you think of more potential problems that might arise in integration? If you’ve been building software for a long time, you don’t need to use your imagination—you’ve probably seen similar things happen many times.
Bringing all of the development together at the end of each sprint instead of waiting until the end of the project can help the team recognize and even prevent many of those problems. There are other benefits to working this way as well: better communication, more involved stakeholders, and project status that’s easier to gauge. When a team breaks a project into phases, it’s called incremental development. Scrum sprints are another way to break your project into increments, which is why Scrum is an incremental methodology.
But Scrum is more than that. Scrum sprints are about more than just delivering working software on a timeboxed schedule. Scrum is also about understanding the value that the software delivers, looking at exactly how that value will be delivered, and changing course if there’s a way to deliver more value. When methodologies and processes like Scrum work like that, it’s called iterative development. That’s why Scrum is both an incremental and iterative methodology.
An iterative process is one that makes progress through successive refinement. A development team takes a first cut at a system, knowing it is incomplete or weak in some (perhaps many) areas. They then iteratively refine those areas until the product is satisfactory. With each iteration the software is improved through the addition of greater detail.
An incremental process is one in which software is built and delivered in pieces. Each piece, or increment, represents a complete subset of functionality. The increment may be either small or large, perhaps ranging from just a system’s login screen on the small end to a highly flexible set of data management screens. Each increment is fully coded and tested, and the common expectation is that the work of an iteration will not need to be revisited.27
We already have a term for taking a first cut at a system, knowing it is incomplete or weak in some areas, and iteratively refining those areas: the visibility-inspection-adaptation cycle. A Scrum team that’s doing incremental development takes the same visibility-inspection-adaptation cycle that they already use for their Daily Scrums and applies it to the project as a whole. That’s the goal of sprint planning, managing the sprint backlog, and holding retrospectives.
Understand what the company needs most, and bring that knowledge back to the team
Understand what software features the team can potentially deliver
Figure out which features are more valuable to the company, and which are less valuable
Work with the team to figure out which features are easier to build, and which are harder
Use that knowledge of value, difficulty, uncertainty, complexity, etc. to help the team choose the right features to build in each sprint
Bring that knowledge back to the rest of the company, so they can do what they need to do to prepare for the next release of the software
The Product Owner has a very specific job to do for the project. He or she owns the product backlog, and selects the highest priority items to ask the team to work on during the sprint. He works with the team during sprint planning to decide which of those items go into the sprint backlog (which is owned collectively by the team), and accepts those items that have been finished by the team on behalf of the company. This means that the Product Owner has to have a lot of authority—and a Product Owner who doesn’t really have the authority to make those decisions (or who does, but is afraid to make them) is the wrong person for the job. It also means that the Product Owner has to have a really good sense of what’s valuable to the company. He or she can talk to other people to figure out what items are more or less valuable, but in the end it’s up to the Product Owner to make the final decision about the priority of everything in the product backlog.
This is why it’s so important that the team and the Product Owner agree from the very beginning of the sprint about what each of those items consists of. When they’re planning out the sprint backlog together, they all need to agree on what it means for each item to be “Done”—not just done, but “Done.” A backlog item is “Done” if it can be accepted by the Product Owner and delivered to the rest of the company. If each item doesn’t have a clear, unambiguous definition of what it means to be “Done,” there will be a lot of confusion and, almost certainly, heated arguments at the end of the sprint. But if everyone has a clear idea of what it means for each item to be “Done,” the team always has a good sense of how far through the sprint they are at any time.
Sprints are timeboxed, often to 30 days (although some Scrum teams choose shorter sprint lengths, often two or three weeks). When the sprint ends, all items that are “Done” are accepted by the Product Owner. Any items that are not “Done” go back onto the product backlog—even if the team did a lot (or even most) of the work for them. This doesn’t mean that team has to back out the work, delete source code, or undo anything. It just means that they don’t get credit for having finished the item until it’s actually “Done” and accepted by the Product Owner.
This is valuable because it makes sure that the team never gives the users the impression that they delivered value that wasn’t actually delivered. It’s always better to be conservative about what commitments were delivered. The sprint review is the only time that the whole team actually stands in front of their users and stakeholders to demonstrate what they’ve been working on for the last 30 days. If they didn’t meet all of their commitments, everyone on the team has to look their users in the eye, and explain what they did and did not deliver. This is a very powerful tool for helping everyone to feel a real sense of collective commitment. This is also an opportunity for the users and stakeholders to ask the team questions, and that human interaction helps everyone connect and get a much better sense of what is really valuable—and they can start to build up a genuine feeling of trust. The more everyone meets and talks about the software like this, the greater the sense of trust, and the more freedom the team will have in the future to build the software. But even though the users and stakeholders are present and talking to the team, it’s the Product Owner who actually accepts the work on behalf of the company.
In (hopefully) very rare cases, the Product Owner and the team discover that the sprint was either very badly planned, or that a serious change has happened that can’t wait until the end of the sprint. In this case, the Product Owner has the authority to break the sprint, halt all work, and move all of the items from the sprint backlog back to the product backlog. This should be an extremely rare occurrence, because it can destroy all of that hard-earned trust between the team and their users and stakeholders.
“Working with this technology will look great on my résumé” (if you’re a developer)
“If I prove myself on this project, I’ll get to grow my team” (if you’re a team lead)
“Meeting this big deadline will get me that promotion” (if you’re a project manager)
“If I land that big client, I’ll get a huge bonus” (if you’re an account manager, Product Owner, stakeholder, etc.)
We all think things like that. And it’s OK.
Every one of us has our own self-interest, and there’s nothing wrong with that. But motivating each team member as an individual isn’t the most effective way to bring a team together. On the other hand, when people work together toward a single goal, they can accomplish much more than they can on their own as individuals. So while we all have a basic need to get something out of our work, when each of us makes that thing our main concern, we get much less done than we do if we work together as a team.
Here’s a good example of how self-interest can damage a project: let’s say there’s a team member who can dump his uninteresting or irritating tasks on someone else, usually a more junior member of the team. Most of us have seen this situation. For example, senior developers are often so busy building new software that they put off fixing bugs. It’s not a coincidence that it’s much more fun for them to build new features, especially if they get to learn new technology along the way. And for a developer doing this, it would be great to have a maintenance team of junior developers (possibly in another city where developers are paid less) who can fix all of those bugs. This is so common that many companies explicitly set their teams up this way, with an “A Team” set of experienced developers to build new features, and a maintenance team of less experienced developers to fix bugs and create patches for software that’s been released.
This is a great, fun, and satisfying way to work for the senior developer who wants to use a “fire and forget” style of writing code, because he’s confident that any bugs he makes will get fixed by someone else without him even having to think about it. But while this is fun way for one person on the team in the short run, it’s a highly inefficient way for a team to operate in the long run. In almost all cases, the person who introduces a bug is the best person to fix it. He already knows the details of the code, and because he wrote it, it’s already written in a style that’s intuitive for him. Handing that bug off to someone else requires communication; sometimes just an email, but often additional documentation as well (like a bug report, or an updated spec). It also requires the person who’s fixing it to spend time looking through the code to figure out what it does. A bug that would take the programmer who created it a few minutes to fix can easily take hours or even days for someone else, especially if he’s more junior, and less experienced.
The “push off the grunt work” pattern is rarely found on Scrum teams because everyone on a Scrum team is genuinely committed. When a team has a culture where the more experienced developer does the “grunt work” in a few minutes when it’s fresh rather than making a more junior team member spend hours on it later, that can be another source of the Scrum “hyper-productivity” and “astonishing results” that seem to elude many teams.
Any team (even non-agile ones) can try to capture this productivity by setting up rules preventing senior team members from dumping “boring” maintenance tasks on junior team members. But a truly effective Scrum team doesn’t need to set up a rule for that, or separate rules for any number of situations. The reason is that on a Scrum team, everyone has a genuine feeling of ownership for every aspect of the project. If the senior team member feels that way, it would never even occur to him to “dump” these tasks—and, in fact, it wouldn’t feel like dumping at all. He would be the logical person to do the work, so doing that work would feel like the most important thing for him to be doing right now (just like in the example of the CEO getting coffee).28 The bug fixes and other maintenance tasks would simply get added to the sprint backlog, reviewed at the Daily Scrum, and like any other task it would get done by the right team member at the right time. (And the last responsible moment for fixing it is probably right now, because it’s still fresh in the developer’s head.)
On an effective Scrum team, everyone feels a true sense of ownership, not just of the code they build, but of the backlog, and of the work that everyone is doing to deliver working software. The sprint backlog represents a real commitment that everyone, even the most junior developer, feels that they’ve made to the users. That’s what Ken Schwaber meant by “collective commitment” in the quote at the beginning of this chapter: that everyone on the team shares ownership of the backlog, and feels personally responsible for delivering the most valuable software that they can to the users—including all of the features, not just the ones that they’re working on.
So how do you get that feeling of shared ownership embedded in the team, so that every single person—from the most junior developer to the senior technical lead to the Scrum Master and Product Owner—will voluntarily take on uninteresting or irritating tasks because he or she cares about the project?
Have you ever done volunteer work? Contributed to an open source project? Joined a club, or amateur sports team, or rock band, or church choir? Think about the last time you joined a group outside of your work or your family. Why did you do it?
You joined the group, and probably put a lot of effort and care into what you were doing for them, because you care about whatever it is the group was organized to do. If you were part of a voter drive, you care about getting people to participate in elections. If you were on a soccer team, you care about winning (and hopefully about playing well). So why should work be any different?
All of us are self-motivated. Minimally, we all work for money. If your job stopped paying you, you’d stop showing up. We have bills to pay, and many of us have families to feed. So paying us—and having a clean and safe office environment, giving us workable hours, and all of those other basic things that make up a work environment—is enough to get us in the door and at our desks during working hours.
But is it enough to get us to really care about building great software?
If you’ve ever worked on a team that isn’t really motivated, you know that the answer to that question is “no.” And the fact is, many of us have never worked on a team that’s truly motivated. But if you have, you’re probably thinking about that team right now, because it was likely the best work experience of your career. When everyone cares about building great software, things just seem to go better: people communicate more, argue less (although when they do argue, they do it much more passionately—and productively), and seem to just get things done.
Teams can be motivated by many things: getting an opportunity to work on a new technology or in a domain that they want to learn about, the possibility of a promotion, a performance bonus, working from home. Teams can be motivated by negative things, too: the boss will be mad, you’ll get yelled at (or less money, or fired!). Those things, positive or negative, motivate people to work for themselves and their own interests, not to come together as a team.
When teams are motivated effectively, it’s because they’re organized around an elevating goal. In Chapter 16 of our book Beautiful Teams, project management author and expert Steve McConnell gave us this definition of what it means to have an elevating goal:
If you’re out digging ditches, that’s not very elevating or inspiring. But if you’re digging ditches to protect your town that’s about to be attacked by an enemy, well, that’s more inspiring, even though it’s the same activity. And so the leader’s job, really, is to try to determine or frame the activity in such a way that people can understand what the value is.
Almost all software is built by teams, not individuals. For a team to work effectively, they need to be motivated together, not just as individuals. Teams are motivated best by elevating goals, which bring them together for a higher purpose that they all care about.
Delivering value can be a very effective elevating goal, one that can motivate the whole team. When a team has an elevating goal that they truly believe in, and they’re given the room to decide for themselves how to accomplish it (and the freedom to take risks) they will work hard to use the tools they have available to solve any problem standing in the way of that goal.
Elevating goals are aimed at value, but the term “value” can seem abstract or disconnected. On an agile team, value has a very real and specific meaning: software is valuable if it makes your users’ lives better. What happens if a senior vice president comes down to the team and says this: “Because of all your hard work over the last few months, we added 0.024% to our third quarter revenues. Great job, team!” That’s not particularly motivating for most developers, not even ones who are paid with stock options.
On the other hand, what if that same person came down to the team and said this: “Wow, I used to spend three hours every day just sorting through these numbers. Your software is so easy to use and works so well that I can do it all in 10 minutes. Thank you!” That’s much more motivating to most developers.
Developers—and by “developers” we mean all members of an agile team, even ones who don’t necessarily write code—are highly motivated by pride of workmanship. We want to build software that’s useful, and that people like and care about. We want our software to do a good job, and we want it to perform that job efficiently. We also want it to be built as well as it can be, which is why we spend so much time arguing about different designs, architectures, and technologies. These are things that teams actually care about. Making our users’ lives better—which is the most direct way that we have of delivering value—can be a real, honest, elevating goal.
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
The reason that this is our highest priority is because delivering value to our users is the most effective way that we have to motivate our teams—and that’s the driving force behind good sprint planning.
Why do we plan delivering a specific feature this sprint, rather than pushing it off to the next one? Because we’ve worked together as a team to figure out which features are most valuable to our users. This is why the Product Owner is so important—it’s his or her job to understand the users, and keep everyone on the team up to date on what they need most from the software.
A lot of managers have a crazy idea that if you don’t keep pushing developers, they’ll slack off by committing to do as little work as possible, to give themselves long deadlines. On most teams, this is the opposite of the truth. Realistically, developers have a tendency to be overly optimistic—which is why most of us have seen many projects come in late, but rarely seen them deliver very early. Don’t try to cram too many features into a sprint. (After all, the users only have to wait until the next sprint to get working software that includes the next set of features.) A good Scrum Master helps the team estimate the work and figure out what they can and can’t include.
Take advantage of the Daily Scrum meetings to figure out if the team is really going to finish everything they’ve promised for the sprint. If the plan needs to be changed, it’s the responsibility of the team to change it. If it becomes clear that the team won’t be able to finish all of the work in the sprint backlog, then the team should move some of it (starting with the lowest value items) out of the sprint backlog and back into the product backlog—and the Scrum Master should make sure that everyone on the team understands the change. Users that are used to seeing working software frequently are generally less jarred by a sprint review that does not include all of the features they’d expected, especially if the Product Owner has done a good job of managing their expectations.
Effective Scrum teams tend to be full of people who understand what their users actually need, and what’s valuable to them. The only way this can happen is if every person on the team understands what it will do for people who use it. How does it make their lives easier? What does it let them do that they couldn’t do before? How much time and effort will it save them? These things really matter to agile developers. The more that you talk about them and make sure that they matter, the better the software will be.
When Roger and Avi came to Eric with news of what had since come to be called the “stakeholder meeting from hell,” they couldn’t understand why he seemed optimistic. A few days later, the three of them left work early and headed to a nearby restaurant to talk it all through.
Eric started the conversation: “Why do you think all those account managers were upset?”
Neither Roger nor Avi really had a good answer. Roger started talking about how they’d done all of this work to become more agile, which in his mind meant being able to constantly adjust to a stream of feature requests, and build completely new products any time they were needed. Avi felt that he’d put a lot of work toward getting “plugged in” with the team, and was able to stream a whole bunch of great ideas from all of the account managers. They both felt that they’d given their stakeholders exactly what they’d asked for. “Look, I’ve got emails from each one of them asking for exactly what we delivered,” Avi said. “How could they possibly get upset by any of this?”
This is when Eric explained why, as an agile coach, he was happy with what he saw, even though the project seemed to be running into trouble. He explained that before, they were like a giant cruise ship that required many nautical miles to make a turn. Now they were more like a fleet of highly coordinated speedboats. It took a lot more communication, but it meant they could turn on a dime. And Avi had gone from being overwhelmed and a little antagonistic toward the team to being a true team member and an effective conduit between the team and their users.
By helping the team to put an effective Daily Scrum in place, Roger empowered the team to self-organize around changing priorities. But a new problem emerged, and Eric explained that this happens to many teams who have made it past the first set of challenges in becoming self-organizing. A newly self-organizing team now has the power to change direction extremely efficiently. And a Product Owner who finds himself embedded in that team now has the power to set that direction.
Eric gave them a good analogy: “Have you ever seen a team of firemen practicing to aim a fire hose? When they’re putting out fires, it just looks like they’re pointing it at the flames. But they had to spend weeks or months learning how to effectively move it, and communicate with each other so they move in the same direction. Your team used to be a garden hose, and now they’re a fire hose. It’s like one of those cartoons where the character is being whipped around by a fire hose that’s out of control—the two of you are scrambling to hold onto the end and point it in the right direction. Now it’s time for you to learn how to move together and point the hose at the fire.”
18 Jim Highsmith, Agile Project Management: Creating Innovative Products, 2nd Edition (Upper Saddle River, NJ: Pearson Education, 2009).
19 Does it seem unrealistic for your team to meet every day, maybe because people are spread between teams or have other commitments? Are you already starting to look for alternative approaches to the Daily Scrum, like replacing it with an online hangout or wiki page? That could be an indicator that your mindset needs to shift in order to get the best results from Scrum.
20 The idea of done done—work that is really, truly done, and there’s nothing else that needs to be worked on—is intuitive, but there are a lot of subtleties. This is another example of simplification, and we’ll come back to this concept several more times throughout the book. Keep your eyes open for it! Now that you’ve seen a few of these simplifications, we’re going to stop calling them out in footnotes, but we’ll still use them throughout the book.
22 Talking about pigs and chickens may seem a bit silly, but Scrum teams really do this. In fact, some older versions of the Scrum Guide actually included a section on pigs and chickens!
23 Just to be clear: a committed “pig” cares about the project’s success more than he cares about anything else in his professional life. There are a lot of other things in his personal life—like his family, for example—that he usually cares more about. If that’s not the case, it’s actually a problem with the team’s mindset, and it will interfere with sustainable pace.
24 Does this advice seem unrealistic? Does it touch a nerve? If the team doesn’t have permission to ignore distractions, then maybe they shouldn’t be considered distractions. A distraction that can’t be ignored is a requirement, and not a distraction at all. If you work on a team where ignoring a “distraction” can be a serious project problem, then your team’s mindset may be a barrier to adopting Scrum. This gives you something to work on with your team.
26 This technique, and other effective coaching techniques, is described by Lyssa Adkins in Coaching Agile Teams.
27 Mike Cohn, User Stories Applied: For Agile Software Development (Upper Saddle River, NJ: Pearson Education, 2004.
28 In fact, someone on an effective Scrum team reading this will probably be confused or even a bit upset just reading about a senior person who “dumps” tasks on others, because it’s such a foreign idea.