Apprenticeship Patterns

CHAPTER SEVEN

Conclusion

Craftsmen take pride most in skills that mature. This is why simple imitation is not a sustaining satisfaction; the skill has to evolve.

Richard Sennett, The Craftsman

When we started writing this book, we merely wanted to provide advice to aspiring apprentices—those who aspire to a craft. However, there are limitations as well as opportunities in a craft approach.

The violins and cellos that were made in the workshop of Antonio Stradivari in the 17th and 18th centuries are considered the finest ever made. They regularly sell for millions of dollars, and over the last 300 years there have been many attempts to reproduce them. However, as written in The Craftsman (p. 75): “the secrets of masters like Antonio Stradivari and Guarneri del Gesù, have indeed died with them. Mountains of cash and endless experiments have failed to prize out the secrets of these masters. Something in the character of these workshops must have inhibited knowledge transfer.” As Stradivari got older and could no longer be as active in the daily life of his apprentices, the quality of the instruments made in his workshop dropped. Since his workshop “revolved around the extraordinary talents of an individual” and Stradivari could not pass on his genius to his apprentices, his workshop died with him (The Craftsman, p. 76).

Stradivari’s apprentices included his two sons, and he had no incentive to hide anything from them. As far as we know he taught them everything he knew. Or rather, he taught them everything he thought was important for them to know. He failed precisely because of all the little bits of tacit knowledge that he wasn’t even aware were part of his skillset. His workshop’s downfall was embedded in all the subtle connections that he didn’t record because they didn’t seem important at the time, and all the tacit knowledge that he applied whenever he worked with an apprentice on some seemingly trivial task.

Stradivari didn’t share his knowledge widely enough, and neglected to teach his customers how to hold his students to the same standards as himself. Ultimately, the experience he had spent a lifetime gaining died with him. However, we should put this downfall into context. Musicians still describe the work of Stradivari’s students as “excellent, but no more than that” (The Craftsman, p. 77). The lesson we should learn from the example of the supremely skilled Stradivari is that “masters should be pestered to explain themselves, to dredge out the assemblage of clues and moves they have in silence within” and that we should push them to make the tacit explicit. Without apprentices who are enthusiastic to the point of being pushy, software craftsmanship will continue to exist only in isolated pockets of quality that have formed around small groups of peculiarly talented developers.

Software development is a craft precisely because we don’t understand it well enough to make it a codified discipline like science or engineering. Despite the best efforts of groups like the Software Engineering Institute and the Agile Alliance, our field is still one where individual skill is often the most significant determining factor in a project’s success. When we use the word skill, we don’t just mean how much computer science you know or the effectiveness of your development process or how much experience you have. We mean the union of all the things it takes to deliver working software. It includes, but is not limited to, all the lessons you have learned from the patterns in this book.

Skill matters so much because we don’t understand what we do well enough to write it down in a format where anyone can apply it and achieve the same results. Our customers would like software projects to be repeatable in the same way as scientific experiments. They would love to be able to hire any team of developers to build their system, safe in the knowledge that as long as the team possesses a minimal level of competence, they’ll get what they asked for. Instead, customers have to settle for putting together a team and hoping that it’s up to the task. They hope that if a team of a similar size with similar levels of experience was able to build something similar in the past, then maybe today’s team can do it too. Unfortunately, the range of skill levels in software development is wide, and the best of us can routinely do things that most of us consider to be impossible.

Moreover, most programmers think they are above average. The sad reality is that, due to the skewed distribution of skill illustrated by the following chart, most programmers are actually below average. This may sound counterintuitive, but if you imagine that we, Dave and Ade, are sitting at a table and Bill Gates joins us, then suddenly most of the geeks at that table have a below-average salary. That is to say, the people on the extreme ends of the curve in programming skill skew the distribution. When you combine that with the Dunning-Kruger (or unconscious incompetence) effect, which we discussed in Create Feedback Loops, you begin to see why so many software projects end up as failures. Often there is a mismatch between the skill level we have and the skill level that is needed to solve the problems we face. All we can do is try to improve our methodology. But no process, no matter how agile or lean, can tell you that what you’re trying to do is NP-Complete or violates the CAP conjecture. These may seem like obscure concepts to you but there are programmers out there who feel the same about regular expressions or HTTP or Unix. There is no substitute for having this knowledge and these skills if that is what a project needs to succeed.

When we say that something is a craft, one of the things we mean is that it is a discipline and a tradition that places a high value on skill. This includes acquiring, growing, and eventually transmitting that skill. We believe true mastery is shown in the effect you have on others by transmitting your superior skill.

In his book entitled Better: A Surgeon’s Notes on Performance Dr. Atul, Gawande tells the story of the physician Ignac Semmelweis (Better, p. 15). In 1847, Semmelweis worked out that “by not washing their hands consistently or well enough, doctors were themselves to blame” for the leading cause of death among women who gave birth in hospitals. He introduced simple practices, like asking doctors to wash their hands with chlorine in between seeing different patients, that reduced the mortality rate from 20% to 1%. However, in the process of achieving his superior results, he managed to alienate or offend his entire profession. Things eventually got so bad that his colleagues started to deliberately avoid washing their hands just to defy him. Semmelweis’s failure to properly explain his ideas or convince others eventually cost him his job, and also cost many people their lives in the 20 years it took before Joseph Lister discovered the same ideas. As Gawande puts it, “Semmelweis was a genius but he was also a lunatic, and that made him a failed genius” (Better, p. 17).

In the same way, many of the patterns and ideas in this book will generate conflict and resistance. This is partly because there are always people who benefit from the current situation and who fear they may lose something if the situation improves. However, we need to learn from Semmelweis’s failure. We can choose to explain our ideas clearly and persuade people to try out our approach. We can also choose to focus on creating a community of organizations that welcome positive change, and seek out people who aspire to mastery.

In software development, we don’t know exactly what constitutes mastery, but we do know what it isn’t. Being a genius, lucky, rich, or famous doesn’t make you a master. These things aren’t essential to craftsmanship. Skill across all facets of software development and the ability to transmit that skill in ways that move the discipline forward are at the heart of the craft.

One of the ways in which we can spot masters is that their students eventually surpass them in ambition and in achievement. Masters understand the danger of unquestioning deference to their authority, in that “the pursuit of excellence can create problems for longevity of organizations as in Stradivari’s workshop. Here the experience of doing high-quality work was contained in the master’s own tacit knowledge, which meant his excellence could not be passed on to the next generation” (The Craftsman, p. 243). As an apprentice, you should aim to become better than your teachers. And if they are good teachers, they should try to help you achieve that goal.

Mastery isn’t something that a person can claim for herself, as human vanity puts a limit on how accurate our self-assessment can ever be. If someone claims to be a master, ask her for her credentials. Does she point to her work? Assessing the work of someone who is more skilled than you are is notoriously difficult. You won’t understand why the things she makes look so easy are so hard. At best, you will be able to tell that this person is more skilled than you. But that isn’t enough for mastery. Being ahead of you on the path doesn’t make someone a master.

Does she point to her qualifications? After all, there’s no certificate for “Master Software Craftsman.” All a candidate for mastery can claim is that her peers and those who she believes are already masters think that she is ready to make the step up. This is admittedly a recursive definition. Only those who are already masters can say you’re a master, and you can’t tell if the people saying you’re a master are good enough to confer that honor until you’re a master yourself. Unfortunately there is no other way for a new craft to evolve. All crafts begin by stumbling into the world with unsatisfactory definitions and unclear standards. We have to put up with these until we have built up a community and a body of knowledge that can clearly demonstrate the skill of its practitioners.

Until then the best way to detect mastery is to examine the quality of the person’s work and that produced by her apprentices. The work and lives of the apprentices highlight the skill rather than the talent or luck of the master. Mere genius is not mastery, but if a person is able to train others to equal or surpass her genius, then it becomes evident that person is a potential master.

The authors of this book are not masters. At best, we are journeymen sharing what we’ve learned about the road ahead. On a scale of 1 to 10 we’d rate ourselves as 9s, but sometimes we meet people who make us realize that the scale goes all the way up to 100. There are many practitioners that we deeply respect, but the craft of software development is still lacking masters. This is not a problem. Software is a new craft—at best we’ve been building software for less than 70 years. So we shouldn’t expect to already have master software craftsmen.

How can we be so certain that true masters do not exist? We can’t. Masters might exist even though we haven’t found them. Absence of proof is not evidence of proof.¹ However, we would expect the existence of master software craftsmen to cause ripples through the whole of the software industry. Not just because their products and tools would be superior, but because their students would be as well. There would be a stream of excellent apprentices and journeymen flowing from particular sources. These students would stick out; their capacity to teach, “learn and change—and to do so faster than everyone else” would mean that they would pull away from the rest of us (Better, p. 227). As with Gawande’s “positive deviants,” these master workshops would be considered strange, but their results would be undeniably superior. Merely copying their practices would deliver significant improvements, but apprenticing yourself to these masters would be the only way to keep up as they continued to get better and better.

If we had told you at the start of the book that there are no masters, you might have been discouraged. Now that you have seen the patterns we have gathered in a few short years and how much of our craft can be learned, we hope you will see this fact as an opportunity and maybe even a challenge. We hope that the apprentices inspired by our work will argue: there are no masters...yet.

¹ Absence of Evidence Is Evidence of Absence: http://www.overcomingbias.com/2007/08/absence-of-evid.html.