We’re back with When Books Could Code, Essay Two, Chapter B, Subsection 3, Paragraph 2, Line 7. I’ll continue the discussion of the first edition of Extreme Programming Explained by Kent Back, published in 2000.

In Part One, we talked up the problem that Beck identified with how software projects run, and in this post, we’ll talk about the set of solutions that he proposes. In the next post or two we’ll talk more about what happened after the book was released and how the second edition updated XP.

The XP Solution

The solution section of the book describes the original twelve practices. To this point in the book, Beck has given XP values, principles, activities, and now practices, and this all feels a bit like epicycles on top of epicycles – one of the apparent goals of the second edition was to clear up the relationship between all these moving parts.

The twelve practices are the signature feature of the first edition of this book. They are meant to be simple, even naïve, when done on their own, but they are more than the sum of their parts because they are “self-reinforcing”. This leads to the main tension of the book, which is between XP as a prescriptive model of twelve things that every team should do, and as a descriptive model of twelve practices that work, but where each team needs to find their own path. Many people took this book to be prescriptive, which Beck says was not his intention. That said, it’s not hard to see how the plain text of the first edition would give you the idea that these twelve things were the One True Way, especially if you wandered into any of the listserv discourse after the book came out.

I think the accurate reading here is that Beck identified a set of practices that he felt were objectively useful, and which he found became even more useful when “extremifying” them, and on top of which were even greater than the sum of their parts when done together. This is, if I may use the adverb, and extremely exciting set of discoveries, and it’s easy to see why Beck was eager to show what his team had accomplished.

I think, though, that Beck was being so careful to make the point that the practices support each other and that doing only one or two of them wouldn’t give you the full value, that he overshot a bit, and the book sometimes sounds like these 12 practices are the Only Way of XP, even if that was never really the intent.

The set of practices is very much colored by the state of software development in the 1990s when the book was written, meaning largely for software that was actually released to users, rather than server-side software that is just deployed. Like a lot of books written in the early 2000s, this book was written in a context that was significantly different the context in which it would later be read.

I would absolutely write an entire series with a separate post on each of the twelve practices because it’s fascinating to me the different trajectories they took. But in lieu of that, here’s a quick tour of the Original Dozen.

Planning Game — I was surprised going back to this one, it was not what I remembered. The planning described here is likely not what you think of when you think of Agile planning, the way planning is discussed in this edition was deprecated almost immediately by the book Planning Extreme Programming, by Beck and Martin Fowler. Most of what you remember about how Agile planning works is from that book, and from the second edition of XP Explained, which picked up on that new vision of planning.

Small Releases — Basically took over the web development part of the world, and now that software is largely distributed online, took over pretty much everything. Even release sizes that would be considered large by today’s standards are still much smaller than what Beck considered small in 2000.

System Metaphor — Even at the time, this was immediately the forgotten practice. The idea — that there is a metaphor used to pull the design together — is a good one, but it’s not necessarily applicable to all systems or possibly it’s hard or possibly it’s not something most engineers care about. In any case, it was rare at the time and it’s rare now.

Simple Design — Defined as a design that uses as few classes, as few methods, and as few actors as possible. This is foundational to moving in small steps – the complexity is only added where needed. I think it’s fair to say that, like a lot of YAGNI-related principles, this one is often honored in the breach. (YAGNI, the acronym for “You Ain’t Going To Need It) is another XP-related term that doesn’t come from this book, it comes from Ron Jeffries’ later book Extreme Programming Installed.)

Testing continually — Beck doesn’t quite stress testing first yet. The chapter on testing starts with “we will write tests before we code, minute by minute”, but the rest of the chapter doesn’t harp on that fact, focusing instead on the tests being isolated and automatic. “You should test things that might break” gets used here. The concept of Red/Green/Refactor comes from Beck’s next book, “Test-Driven Development”. (Martin Fowler’s Refactoring book, also published about the same time, assumes that you are doing testing first). I will note that Beck spends time here defending the idea that programmers will write tests at all, let alone write them first.

What happened to testing as a practice after this is chaotic, but the primary upshot, at least in the Ruby world, was that writing tests did become part of the expectation of production code, but that test-first development turned out to be a much harder sell overall, for reasons that are still a little confusing to me.

Refactoring — Between this book and the Martin Fowler Refactoring book (definitely a future book I want to write about), the XP/Agile folk were successful in promoting the idea that refactoring was a thing that you could do continually in small steps, as opposed to something you would take a six week break to do every couple of years. I think adoption has been imperfect, but the meaning and impact of the term definitely changed.

Pair Programming — Paring was arguably the most radical practice that XP suggested. “It is a subtle skill, one that you can spend the rest of your life getting good at”, Beck writes, and that bit of advice was largely overlooked.

I have a million things to say about pair programming. Here’s three:

• As far as I can tell, the number of Ruby/Rails shops that actually pair regularly is now effectively zero. I’ve worked at a lot of places that claimed to pair, and in 25 years, I’ve worked at exactly one place that actually did it regularly.
• Pairing is hard, and even while acknowledging that, Beck understates some of what makes it hard, sample quotes include: “What if two people don’t get along? They don’t have to pair” and “What if someone refuses to pair? They can choose to work like the rest of the team, or they can choose to look for work outside the team”. I submit that those are both glib glosses on hard problems.
• Pairing was always a hard sell to management, and the logistical problems are real. What if you only have three developers on the team? What if you are in an open space next to teams that need quiet? What if you are all remote? And that’s before you get into any interpersonal or power dynamics. “In my experience,” Beck writes, “pair programming is more productive than dividing the work between two programmers and integrating the results”. That has not been my experience, and I think it’s quite hard to achieve in practice.

Collective Ownership — Not much to say here, I’d say this got mostly adopted in the sense that teams don’t explicitly have individually owned code, but it’s still pretty easy to get code silos on teams that aren’t trying hard to avoid them.

Continuous Integration — Probably the biggest success of the XP practices, most teams in the web world take this even farther and do continuous deployment, and the definition of “continuous” in this book (integrate at the end of each iteration) is far too slow for most modern teams. There’s been a definite tension pulling teams out of Agile/XP style iteration planning because they want to deploy all the time.

40 Hour Week — “The XP rule is simple — you can’t work a second week of overtime”. The goal here is laudable, and I think that directionally many parts of tech got better at this, at least for a while. In the sense that this is a developer practice that developers control, I think that this is still largely aspirational.

On-Site Customer — The idea that a coding team should have continual access to somebody who can make product-level decisions is both obviously beneficial and remains beyond the operational capacity of many projects. The couple of times that I was able to work like this as a consultant were truly great, though.

Coding Standards — This, I think, has gotten pretty strong adoption. In the Ruby world, most teams use some flavor of Rubocop on all code, and nearly every language community that has sprung up in the last 25 years has a built-in or standard default linter or formatter or both.

I’d say the upshot here is that many of the individual practices have been widely adopted. The concept of this group of twelve being adopted as a group and reinforcing each other, that basically didn’t happen.

But see my post on entropy from a few years back for a slightly different look at what happened to the XP practices as a group.

Implementation

Section three of the book is about implementing XP and it starts with great advice:

1. Pick your worst problem.
2. Solve it the XP way.
3. When it’s no longer your worst problem, repeat.

This is great, it’s a little bit in tension with the idea of the twelve self-reinforcing practices, but it’s generally good advice. (I note that retrospectives are not an XP practice in this book, they were added later.)

Beck says:

Solving the most pressing problem also addresses the objection to XP that it is “one size fits all”.

But then later:

Step -1: Rearrange the furniture so you can pair program and the customer can sit with you.

But even later, quoting a book on teaching Skiing:

You will only see 20% of the improvement when you have done 80% of the exercises.

This poses a dilemma. Is XP all or nothing? Do you have to follow these practices to the letter or risk not seeing any improvement? Not at all. You can get significant gains from parts of XP. It’s just that I believe there is much more to be gained when you put all the pieces in place.

What I think is happening here is that Beck legitimately believes that the 12 practices are the natural end state for an XP team, but is offering on-ramps for teams to get up to speed. That said, I think the all-or-nothing question is the most muddled message in a book that is generally quite clear.

There’s a section about “What Makes XP Hard”:

• “It’s hard to do simple things”
• “It’s hard to admit you don’t know”
• “It’s hard to collaborate”
• “It’s hard to break down emotional walls. The smooth running of an XP project relies on the smooth expression of emotions.”

So… I mean, this is all directionally true, and it’s even directionally true that this is what made XP hard. But it isn’t exactly unrelated to the early community sense that if you failed at XP it was probably your own fault. (Ultimately, I think the missing word here is “trust”, but that’s another story.)

Then a chapter about when you shouldn’t try XP, which I very much appreciate. “The biggest barrier to the success of an XP project is culture… Any business that runs projects by trying to point the car in the right direction is going to have a rough time with a team that insists on steering”. True, I think, but also understates the extent to which many businesses are going to have people that want to steer. And I think it undersells the benefit of steering at larger scale. Or at least the perceived benefit of steering at large scale.

“Another culture that is not conducive to XP is one in which you are required to put in long hours to prove your commitment to the company.”

“Really smart programmers sometimes have a hard time with XP” (I think this is more accurately expressed as “raging egoists have a hard time with XP”. In my experience really smart programmers who are not raging egoists are fine with XP. Enthusiastic, even.)

“Size clearly matters, you probably couldn’t run an XP project with a hundred programmers. Nor fifty. Nor twenty.” Not that people didn’t try… But then, “the biggest bottleneck in scaling is the single-threaded integration process”, I think that one actually got solved, and the problem with larger teams is simply that it becomes increasingly risky to have large teams that don’t coordinate.

You also shouldn’t use XP:

• If you are using a technology with an inherently exponential cost curve. I think in practice this means if you have a lot of hardware customers or something where making a change has some kind of huge direct cost.
• In an environment where a long time is needed to get feedback
• If you have the wrong physical environment. “If you have programmers on two floors, forget it”. I do wonder the current state of Beck’s thought here as remote teams became more of a thing.

Then What Happened?

This book went out into the world with a bang. Many, many teams wanted to try it, and many struggled. It was clear that the ideas in the book resonated with developers, and the idea of having a process more focused on doing things than planning things was a big draw. I might be overstating it, but there was quickly a sense that the book over-indexed on the specifics of the few teams that were the initial adopters.

There was an XP listserv in the very early 2000s and I’m pretty sure it’s archives have gone into the Vast Beyond (the web archive has a main page, but not the individual messages, as far as I can tell). I would pay genuine money for a chance to read through those archives.

I’m looking at an archived monthly summary list from 2000 with the heading “no excuses programming” and messages from Kent Beck and Dave Thomas, and the specific messages aren’t stored, and to say I’m curious about the content of those messages is a wild understatement.

Anyway, I lurked on that listserv for a while, and I remember it being a lot of “I tried XP and it didn’t work”/”well you must not have been doing it right”, and in retrospect that should have been a stronger signal that it seemed to be at the time.

There were a series of follow-on books within a year or so that explained facets of XP in more detail, responding to feedback, extending and changing some of the practices, like planning, and adding new ideas like retrospectives. As we’ll see, the second edition was very much an attempt to consolidate.

XP was not the only lightweight process being proposed around this time, and in late 2001, seventeen people involved with XP, Scrum and a couple of others got together in a ski lodge and produced the Agile Manifesto, which is an umbrella term for the bits of the various processes that all these people could agree on.

(Parenthetically, somebody should do an oral history of this while enough of the original seventeen are still around to participate.)

From that point “Agile” became a huge buzzword, everybody wanted to be “Agile”, and the definition was fuzzy enough that almost everybody could claim it (boy, do I have a story… next time).

Next time, we’ll talk about how the second edition of XP Explained consolidated all this feedback, and then some thoughts on where XP and Agile went after the book came out.