Noel Rappin Writes Here

Looking at it very abstractly, the time a software team spends on a project goes into one of three buckets.

1. Actually writing the correct code.
2. Rewriting code, either because of bugs, miscommunication, or changing requirements.
3. “Process”, by which I mean all the other stuff a team does to share the status of the project with each other.

The size of the first bucket is dependent only on the size of the problem itself, although the problem is that it’s hard to predict the contents of that bucket in advance. The other two buckets are related to each other, in that a team that is better at efficient processes would tend to do less rewriting because they would have less miscommunication.

If I tell you that one team delivered a complex product in less time than another team (assuming that for some crazy reason I’ve asked two teams to solve the same complex problem), do you assume that the difference is because one of the teams was much faster at writing the correct code, or do you assume that one of them was much better at efficient process or avoiding rewriting code?

Even though we have a rich mythology about 10x engineers, for any kind of team size and complexity, the real, long-term difference between teams is almost certainly how they manage to avoid putting a lot of time into the rework bucket. (Slight tangent: the largest difference between teams that estimate accurately and inaccurately is likely how much of the problem complexity they just miss up front, rather than inaccuracy in estimating individual tasks). (Can I prove any of this? Of course not.)

Once again, wildly oversimplifying, there are two time-honored techniques for managing change over the life of a project.

1. Spend a bunch of time at the beginning of the project gathering all the requirements, anticipating every single avenue of change, and taking it all into account before you start.
2. Spend a very little time up front determining the highest priority actions, start coding, but code and work in such a way as to allow for the possibility of later change. This involves getting continual feedback on what you are doing.

I feel like I’m kind of tipping my preferences here by my wording, though honestly, reading back on them, they both seem kind of ridiculous when taken to their logical conclusion.

The thing about the up-front requirements gathering is that it’s impossible. You can’t know all the ways things will change before the project starts, not without limiting change during the coding cycle in ways that real customers will find way too confining. (Or, you could be the military and have an infinite amount of money to specify super-exacting requirements, I guess)

The other thing about up-front requirements gathering is that, even knowing that it’s impossible, it is crazy tempting. Many of us got into engineering to solve problems, and the idea that we can solve problems better with just a pinch of up-front design is, well, hard to avoid. Sometimes it’s even true, though my idea of the amount of up-front architecture that it’s worth starting with tends to be on the order of “hours to days” rather than “days to weeks”. (That said, as a practical matter, graphical design often needs to be just a little ahead of development.)

Which brings us to Agile. The subtitle of Kent Beck’s original XP book was “embrace change”, and central to the idea of XP in particular and Agile in general is the idea that change is coming and you are better off being ready for it. Many of the original XP practices — Test-Driven Development, Pair Programming, iteration planning — have the result of slowing down initial development, at least slightly, with the goal of dramatically reducing rework.

(By the way, this is my third attempt at writing this post, and both of the other two times, I ran into a deep digression about my own history with XP. I’ll spare you for now.)

Ultimately, even if you don’t like the XP practices (maybe you think TDD is dead, or you are allergic to pair programming), and even if you think the Agile Manifesto is vague and outdated, you still need to have some way to deal with change over the life of your project, because change will happen.

And if that way of dealing with change isn’t something like:

• Take a small step
• Verify that it was the right step
• Repeat

Then you are probably going to feel some pain. On a related note, doing this:

• Every now and then, think about what the biggest source of pain and friction is in your project.
• Improve it
• Repeat

Can lead to pretty dramatic team improvement over time. A regular retrospective that comes up with even one actionable change in your team is a big deal.

The XP practices are all designed to allow for quick feedback and continuous improvement, but not all the practices are suitable for every team. Every team, though, has the need for quick feedback and response to change. If you don’t like pair programing, do pull requests, if you don’t like TDD, write good functional tests for feedback. If you don’t like talking to your stakeholders… well, then you are in trouble, you should talk to your stakeholders.

This post didn’t go where I thought it would go (at one point, I was going to write an impassioned defense of old-school XP index card walls). Maybe someday. In the meantime, what can your project do to take smaller steps, or to eliminate friction and improve communication?