Noel Rappin Writes Here

Speed matters. But not precisely.

There are only two things that matter when thinking about the speed of your automated tests:

• How fast can you run the relevant set of tests to let the tests be helpful in development?
• How fast can you run a complete, green build for deploy?

We’re talking about the first one in this essay.

In development, you want to be able to run the tests you are writing and a subset of tests that are most likely to break based on your changes. (I mean, ideally you would easily run all the tests, but let’s not go crazy).

The exact time doesn’t matter. What matters is which of these six groups your test speed fits in.

1. So fast you don’t notice
2. You notice, but don’t have time to lose focus or change tasks
3. Enough time for you to break focus, and check Email or Slack
4. Enough time to get a Diet Coke from the fridge
5. Enough time to eat lunch
6. Effectively infinite

If you are a modern web developer, you’ve likely never experienced tests in group one here, and you may not even believe that they can exist. (Just to confirm, I goofed around in a Smalltalk image for a while and you can live in group one there for quite a while.) You’ve probably only experienced group two on very small projects.

I don’t think it’s unusual in large Rails projects for them to be in group three if you want to run one new test, and easily groups four, five, and up if you are running any larger sequence of tests. On the other hand, the sample project for the book has a very small number of Cypress tests that run, more or less, in group two.

Test-Driven Development, in its most XP form, depends on being able to run tests very frequently. Once a minute or more, at least some of the time. Being able to run the tests that frequently is what enables you to write the tests and code in very small units and it’s what enables, say, refactoring in the small, baby step by baby step, and is also what enables TDD to prevent over-design. One of the hardest things I’ve found in teaching TDD is how much easier the process goes if the tests run very fast, it’s almost impossible to give the sense of it in text, because text takes longer to read then the tests should take to run.

Once upon a time I was taught a human factors rule of thumb: if you make a task 10 times faster, you’ve fundamentally changed the task, because people can use the faster task in completely different ways. A tenth of a second is three orders of magnitude from a minute and a half.

Bad things start to happen once you the test runs move into group three. It’s impossible to run tests once a minute if even the simplest test takes three minutes to run. Once the tests run long enough to break your concentration, you rationally start looking for processes where you run the tests less frequently. And your concentration is broken, so are continually changing context.

If all your experience is in groups three and up, that’s probably the process that feels natural to you. As for me, somewhere along the line, I developed habits, and one of them is that I tend to use tests as something of a spellchecker. This is a great habit if the tests take a couple of seconds, but can be frustrating if the tests take minutes. (Or when you blow a 40 minute CI build because the linter failed, which is a separate issue…)

Okay, fine. TDD is dead, and all that. But even if you aren’t writing tests first and even if you aren’t using them to guide the design of your code, tests are still feedback. And in other parts of the development process, we’ve all been trying to get faster and faster feedback.

But with tests, it feels like we are comfortable with making that feedback slower, and in a way that makes the feedback get slower and slower the more complex the code gets. (Dave Copeland would want me to point out here that a lot of coding communities don’t have automated tests at all, and yes, that’s a lot slower.)

One thing I’m trying to do in these essays is think out loud about how testing and other XP processes can work better in these conditions. For years, the more-or-less official TDD advocate position about testing when tests are slow was either time travel to not write the slow tests in the first place, or an awkward refactor to allow new code to be written with faster tests. Neither of these options strikes me as particularly practical.

But I don’t know what the best development process is when a single test takes a minute or more to run, or when all the plausibly affected tests takes 10 minutes or more to run. I just don’t think that what I’m currently doing or seeing is that best process.

Which brings me to these “entropy essays”. I’ll be talking about ways in which XP and XP related processes like testing, object-oriented design, pairing, and so on have fared when they come in conflict with web projects and with developers that aren’t fully on board with the process. We’ll talk about TDD, Objects, Pairing, CI, and whatever else seems to fit. My goal is to help you think about whether the process you have is suiting you well, both your team process and your individual hour-by-hour process of writing code.

If you think this’ll help you, keep reading, and share this newsletter with your friends and colleagues.