Story points: there’s no right way to do it

Story points as described represent an attempt to abstract estimation away from “amount of stuff done per unit time”, because we’re bad at doing that and people were traditionally using that to make us look bad. So we introduce an intermediate value, flip a ratio, and get the story point: the [meaningless value] per amount of stuff. Then we also get the velocity, which is the [meaningless value] per unit time, and…

…and we’re back where we started. Except we’re not, we’re slower than we were before, because it used to be that people asked “how much do you think you can get done over the next couple of weeks”, and we’d tell them, and we’d be wrong. But now they ask “how big is this stuff”, then they ask “how much capacity for stuff is there over the next couple of weeks”, and we tell them both of those things, and we get both wrong, so we still have a wrong answer to the original question but answered two distinct questions incorrectly to get there.

There’s no real way out of that. The idea that velocity will converge over time is flawed, both because the team, the context, and the problem are all changing at once, and because the problem with estimation is not that we’re Gaussian bad at it, but that we’re optimistic bad at it. Consistently, monotonically, “oh I think this will just mean editing some config, call it a one-pointer”-ingly, we fail to see complexity way more than we fail to see simplicity. The idea that even if velocity did converge over time, we would then have reliable tools for planning and estimation is flawed, because what people want is not convergence but growth.

Give people 40 points per sprint for 20 sprints and you’ll be asked not how you became so great at estimation, but why your people aren’t getting any better. Give them about 40 points per sprint for 20 sprints, and they’ll applaud the 44s and frown at the 36s.

The assumption that goes into agile, lean, kanban, lean startup, and similar ideas is that you’re already doing well enough that you only need to worry about local optima, so you may as well take out a load of planning overhead and chase those optima without working out your three-sprint rolling average local optimisation rate.

How retrospectives ban shoes

At the end of each sprint, we hold a retrospective. The book “Agile Coaching” by Rachel Davies and Liz Sedley says:

An iteration retrospective should help the team explore the following:

  • What insights do they have from the last iteration?
  • What areas do they want to focus on improving?
  • What ideas can they act on in the next iteration?

Take this too literally, and you end up adding the thing next time that stops you getting wrong the mistake from this time. “That time, the explosive was hidden in a shoe, so let’s add shoes to the list of banned items.”

What if it was just a mistake? Do you need to change the way you do everything to fix a problem you encountered once?

Retrospectives need to take a longer-term view. How are we doing, and how specifically did that change this sprint? Do we need to change how we do all of our tasks because it didn’t work for that one task? How many hundreds of tasks has our process worked for?

I discussed this with Steven Baker who described a team where the retrospective facilitator kept a risk register across multiple sprints. Yes, this thing happened and it was bad, but how bad? How often has it happened before? How likely is it to happen again? How bad would it be if it does?