Gaming the metrics is GOOD!

I’m going to start off by saying that I know Scrum and metrics don’t necessarily get along. But I will also acknowledge that it’s a necessary evil in most cases. And in a lot of cases it doesn’t have to be evil. Metrics are simply: a method of measuring something. In Scrum, we measure a lot of things. We measure the size of our work items, we measure the effort or time it takes to complete them. We measure our accuracy. All of this is in our quest to become predictable as a team and to improve (and we need to start with a baseline measurement to know if we’ve improved). But when others start taking notice of our metrics, that’s when things get tricky.

Should metrics (e.g. burn down charts, velocity) be used to measure team performance? Well it depends what you mean…

If you mean should teams be compared against each other to see who has the higher velocity and more perfect burndown? HELL NO!

If you mean should people outside the team look at the team’s metrics – well in the quest for transparency, yes. And it’s hard to keep curious people from looking at things anyway.

If you mean should a team be punished for poor burndowns and an inconsistent velocity? No, not punished, but the symptoms associated with those results should be addressed.

If you mean should a team measure their own performance and trends using these metrics? YES!

In most organizations, I don’t think we will get away from management wanting to know when things will be completed and with how much certainty. And sometimes these metrics are used for that, and I think that’s ok. Even if it goes so far that management is looking at burndowns and velocity and knowing what to look for, it’s all in the name of transparency as long as it’s not being abused. Some say this can lead to bad team behavior; gaming the burndown, gaming the velocity so it looks better outsiders. And in some cases, I’m going to say, yes it does happen and that is ok; good even.

I’ve heard some great comments once teams knew their burndowns were being noticed. “Let’s split that story so we can close it faster and our burndown looks better.” “I think we should save that for next sprint because we may not get it done and it will carry over on our burndown. Let’s commit to a smaller story instead.” “Let’s timebox the spike so we don’t spend the whole sprint working on it.” “Can QA test this as we’re developing? Can we pair to get it done faster?” In this case, the metrics which should be helping us to identify problem areas, are driving the team to improve on the problems. In fact, they seem to listen to the burndown more than they listen to the ScrumMaster insight (mine) on these things. As long as the behavior is in the correct direction, I don’t see a problem with gaming some metrics if it reinforces the correct behavior.

As always, this needs to be monitored to ensure things don’t go awry – if it turns into “Let’s remove these two stories from the sprint because we aren’t going to finish them in time and make our sprint commitment” – that’s a problem.

This was not an overnight transformation. I started by showing my teams their burndowns, looking at velocity trends, and detailing how to interpret them. I told them what an outsider would see and what questions they would have not knowing anything about the team or what they are working on. I explained how they could indicate where some problems might be (e.g. not much downward progress until the end of the sprint – stories are too big, too much work in progress, mini waterfall). Most importantly, after doing this for a few sprints, I had the team explain what they saw when I brought up their burndowns and velocity charts. By having the team interpret what they were seeing, equating it to the work they knew was going on, and thinking about what an outsider would see, it helped to reinforce the behaviors that were needed for change.

The other important thing I instilled was the importance of comparison between sprints. I made sure to quell any concerns that teams were being compared against other teams (still a no-no). But when we looked at a series of the team’s charts over a release, they could see trends and improvements (and also some tough sprints that we learned from). So when I hear things like “Let’s do x instead to help our burndown.” with a sideways challenging glance in my ScrumMaster direction, I give the thumbs up! Game the metric away – the correct behaviors have followed!