Agile Metrics

Metrics can be useful indicators of how things are going. They can help you identify areas to investigate / learn more. They can also be misused.

If you use a metric as an indicator, then you recognize that it is just part of the puzzle. It is not representative of the situation, just a hint. Nobody gets blamed if it is bad. Nobody gets rewarded if its good. It is something that you and the team can use to get a feel for things that are helping or hurting the team's progress. If you use a metric to evaluate people or teams, then several bad things can happen as people try to optimize it.

First, they can cheat. Most metrics can be manipulated in some way. For example, velocity. I can easily increase my velocity by increasing the size of my estimates. I haven't improved my productivity at all, but I've made it appear as if I have. At this point, not only is not valid for evaluation purposes, but we've also lost the ability to use it as an indicator.

Second, they can improve the metric at the expense of other things which might be equally or more important. It is a fool's errand to try to capture everything in a few metrics. If I focus on revenue, then this can be increased at the cost of expense. If I focus on margin, then this can be increased at the cost of innovation / market leadership (since expense is easier short term to affect than revenue, I can increase margin by cutting expenses at the expense of the future). And so on.

Given the above disclaimer, what metrics are useful for agile teams? In my experience, these tend to evolve as the team evolves. Initially, it might be things like % of days that the build passed. Then it might evolve to the percentage of stories accepted in the iteration. Then the number of defects open. Then maybe code coverage. Each helps you to focus on areas that might be causing the team trouble. Velocity is a good indicator throughout if not used for evaluation.

A lean metric of interest is the story cycle time. How long does it take from the time a customer requests a story to the time that they can use it? A value stream map is a way to trace this and identify where the time is being spent. It will tell you a lot about the organization as a whole (not just the engineering organization).

The point is to identify areas in which you are having issues, to come up with indicators to help you see how you're doing at resolving them, and then to move on to the next areas. You might keep the past indicators to ensure that you don't regress. But overall, they're just indicators.