Getting to Zero Defects

It has long been accepted that defects are an unavoidable consequence of developing software. Large software projects tend to keep track of these in a defect management tool with the goal being to keep defects tracked there to a reasonable level (sometimes reasonable is defined as loosely as hundreds of defects). But, is an accumulation of defects really unavoidable? And until a team determines how to avoid them, how do defects fit into an agile process?

One of the goals of an agile process is to stay releasable. If after the end of each iteration, you could be released (being releasable doesn't mean that you release every iteration, just that you could), then you've made it easy to change course based on what you've learned. If on the other hand, you don't stay releasable, then you've got a list of things that must be done, thus reducing your flexibility.

Defects can prevent you from releasing. One way to start is to incorporate your stance on defects into your definition of done (criteria for completion that apply to all stories that you take on). Initially this might be that the story is only done if any critical or severe defects that it introduced are addressed. Over time, you can evolve your definition of done. Eventually, it might be that is only done if all defects introduced are addressed.

If your regression testing is not yet done within the iteration, then you have the possibility that stories have introduced regression in non-obvious ways (the potentially obvious ways being tested in the iteration). For these, you can allocate some time in each iteration (i.e., lower your velocity to give time to take these on). If you have an existing set of defects, you can also use this time to work on reducing those. Customers are another source of defects that must be factored in.

One question that often comes up is whether to keep the defects in the backlog as stories or to keep them in a separate defect tracking system. I tend towards capturing those that aren't resolved by the end of the iteration in a defect tracking system. And then relating them to stories for tracking within the iteration that they're to be addressed.

I've seen the above approach yield tremendous results in getting defects down from traditional levels. Lean gives some guidance to take it even further.

First, if you find defects, figure out how they occurred and correct the process so that those kinds of defects won't happen again. In other words, if a defect occurs, the system is broken and must be fixed. The 5 why's (asking why until you get to the root of the problem) can help here. Why did the defect occur? Well, why did that happen? Etc.

Second, if you're never going to fix a defect, why track it? Decide if you're going to address a defect. If not, remove it from the list. Tracking these lingering issues has a large cost. First, they can obscure more serious issues. Second, they can obscure the real state of the product since many of these might no longer be issues or even relevant. You might still record these somewhere so that you can later refer to them when investigating new issues, but in this case they are there for research purposes, not to track.

Defects are a form of technical debt. By eliminating them, your maintainability will improve and your customers will be happier.