Managing Bugs in Scrum and Agile Projects

As I was revising and writing new content for The Scrum Field Guide, 2nd edition, I emailed my friend Bill Hanlon at Microsoft to get his feedback on my existing chapter on bugs. Bill and I worked together years ago and I often value Bill’s opinion more than I value my own. The guy constantly innovates on things that others would say are good enough. Things are never good enough for Bill, and he’s constantly experimenting and trying new things.

Bill and I began a fantastic dialog about bug management that didn’t conclude until after my book deadline had passed. I’d like to share with you Bill’s approach, how to get started using it, and how my own approach has changed as a result.

Start with a Good Codebase

Bill defines a good codebase as one in which it’s quick and easy to fix bugs. The metric he uses to measure the goodness of a codebase is Average Bug Age (ABA), sometimes called bug cycle time. Bill postulates that the ABA of open bugs is the key to determining if a codebase is of high quality and debt-free.

If a company has a good codebase, then the following will be true:

1. Since it’s quick and easy to fix bugs, most of the bugs have been fixed already. This yields a high-quality product.
2. If it’s quick and easy to fix bugs, then adding new features is easy, too. Therefore, team productivity is high.

Now that we have established what constitutes a good codebase, let’s explore the actual fixing of bugs.

Return Bugs to Their Creators

Bill and I both do real-time bug fixing. In The Scrum Field Guide, I advocate fixing high and mid-priority bugs in real-time, but putting low-priority bugs on the product backlog. Bill fixes every bug in real time, even the low-priority ones. We’ll come back to that whole putting-on-the-product-backlog later. First, I want to tell you about Bill’s novel approach to real-time bug fixes.

Bill’s teams have a rule in place, where the person who wrote the buggy code is required to fix it before that person is allowed to do anything else. The flow works like this (I’ve revised some of Bill’s terminology but the flow is the same):

1. Team member (the discoverer) finds a bug and finds who wrote that bit of code (the originator).
2. The discoverer gets up, goes to the originator (if they are not already working together) and says “I’ve found a bug in some code you wrote—let’s fix it together.”
3. The originator immediately stops working on whatever they were working on and the offender and the discoverer work together to fix the bug.

What this means is that if a bug is found by anyone on the team, they identify which team member wrote that code, go to that person with the bug in hand, and say “fix it with me,” resulting in the originator having to stop whatever they are doing to fix the bug.

Bill told me that while bug-prone team members tend to hate this rule, the benefits far outweigh any negative reactions. The biggest benefit for Bill is that the bug-prone members tend to spend a good deal of their time fixing old bugs instead of writing new code. As a result, the best coders end up writing the vast majority of the codebase. The rule has three important side effects:

1. It limits the damage a bug-prone team member can do, and
2. It rewards team members who write clean, effective code—usually the ones who have figured out how to do TDD and how and when to refactor—by freeing them up to them write even more code.
3. It provides the buggy team members with incentive to learn to write better code. And that results in overall faster teams, and a more stable, higher quality codebase.

Stop Tracking Bugs—Yes, All of Them

At this point in the conversation, I was jumping with excitement. It only got better when we started talking about bug tracking systems. I was sharing my frustration with Bill on how a customer of mine was arguing profusely for keeping every single bug logged in a tracking system. I said it was a waste of time and that I’d only track the bugs that were not fixed in real time—the low-priority bugs I’d added to the product backlog. Bill told me I was crazy to track even those bugs, and he had the data to show me why.

A few years ago, his teams stopped using a bug tracking system. Why? Because team members said, “It takes too long—it adds too much time to the bug-fixing process.” The teams said that having the discoverer write up the bug in a bug tracking system, and then having the originator read it and ask questions, slowed things down too much and seemed like doing a process for the sake of doing a process, not for building a quality codebase.

These days, Bill’s teams only use a bug database for externally found bugs. These bugs are categorized as follows:

1. Not a bug: the bug that was filed in the tracking system was not a bug at all and so was immediately closed as “won’t fix.” The vast majority of the time, the bug report was created by a confused filer.
2. A real bug: in this scenario, the bug-fixing policy I described earlier kicked in and the bug was fixed and closed immediately.
3. Feature request: it’s true that sometimes people get confused on what is a bug and what is a new feature request. If a bug report is found to be a new feature request, the request is transferred to the product backlog, turned into a story, and prioritized by the product owner. The bug is closed.

Fixing bugs in real time and abandoning a traditional bug tracking system made his teams so much faster than his peer teams that his manager took notice and started implementing it on all her teams.

Fix the Low-Priority Bugs in Real-Time or Dump Them

Putting bugs on the product backlog was next. Bill and I went back and forth on the concept of putting bugs on the product backlog. I stated my case over and over again on why I put lower priority bugs on the backlog, and he asked me to just give his method a try. Luckily, I found a customer of mine who was willing to experiment with me, and you know what? I agree. Don’t put bugs on the product backlog. Just fix them or mark them as won’t fix. Not only did the average bug age go down with this approach, but there was also higher quality code, delivered faster.

Using Bill’s approach has fixed one of the biggest issues I’ve had with teams over the years: the debate over the quality of new feature development. Having the ability to file a bug and have it on the product backlog means there is a way to deprioritize quality by moving bugs farther down the product backlog, and instead focusing on delivering stories. It just creates extra work for the team to triage, and keeps the debate of quality versus new stories/features alive. That’s a debate I’m tired of having. Bill’s approach eliminates that conversation altogether.

What about you? Are you still tracking bugs or trying to measure each bug’s worth versus a new feature? Do your team a favor and try Bill’s approach for a sprint or two. You’ll be amazed at the results.