Bugzilla

From APIDesign

Current revision

Bugzilla is a bug tracking system backed by SQL database. Many open source projects use Bugzilla. NetBeans uses it too.

Throw Away Your Bugtracking System

I've noticed very nice and inspiring article few months ago. It somehow captures what I always felt deep inside of me: Leaving tons of bugs open is useless.

In March 2010 I started to practise this new bug fixing life style by announcing that I will close every issue I find that does not match the project owner expectations. I started with 300 bugs in modules of my responsibility back then. Now, half a year ago I have 15 open issues. I did what I could with issues laying around for ages. Some of them came back to me, so I needed to try once again, but overall I think the article was right: It does not make any sense to keep issues open.

Either fix them, if they are reproducible (don't forget to write a test to prevent your application amoeba from shaking). Or enhance logging and close as worksforme and wait. Or give up and honestly say that you don't care (e.g. close as won'tfix). One question however remains: who's the project owner on an open source project like NetBeans?

Open source Project Owners

The NetBeans bugzilla is full of issues that nobody addresses. The team is always unsure what to do with them. On one side, there is a desire is to resolve them - our release dashboard looks bad with few thousands of P3 or P4 bugs (which are usually enhancements anyway). On the other side, we are tempted to believe that one day someone will take over the bug and thus we shall leave it open.

As far as my experience says, nobody ever took a bug just because it was open. Those issues just annoy all of us for ages. From time to time, there is a business need (e.g. someone is willing to pay) to solve some of them, but that happens with issues resolved as won'tfix as well. It is easy to reopen them in such situation. There does not seem to be a value in keeping issues nobody plans to work on open.

The throw away your bug tracking system article is really inspiring. I understand NetBeans is not real agile project, but never the less, we apply some agile principles. Does an enhancement violate expectations of product owner? Who can decide that?

First of all, we need to seek product owner. Product owner is the one who can make decisions about the product. Certainly my employer can tell me what to do. If my company thought some behaviour violates expectations, I'd had to fix it. This is clear. But who can be a product owner in a open source project like NetBeans?

In an open community, probably everyone can ask to be seen as a product owner. However, as product owner has to be able to make decisions, it is proper to ask what kind of decisions a community member can make? Well, a community member can decide and choose level of own participation in the project. In particular, every open source contributor can decide how much help to donate to eliminate the perceived violation of the behaviour.

In order to keep open source open, it is important to respect everyone's right to eliminate such violations. However one has to act, not just talk. In open source, it is the code that counts. If one is not able to contribute that, then we need to question whether such person can play role of a product owner.

User Point of View

Of course, users of open source products may feel differently. By rejecting to track enhancements and honestly closing them (as there is no project owner that would be willing to sponsor the work), one risks losing the community. The previous section mentioned that code is the only thing that counts. That was exaggeration. Community counts as well. Thus the Throw away your Bugzilla position balances on the edge of making community more proactive, and losing it.

Certainly, with IDEs, users are mostly software developers themselves, and therefore are more likely to have the skills to contribute to the development. As opposed to, say, KDE users. However this does not make much difference. We even tried to exploit the development power in our users. But so far (e.g. after ten years of NetBeans IDE existence) no luck. Time to give up. From this perspective a bug reported by a random KDE user is no different than a bug reported by random NetBeans IDE user.

On the other hand, what is different is the attitude, honesty. My experience with mega sized open source projects like KDE is that when I report a bug for Amarok, Kopete, Xine or other related projects, people don't say "fix it". Rather than that people say nothing. There is just a uniform: silence! That experience leads me to conclusion: most of the open source projects just ignore requests that don't fit into their plans. I'd say, this is wrong. I'd rather see a honest response: we don't care. Few basic hints (for example I don't know how to debug various libxine.so libraries) would help too. If there was a buddy to help me with initial steps, I could solve the problem myself and donate to the product. But nothing. Just a silence for months. Then they close the bug with worksforme or incomplete status.

The above however applies for bugs which are not real bugs, more feature requests. It is enough to report problems in a way that gets attention of project owner. For example once my bug X server does not work on Xvnc under vserver (which has been ignored for ages) have been renamed to Segmentation fault in X server color setup routines. Immediately my report got appropriate attention and was fixed almost instantly. Why? Because every developer could suddenly understand that crash of the application is something that requires project owner attention (without necessity to even understand who's the project owner).

Two Issue Trackers

The basic difference between throwing away bugzilla and keeping all user wishes in a database is in the purpose of the bug tracking system. The agile teams either don't need it at all, or need it only for short time issues. In case of open source teams that like to be agile, the bugzilla is perfect for communication with users, commenting, attaching, tracking status of obvious bugs (segmentation faults, NPEs, other exceptions) which clearly need to be fixed.

Some users however believe that their inquiries should be treated as bugs while they are more a feature requests. If reports of this kind are left open, then there can be tons of issues that represent user wishes, but are generally left ignored. Unless issues have sponsor, some project owner willing to accept them as an work item, they are useless. As argued in the throw away essay, such situation needs to be prevented.

From time to time there are users willing pay for their issues to be resolved, within their financial possibilities, if that would get them a higher quality product. They care about the issues and contribute to the project ecosystem by reporting the issues. They don't care whether there is a project owner to treat their issues right. In the end, software is made for its users. Users mostly don't write the specifications. If something violates common user expectations, or has some other usability flaw, it is a defect of the software no matter what. If it doesn't violate the specification, then it is a specification bug or omission.

However tracking such feature requests (which are not seen as bugs by any active Project Owner) will result in thousands of valid enhancements reports. That goes directly about the approach of throwing them away. What can be done about that? Maybe we need two bug trackers, one for bugs against the specification, and another one for specification bugs?

Bounty System

For open source project it makes sense to create a system where people willing to become Project Owners (or at least a feature owner) can meet those who want to do the work for them. This is slightly different bug tracking system than bugzilla. We tried to implement it few times under the name of NbBounty (but never finished it; we are not the right drivers; we get monthly wage by our employer).

The system is not complex. From one side the feature owners can throw in their coins in to motivate someone else to implement features they would like to see. Bugs still belong to bugzilla, but feature requests need to be supported by bucks. In current society money are common indicator of engagement. Open source projects need a way to find perils among the dust. By sorting the feature requests by amount of available donations, the developers can make a qualified judgement and choose those that are worth to implement (worth in terms of usefulness as well as available reward).

Maybe we don't need special bounty system. Let's just close issues without project owner as wontfix and add a white board status donation-required. Users can then add a comment describing the amount of $ they are willing to donate if the issue is implemented. This requires a bit of trust, but for beginning it might be enough. Of course we need to change the way we think about meaning of won'tfix. It does not predict the future, it does not mean "the issue does not and will not ever deserve to get fixed." For such bugs bugzilla is using invalid status. Issue marked "won't fix" means a valid problem, but one that nobody currently doing work on the software cares about.

Actually I have a little illustrate story showing potential success of such donation system. Niklas, who made me wrote this whole essay just replied:

me: Anyway get ready for your issue to be closed unless you promise to donate a buck.
niklas: I'm ready for it to be closed. :)
niklas: And given a donation system, there are more important issues I'd want to be addressed first.

This is exactly the feedback every open source project needs! Clearly there is a wish and clearly nobody wants to sponsor its development. What else this issue deserves than won'tfix status and donation-required remark?

External Links

• Sort of common way to deal with growing number of issues: Cascade of Attention-Deficit Teenagers - e.g. don't close one by one after proper evaluation, rather close all of them in a batch once per year. Needless to say I prefer the one by one evaluation and see it more honest.
• This essay was linked from Hacker News. I guess that helps (maybe together with the article title). I have not yet seen such amount of traffic on this website...

<comments/>