The Quest for a Good Bug Tracker

Let’s start with the documentation aspect.  We generated 30 plus pages of assumptions, clarifications and questions from an 8 page document we received from the client.  The document we received is the crux of the project so it doesn’t surprise me that it generated that much.  In fact, once the client answers the questions it’ll most likely generate yet another set of questions.

It’s hard wrapping our minds around the agile process.  All of our documents are ‘living’ which means that from sprint to sprint they might (and most likely will) change.  Documenting the documentation changes is becoming important.  From my engineering days this is completely backwards.  Requirements documents were set in stone and were the bible and it took an act of God to change them.  So in the end, all of us are adapting.

Our agile tool, Rally, has some built-in defect tracking software.  We are now generating defects (i.e. bugs) in our prototypes and have to start tracking them.  Rally’s own set of defect tracking is severely limited in a number of ways.  It just doesn’t seem to follow the life cycle of a bug properly:  report-assign/feedback-fix-evalulate fix-close.  Their default states just don’t follow that and while I’m sure we could have added our own status’ I think our biggest limitation was the interface.

The interface just isn’t very rich and the Rally interface is just setup wrong for our needs.  They have the ability to add custom fields which is nice but their inline edit doesn’t always work the way you would like it to.  So you end up using the editor screen which is clunky:  it doesn’t expand to fit the available screen space and that means you end up scrolling to get to your required fields.  Slow and painful.

So I went looking for alternatives.

Our Subversion host automatically gives us Bugzilla and Rally has an integration with Bugzilla.  This was my first stop.  I’m not sure why Bugzilla is considered one of the best bug trackers around.  I (and everyone else on the team) found it very hard to use.  Maybe it’s because all of us are Mac users and we just found it confusing.  I was tasked with setting it up and found setup to be very cumbersome.  The people entering bugs didn’t like it either and then the integration piece didn’t work so well in that there’s really two components of any bug (the fix by the developer and the verification of the fix by QA) that Rally doesn’t create automatically.  So rather than beat our heads against the wall any longer I did more research.

TestTrack Pro
I’ve used this tracking system before.  Unfortunately we’re using hosted servers (i.e. hosted) and there doesn’t seem to way to get this to work for us.  Everything I’ve seen on their website says that you need a dedicated machine for the server.  I even tried finding a hosted TTP and couldn’t.  Since I didn’t have a lot of time to set it up I gave up on this solution.  Perhaps when our QA team comes on board we’ll revisit this solution since Rally has an integration tool for it but it’s also a bit pricey and before I commit to it again I’d want to use it for a while.

I’ve never had good luck with this tool so I didn’t bother with it.  It’s been very buggy in my experience.  Other developers I know have had good success so your milage may vary.

We’ve used Mantis for a long time.  The other team members have used Mantis on their other projects as well so in the long run this really wasn’t much of a contest.  When we made the decision to switch I had it installed and setup in under an hour.  The project setup seems to be especially easy to understand and use.  The Change Log is very handy for documenting changes in any particular build and we’ve found that the email process works well for us.

I must admit that the MyView page takes a while to get used to but once you ‘get it’ it makes life easy.  With one glance you can see issues assigned to you, unassigned issues, recently changed issues, issues you submitted and issues that you’re monitoring.

Maybe I like Mantis because its interface is compact and doesn’t require a lot of scrolling and clicking.  The interface just seems to be in the right spots for me.

I would have liked to taken a look at FogBugz (Real Software uses this) and others but time just didn’t permit.  Frankly, now that we’re up and running on Mantis it’ll have to be something compelling to get us to change.

I mentioned this in one of my RB Developer columns, but I think it’s worth repeating.  A good bug tracking system that is accessible by your clients, as well as your team, is very important.  It’s important not only from a development standpoint but from a sales standpoint.  When you’re making your pitch to the client you should mention that you have a bug tracking system and that they’ll have access to it and they’re expected to use it (rather than email and phone calls).  You’re helping the client see that you have a process for your software development.

What are your thoughts on bug tracking systems?