Bugrate

You know the feeling? You join the dev-channel, grab yourself a ticket in trac and start to reproduce. And while reproduceable you analyze what’s the root of the bug.  But while digging through core-code you stumble over multiple areas you think which must be touched to fix the issue. And then you discover unfixed related bugs and so on and so forth.

Finally you end up with the assumption that you just opened a can of worms and you’re unable to move any further. But you had the best intentions to just fix a bug and help with the project. What went wrong here? As with any program that is having a legacy code base (as WordPress does) there can be multiple things that are just going wrong. Over the last week I concentrated on one facet of that big bunch of problems the project is facing: the number of open bugs. I did this because I have the feeling that the immense number of bugs are eating the project’s developer resources up and prevent more and more people from actually providing patches. Since WordPress is an open source project this does just mean that it prevents you from using the software.

So I wanted to know how many bugs there are actually open and how that evolved over time. To get the number of currently open bugs is quite easy, you can just query the trac system (1316 open tickets while I’m writing this). But how did that evolve over time? How many bugs are “normal” in the project? To answer this, it is a need to gather the historical data as well. Since the trac software does not provide such a kind of data in  a custom query, I needed to create a script that gathered the data and did the analysis of it. Finally today I was able to finish it after having problems to get the exactly correct numbers:

From what I’ve learned so far is, that the overall developers in the project can not cope with the number of bugs the software has. As the graph is showing drastically over time there are more bugs than could be fixed and the number of bugs are constantly growing. Today we have 1316 open tickets, a new all-time high as my stats are showing.

This raises questions about the root(s) of this situation and how to deal with it in the future. Even if the graph is showing a concrete picture, it is not so  easy to figure out the right actions to be taken in the project. The first impression to just “fix all open bugs” as you can hear the one or other developer now more and more often just does not really help because this goal can not be achieved (we can not get all bugs fixed, that won’t work). Even I sympathize with the underlying idea, I must admit that I do not think it’s that easily made. I strongly believe that taking care of the bugs is only one side and it must be combined with other actions to help the project to become healthy again.

About these ads
This entry was posted in Hacking The Core and tagged , , , , , . Bookmark the permalink.

5 Responses to Bugrate

  1. Pingback: I’m inaugurating a new trac keyword, “… « WordPress Development Updates

  2. Interesting, I’m not sure looking at raw ticket count is necessarily a good metric because a lot of tickets don’t actually have enough information to determine if a bug exists and it is likely that a percentage of them have actually been fixed but not marked as such just because someone found the same issue again and it was not identified as a duplicate.

    The project would really benefit from people spending a little time going through these old tickets and closing down ones which are already fixed or do not have enough information to reproduce.

    I would be interested to know how you got the historic data for this?

  3. hakre says:

    Metrics have always their shortcomings, sure. I wanted to have the numbers at hand to better see whats going on. I once guessed that 10 bugs stay opened per week but now I can excatly say: the number of open tickets grow by 2.71 pieces per week. That is far better than guessing (and that a guessing can be far away from the concrete numbers should be no surprise at all).

    I can imagine that for some of the developers it is hard to see those numbers but I do not put them online here to blame someone. It’s a situation we all share together. I mean this is an open source project and it’s everybody’s responsibility to add something and there was a lot of growth over the years. If I can offer some metrics I’m happy to help.

    Other projects that got created in a compareable situation have similar problems. No need to not say that as well. I think it is interesting to find out more about these dynamics. How to manage that Brownfield extravaganza? Can be quite philosophical as well.

    And I think it can be good to have some numbers at hand because they show a global view from above. Here this shows, that the number of opened tickets grows over time. It does not mean that a certain bug gets not properly fixed or that there aren’t any dupes (hey, I know about the dupes in there; and the stats do not reflect branches precisely and the like), it just shows that in general that it’s likely that a larger amount of bugs won’t get fixed in wordpress.

    Again: I personally did not need a statistic to know that, but it helps to get a better picture. The graph is really impressive, I have never seen that in a project, but I have not compared to joomla for example. With one joomla developer I was openly talking about such problems last year and I think that those are exceptional cases we have in developing we should just play more openly around with to find out more.

    I mean substract a large percentage of 10%, 25% or even 50% for the cases you name, then the amount of open tickets still is enormous. I know that we can not get to an “all tickets closed” situation from now to the next release but normally I would say: Tend to zero bugs. Get the software reported-bugs-free. And that’s only a metric again, because as a bug squasher you need to fix/rate each bug individually. And some of those are hard to fix.

    About the database: The data has been provided by wordpress trac. It keeps track of opened and closed tickets over time. It could be interesting to get something similar for similar software projects.

    Additional I think it would be good to have this with some other time-scales. Like let’s say the last week, month, 3 months, one year and overall. So we can see how this performs in beta / release and after release (and it’s not only showing that immense growth but we can better see a more current performance). Let’s see. Stats can be only one fragment.

  4. Pingback: Logging WordPress Download Count

  5. Pingback: WordPress Bugrate Stats Reprise | hakre on wordpress

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s