It’s Not About the Release Schedule

There’s been a lot of chatter about my recent blog posts.  I am not really opposed to the 90 day release cycle – that’s not the problem (and as such the feedback request is poorly titled).  The real problem is the quality of the releases that happen every 90 days.

Each release cycle brings many bug fixes and that’s great, but it also introduces new bugs and changes in functionality.  This happens for new features and for other changes (including refactoring that’s going on for Cocoa).

I realize that there will always be bugs.  It’s next to impossible to ship a product that’s perfect – especially something as big as REAL Studio (that spans three and soon to be, hopefully, four platforms).  But, reporting easy-to-find bugs after the release gives the product an unnecessary black eye.

Part of the problem, I feel, is the beta program.  The mantra has been to test your projects with the beta build and ‘test everything’.  Since my time is valuable, that’s akin to saying “test nothing” because, like REAL Studio itself, my projects are rather large.  It either compiles or it doesn’t and that’s not testing the product, that’s smoke testing (electrical components work on smoke and if you let the smoke out of the component it fails – sorry, electrical engineering joke there).

No one tells the beta program to test “x” feature – we just get a list of bug fixes.  I know I would like something specific to test to get the most out of my time.  Harness the people that have already volunteered their time and effort!

The Feedback system has a lot of bugs and feature requests.  Many of them are very old (they predate this incarnation of feedback system).  Some of them should be closed and some need to be addressed.  Should feedback items older than 6 months get an automatic system vote and then get another vote every 6 months thereafter?  That fact that there are a lot of open bugs is worrisome.  Should bugs and feature requests get the same level of weighting?  Should they have separate areas so that priorities for feature requests are different than those for bugs?

As someone who has spent considerable amount of time and effort trying to reduce Windows flickering I feel I can say that Windows builds right now have some “issues”.  You can say anything you want about the individual controls, I think the issue is with the Window itself.  Put a background image into a window and you might as well put sunglasses on to avoid a headache.  I bring this up because you don’t see this issue in Mac OS X (because the OS double-buffers the window).

Does REAL Software need to spend more time using Studio in Windows?  I dunno, but Windows is not like Mac OS X.  There is quite a bit of inconsistency between platforms.  Some of that is the OS and some of it is REALbasic.  I’m not enough of a Windows expert to offer any hints, but I do know that if I run into problems it will inevitably be on the Windows side (see blog posts on memory leak issues).

I know some people are pretty upset with the new LLVM compiler for RBScript.  It doesn’t work for many causing them to stay with an old version of Studio.  The problem is that the beta program said it was broken and it was released anyway.  Again, I realize that you can’t fix 100% of the bugs before release but when the major users of RBScript are saying it doesn’t work – perhaps it should have been delayed.  Maybe there should have been an option to use the old compiler.  I have no idea how much work that would have been but it would have resulted in more testing and a happier bunch of RBScript users.

So rather than focusing on the 90 day release cycle, let’s focus on the root causes of our unhappiness.

These are just a few of my thoughts.  What do you think?  Did I miss anything major?

9 thoughts on “It’s Not About the Release Schedule

  1. I disagree — I think the release cycle is definitely part of the problem. No dev tool updates at such a rapid pace, and while some may think that’s a great selling feature, most professional developers understand why this is a major drawback. Every time a new release comes out, you have to validate *all* of your existing code bases that you wish to keep up to date. That is a time consuming and expensive process. Failing to do it means you actually have no idea if it’s safe to stay current for any given project.

    There’s a reason many large ISVs still use Visual Studio 2008 instead of the newer Visual Studio 2010. It has nothing to do with the quality of the software, and everything to do with the expense of switching. They just finally moved off of VS 2005 by the time 2010 is released!

  2. Thanks, Aaron. Your argument, I think, is pretty close to mine in that the short cycle undermines quality. With a cycle that short, major changes really shouldn’t occur *every* release because there’s no way you can properly test it and ensure it all still works.

  3. I’ve never understood the fixation on the 90 day cycle??

    It seems to me that by suggesting a longer release cycle everyone’s presuming that RS would put more effort into making sure that each Feature/bug fix was tested better.

    That doesn’t make any sense! Seems to me that would be a very unlikely presumption.

    Given RS’s apparent focus on “new stuff” and “quantity instead of quality” I could only presume that we would just end up with 2 ‘old’ releases worth of new features and bug fixes, every 180 days.
    Still with the same lack of quality, and since new bugs would only be reported in 180 day cycles we’d only get new bugs fixes released every 180 days.
    None of that’s a better situation.

    All the chatter about the release cycle just detracts from the real problem.

    @Bob
    “The real problem is the quality of the releases”
    That’s the REAL problem, everything else derives from this.

    and it seems that Geoff Perlman agrees;
    “A 1 year release cycle would not produce a more stable product.”

    @Bob
    “The Feedback system has a lot of bugs and feature requests. Many of them are very old (they predate this incarnation of feedback system).”
    Aren’t the only ones that ‘predate’ all the ones from FogBugz?
    When they originally moved to FogBugz, 2ish? yrs ago, they started clean didn’t they?
    So every time it gets ‘too full’ just delete it all and start again?
    That would actually be a good idea, IF they changed the focus from ‘new features’ to ‘kill the bugs’.

    @Bob
    “Should bugs and feature requests get the same level of weighting? ”
    Absolutely NOT, just in case that wasn’t clear.. NO NO NO.. 😉

    @Bob
    “Should they have separate areas so that priorities for feature requests are different than those for bugs?”
    yes yes yes.

    Some new features RS would just ‘have to have’, that’s fine, but ‘generally’…
    – New Features should be driven by the ‘voting’.
    – RS should ‘float’ the idea of a ‘new feature’ before implementing it.

    Bugs generally shouldn’t be around long enough to get voted on,
    but the ability for RS to say “Hey, bug x looks like being a major can you give us an idea of what you think is more important” would be good.

    RS needs to concentrate on quality, quality and more quality…

    PS:
    Just luv some of the comments in your “Stop the 90 day release” request.

    -GP – “and I have no plans to change that even if this case somehow managed to become the number 1 case.”
    Wasn’t being the No1 request used in defense of why iOS is important to RS?
    When your customers scream loudly what they want you take no notice??

    -GP – “When you create cases, we need you to tell us the problem you are facing, not just suggest a solution.”
    Cause all the intelligent lifeforms are already at RS?

    -Chris Musty – “BTW “Thats life” does not cut it with my clients.”
    Mine neither.

    -FABRICE GARCIA – “Success is a bad teacher. It drives intelligent people to believe they are infallible.”
    Probably the best reQuote… if GP thinks about it all.

  4. I’ve kept my mouth shut, but have found these posts very interesting to watch. It is hard to believe that it has been only 8 months since my blog post “REAL Software, the opposite of love is not hate, it is indifference” at http://pariahware.com/blog/?p=248

    Have I added Bob’s bug report to my list? Nope. Have I used any of the betas recently? Nope. What have I done? First I set an e-mail filter to filter out all Web Edition content from my in box, then I unsubscribed from the NUG, and finally, I let both my MBS and Einhugur licenses lapse for as long as I could, only renewing updates about a week ago. It appears that i’ve reached the indifference point.

    REAL Software is Geoff’s company and he can do with it what he wants. For all the complaining REAL Studio users are doing, if they are still giving the company money, nothing will really change. If developers get mad enough, they will leave and REAL Software will either fail, or more beginning programmers will find the language, pay the money, and keep REAL Software in business.

    I, for one, hope that Cocoa is done by March 5, 2011 when my “subscription” runs out because at the moment, I have no intention of buying another license. In reality, I may be forced to update once more (as I was with the recent round of plugins), but honestly, it will be a hard sell.

    I commend Bob’s passion for wanting his development tool to be better. I used to be there with him, fighting the good fight. I simply no longer believe it will happen. Unless, perhaps, RS’ revenue begins to fall.

  5. Slightly on topic – Living in the southern hemisphere I have no idea when my northern counterparts fall starts but it is well an truly spring here so I would assume you guys are just the opposite and indeed are in fall. If this is the case, is the release of web edition being delayed to ensure its stable? What seems to be the delay given that there are quite a few videos floating around the web?

  6. The time period of the releases is not the problem.

    The problem is that the releases are crappy quality which makes us dependent on future releases which are also crappy quality continually causing us delays and headaches. Eventually expiring our subscriptions causing us to pay more money and continue to be held hostage by this company.

    If each release was high quality with very few bugs the developer wouldn’t need to flip flop between versions of the IDE while developing his project because there wouldn’t be a need to flip flop which is the way it is supposed to be. That negates the 90 day cycle dependency issue.

    New features should be put on hold until almost all of the bugs are fixed and every bug that any customer files a report for is fixed.

    No exceptions.

    Why?

    Because every person that purchased this product has the right to receive a working version of it. Anything short of that is a rip off.

    Not one damn new feature should be worked on while there’s broken features in versions customers paid for.

    Not a single customer should of been sold a version that doesn’t fully work and certainly never should a customer’s subscription ‘run out’ sticking him with a buggy version.

    The only reason we’re used to receiving so many bugs in each version is because REAL Software has always distributed buggy versions. This is the choice of REAL Software and it never should of been this way and customers should not put up with this.

    When you buy a product it better work or the company better fix it for the included price which the customer already paid.

    If the company doesn’t fix their broken product and then tries to charge you more money to fix what you’ve already paid for where I’m from that’s called FRAUD or a rip off.

    We’re tired of excuses and we know the truth.

    REAL Software has plenty of resources to fix REAL Studio and provide us with working versions.

    Stop wasting developer time on making new features until you fix what you’ve sold us.

    Someone else mentioned no longer purchasing REAL Studio or any other REAL Software product until the quality has noticeably improved.

    I’m starting to agree with that.

  7. A new Feedback Item was added today that I think is worthy of my limited priority votes. feedback://showreport?report_id=14287 is a request to separate Feature Requests from Bugs. Add it to your priority list if you like it.

  8. @Aaron Ballman
    “Every time a new release comes out, you have to validate *all* of your existing code bases that you wish to keep up to date. That is a time consuming and expensive process.”

    Exactly! I got tired of testing new RB releases to find out that there were always problems that prevented me from moving to a new release. That’s why I stopped testing new releases 3 years ago and was just checking the NUG/Forum/Feedback for bugs on new releases that would affect my projects and the release notes to see if any of those bugs were fixed.

    @AnonyMr
    “Someone else mentioned no longer purchasing REAL Studio or any other REAL Software product until the quality has noticeably improved.”

    It worked for me for 4 years and only renewed for 2010r1.

    I’m *slowly* moving my old projects to 2010r1 – it consumes a lot of time testing everything to make sure there’s no bug that might affect a specific project. I’m very careful moving my old RB projects to new RB versions because I was bitten more than once and I don’t trust any new release of RB since 2006.

    And I didn’t waste (and will not waste) any time testing new releases and will not renew my subscription unless I notice that something has improved a lot in terms of quality… but if this continues like the latest 3 releases I might have to wait a few more years or just move to a different dev tool. I love RB… but I do love more my biz and customers.

Comments are closed.