REALbasic 2009 R5 A Disappointment

I originally intended to use this space to announce that my review of REALbasic 2009 R5 was posted on the ARBP site.  I wanted to say that this release is a good one that it adds some nice new features, enhances some old features and offers a plethora of bug fixes.  Instead, I’m going to go on a rant about the lack of quality in recent REALbasic releases.

Before I start my rant I have to preface this by saying I know and respect many of the REAL Software employees.  I am friends with several current and former employees and am on friendly terms with many of the employees as well.  This is not personal.  I’m a supporter of REALbasic when I can but given the debacle of the R4 and R5 releases I have to call them out on the carpet for some well deserved tongue lashing.

Quite frankly, their quality control sucks.  Release 4 had bugs in their new Build Automation and Reporting tools which made it unusable for me (but okay for most users).  Release 5, which had a shortened schedule, has an ugly SQLite bug that has the potential to silently create an infinite number of bugs.

This new bug was caused by a change to the column type returned by the recordset in SQLite databases.  I’m not sure of the details but it has something to do with the index number.  It’s supposed to be zero based but was one based and now it’s reversed (or visa verse – like I said I’m not sure).  I don’t know if if was the documentation what was wrong or the implementation in the SQLite plugin but regardless, the change was not done properly because it potentially affects all existing SQLite users.  Ideally a new API should have been introduced and the old one deprecated.

You can see this manifested in the Reporting tool in R5.  If you look at my example project of REAL Reports from the ARBP source code repository you’ll see that a varchar column is interpreted as an integer.  (Note:  To use in R5 you’ll have to update the namespaces in the project).  The sName field returns what looks like a number rather than the text data that is in the database.

This is not good.  It indicates that quality control is poor at REAL Software.

I’m part of the beta program.  I tested the reporting tools and submitted a bug on November 18th about an unrelated bug having to do with Group Footers.  This bug did not manifest itself then (I’m not sure how far in the beta process it was but it’s late considering that it was two weeks before release).  So this means that major functionality changed late in the beta program.  This is doubly worse is that last week was a major holiday in the United States and most people take additional days off.  They made a major change with fewer beta testers than normal in the pipeline.

To say that R5 was released too early is an understatement.  It’s been a year since REAL Software released their build engineer and senior engineer and up until R4 I was happy with the quality and stability of the builds.  After R4 and R5, however, I now have to seriously test the official release to determine if it doesn’t have a major bug that affects me or my clients projects even though I tested it during the beta program.

To say that I’m disappointed is an understatement.  I feel that my time in the beta program has been a waste.  Why am I spending valuable time beta testing if RS isn’t going to give us the time and listen to us?  If the testers say don’t release because of ‘x’ problem shouldn’t that be a clue that something is wrong?

Who’s in charge of development?  Aaron (senior engineer) and Nathan (build engineer) used to manage the releases but now I don’t know who has the final say.  The development of REALbasic is too important to leave to the schedule of sales and marketing.  Someone has to draw the stick in the sand and say, “NO!” when it’s not ready.  It’s not JUST REALbasic that is affected – it’s everyone that uses RB that gets hurt by RB bugs.

If RS isn’t going to properly test their own product what does that say about their process?  I’d rather there be fewer new features and bug fixes and more time spent on testing so that each release is rock solid.  It sure beats what we’ve had these past two releases.  Let’s hope that we don’t have to wait 90 days to get a fix for Release 5.

Thoughts?

10 thoughts on “REALbasic 2009 R5 A Disappointment

  1. Pingback: BKeeney Briefs » REALbasic 2009 R5 A Disappointment beta version

  2. I agree with you Bob that there have been some bugs in the new Report Editor and Build Automation features in R4. We have addressed most of them in R5. Regrettably, the REALSQLDatabase class which was rewritten from scratch for R5 was not tested well enough by us internally or by beta users so some bugs were missed. We are working on resolving this right now and will be shipping a 5.1 release around the middle of this month.

    However, none of these bugs should have gotten past the release. As a result, we are working on plugging the holes in the product quality process to reduce the likelihood of this happening in the future.

  3. I just saw on Twitter that REAL will be releasing R5.1 in mid-December. It sounds like RS is guilty of not performing regression tests.

    If this problem only manifests with the IdxField function, then I most likely won’t be affected since I usually use the Field function. That, and, I generally stick to PostgreSQL.

  4. From what I understand, the problem is with the ColumnType property of the RecordSet. It used to be 0-based and now it is 1-based.

    Obviously the reporting stuff uses ColumnType, but if you don’t use the reporting feature or the ColumnType property then you’ll probably be OK using SQLite in R5.

    Anyway, good to hear they’ll be doing a 5.1 to address this.

  5. Quality assurance, unit testing and regression testing, not to mention other software testing techniques, are critical for releasing bug-free software. In other programming languages like Java, C# and Ruby, there are IDE plugins available to integrate these tests seamless into code (and sometimes either write the code or flag code that is not sufficiently tested). Because IDE plugins, compiler syntax tree information (or at least a published grammar), and symbol table information are not available, testing is extremely limited and easier to overlook in REALbasic.

    I’ve been harping on this point for years, like a broken record. Maybe I’m getting bit annoying about it, but I do it because I truly want to see REALbasic succeed. The alternative for many developers is to be forced to program in _both_ XCode/Cocoa and Visual Studio (and a third tool and language for Linux) — vastly different tools and programming models — for every project. Twice (or three times) the programming! For REALbasic to become a serious enterprise tool, it MUST allow enterprise-level testing tools to be integrated by them or third-party developers into the IDE. I spent months trying to do this myself, but had to give up due to the above-mentioned limitations. I’d still be willing to to do it if it was possible.

    Yes, this would take a little more work on RS’s part — approving IDE plugins to ensure that they did not have side-effects that could cripple the IDE, reworking the compiler front end to export and import syntax tree and symbol table information, and providing an API to developers to safely read/write or rearrange this information — but other companies have done this and built a very large an successful third-party programming tool market that enhances the power of the IDE, resulting in even more sales for them. Geoff, if you’re reading this, please think about it.

  6. Was the beta group notified that the REALSQLDatabase class was being rewritten? Knowing that might have prompted people to be more vigilant.

  7. @Joseph
    Heck yes we knew it was rewritten. We found and reported critical and very serious bugs as far back as the first Alpha. Like I said in the original post, the bug did not manifest itself 2 weeks before release when I submitted a (unrelated) bug so I’m left to conclude that the change occurred within that two week period, one of which was the week of Thanksgiving.

Comments are closed.