REALbasic Beta Program Survey Results

The Association of REALbasic Professionals did a survey late last year about the REAL Software beta program.  You can see their writeup here.

There was nothing earth-shattering in the results.   With a self-selecting group (via the survey name) they found that roughly half of all the respondents participated in the beta program in some form or fashion.    Of those that didn’t participate, most ARBP members didn’t even know it existed.  I’m shocked that people didn’t know about it.  I guess that just shows how long I’ve been part of the RB collective.  🙂

Unsurprisingly, the Non-ARBP folks that were not part of the group, when asked why they weren’t part of the beta program, responded most often with some form of “I’m a newbie.”  It would be nice to get more of these folks into the beta program but given their timidity with RB to begin with I’m not sure they’re confident enough to know what a bug is and what it isn’t.

Just for the record, the way to sign up for the beta program is simple.  Log into your account on the REALbasic website and select “Beta Program” from the menu options.  You have to have a current, valid license for any version of REALbasic.  That’s it!  You then start receiving the beta list emails and get access to any current alpha, beta, or final candidate downloads.

A lot of the survey respondents said they participated in the beta program to test their product before it was released as well as making REALbasic better.  This makes sense as most of the participants are using REALbasic professionally and don’t want to be surprised with a new version and know (mostly) what the current bugs are.

The survey asked a question about having some form of compensation for beta testers.  Among the ARBP members it was 50% yes and 49% no.  Non-ARBP members said no 66% of the time.

I’m the one that suggested this question because I’ve probably logged around 50 RB bugs in the past couple of months.  Some of this because of my high level of interest in the new reporting system and some of it from producing many hours of REALbasic training videos.  I sometimes feel like I’m being asked to do my own job and then RS’ and the only reward I get is a better product (not that that’s a bad thing).  From the mixed results, the professionals have similar issues.

Finally we asked the question, “Do you feel the beta program is improving REALbasic?”  Both groups were overwhelmingly Yes with 70% of ARBP members with 90% of the non-ARBP members.  I think the interesting part lies in the 30% that didn’t say yes in the ARBP group.  What are they unhappy about?

From my own experience in the beta program, too many things are being changed when the product goes to Final Candidate status.  Final Candidate should mean that they think there are no more changes to be made and therefore this build is going to be THE release unless a show-stopper bug is found.  Unfortunately, we saw the results of last minute changes with RB 2009 R5 when a change late in the cycle caused reporting and the database to be broken.  The result was RB 2009 5.1.

So what are your thoughts about the beta program?  Do you like it?  Is it flawed, or is it just the nature of a product that’s always changing?

2 thoughts on “REALbasic Beta Program Survey Results

  1. I signed up for it, then promptly signed off of it when I was getting flooded with emails and I had no context for what the conversations were about. A beta discussion forum of some kind might be more helpful than an old style mailing list.

    Secondly with a release schedule as often as realbasic’s I question whether the beta cycle has much bearing aside from being free software testing for them… I highly doubt any beta feedback outside of bugs is seriously considered and wedged into the project timeline before release.

  2. @Phillip Zedalis
    Yeah, I prefer the forum idea myself but some of the old-timers would have a fit. If you remember they tried to do away with the NUG mailing list and some of the old-timers threatened to leave RB altogether.

    I switched to digest mode soon after getting into the program and even then I can still get 5 or 6 digest emails per day. It *can* be overwhelming.

    I disagree with your assessment, though, of the beta program. They do ask long term questions not directly related to the next release. For example, they asked pointed questions about reporting and the EditField split a couple of releases out so they could start preparing for them.

Comments are closed.