REALbasic Beta List Conundrum

I’ve been part of the REALbasic Beta List for a few years.  I got involved  back in the early days of the new IDE because I was tired of finding bugs in new features, and in a few cases, new features that never made it out into the public release (though documented as such).  So I make a point to kick the tires of the new features and I generally do this by writing them up as a quick review article that I’ve been posting on the ARBP site.

I’m now testing the 2009 R4 release and I realize that I started feeling guilty about not testing more (a new feature in R4 is one I’ve been asking for a long time).  After talking it through with another developer I came to the conclusion that I’m donating my time to Real Software.  Yes, donating because, as a consultant, I get paid good money to do coding and testing for clients and since I’m not compensated from RS in any way, it is pretty much a donation.

I guess one could argue that I get compensated with a less-buggy, better new features, product.  That’s not very motivating to me, somehow, mainly because I’m paying really good money for that product for the privilege to do that!

Then I came to the conclusion that if I were to be compensated in some way I’d do more testing because it’s worth more to me.  Now I’m not suggesting a free copy of REALbasic because that wouldn’t be economically feasible.  But it does strike me as somewhat unfair that I’m spending time beta testing a commercial product.

I’m not sure what would make it seem more fair.  A discount on upgrades?  A higher support priority?  A t-shirt or coffee mug?  Public recognition?  I dunno.  Certainly the problem with any sort of compensation is how to enforce it and qualify it.  If I log one bug is that enough?  What if that bug is really a feature request?  If I report a hard crash is that more important than a misspelled word?  See what I mean?  It’s a slippery slope at the very least.

Do you think the beta program works?  How do open source projects recognize their big contributors?  Do participants need some compensation?  If so, how do you compensate the one-time reporter versus the consistent reporters?  What are your thoughts?

6 thoughts on “REALbasic Beta List Conundrum

  1. I’m with you. I was once in the beta program as well. I left the program when the emails coming in were more argumentative, and held quite a few help requests from others on unrelated topics, etc. I was of your thinking. I get paid to do things like this normally…and if RS REALLY listened on everything, I’d have no problem with it. However, I found that a large portion of the things reported were either never addressed, or marked as fixed only to crop back up later on.

    REALbasic is an awesome environment, but I found that the beta program was taking up far too much of my time to be worthwhile. It’s a delicate balance, and only the individual can decide if it makes sense for them.

  2. As a long-time user of REALbasic, I’ve never been comfortable with the way the beta testing program works.

    On the one hand, only real world testing will tell whether features are working. That’s because 1) real world projects make assumptions that in-house projects may not make, and 2) real world users bang on features differently from in-house employees. So I see how there’s benefit to the product (and by extension, the community as a whole) to be an active beta tester.

    However, the flip side is that it’s a crutch. I see a lot of things go out to the betas list that look and feel like “throwing it against the wall to see what sticks.” It appears to be the only form of QA that REAL uses, to be entirely honest. So it’s a double-edged sword to me. Without the beta program, I think REALbasic would be a much worse (at least more buggy) product. But with it, REAL basically can continue to use its crutch and never “grow up” as a software company. So in the long run, I think the beta program hurts everyone — even if it gives a short-term benefit.

  3. Without getting into specifics, this last beta has been trickier to test than others.

    In general, I feel I get plenty of value from the beta program even though I’m not formally compensated. I like seeing what the new features and fixes are early, so I can plan for them. I like new things, so I find it fun to try out a new beta of RB. It normally doesn’t take me all that long to test out copies of my projects on beta versions of RB.

    Apple’s Mac OS X and iPhone beta testing process is much the same (except they charge you extra $$ for the priveledge of even having access to their betas). As is Microsoft’s stuff that you get in MSDN.

    I think beta testing will remain a pro-bono sort of thing. If it’s in my best interest, I’ll do it. I beta test REALbasic, but I don’t beta-test Visual Studio or Mac OS X. Strangely, I do beta-test Windows. I’ll beta test any web browser I can get my hands on!

  4. A mug would be cool. And not one of those plastic thermos ones with the adhesive sticker that says “not dishwasher safe.” That basically means it’s never going to get washed.

    A couple of benefits you failed to mention is that there have been times when beta feedback has steered RS such as in adding the SQLserver “unlimited” to the Studio version. I count that as a win although I still haven’t cracked it open and started using it yet.

    Another is that you do have a month or so headstart on how to use the new features. And get to benefit from discussions that occur when people first start using them.

  5. Joseph :

    A couple of benefits you failed to mention is that there have been times when beta feedback has steered RS such as in adding the SQLserver “unlimited” to the Studio version. I count that as a win although I still haven’t cracked it open and started using it yet.

    Another is that you do have a month or so headstart on how to use the new features. And get to benefit from discussions that occur when people first start using them.

    I’m not sure that I consider those benefits to the beta program. It’s not like you can use the beta versions on your shipping software (or at least you shouldn’t) which also means that you have to isolate your projects between the current release and the betas. How many times have beta features been pulled from the final release?

    IMO the beta feedback doesn’t change their minds THAT much. Bundling REAL Server with Studio didn’t cost them anything extra because REAL Server was already a product and it was added only after HUGE negative reaction (which the alpha list said would happen and their feedback was ignored).

    The 90 day release cycle isn’t that long. I don’t see how seeing new features/bug fixes early really helps a developer out other than having the satisfaction of ‘knowing things that others don’t’.

  6. I recently completed a beta program for a major AV and Windows 7. The program had a nice Excel task-list of stuff to test and then I rsn the software for two weeks after completing the list of things to test. I was amazed of the reward that was three licenses of the released Pro version for a year!
    IMO RB:s beta is different. Normally they throw the beta at you highlighting two or three major changes and during the cause of the beta other issues crops up that needs to be fixed before release. RB’s betas are much more of work-in-progress than other beta programs I am involved with and takes much more of my time than any other beta programs.
    A small kickback could be in place (5% to the renewal for next year wouldn’t hurt bottomline I think) but hey – you volonteer to participate…
    My main reason for being part of RB beta is to make sure my main project behaves in the next release and to see the discussions over new features. RB is really good at listening during the betas, some things have been already decided and won’t be changed of course but at least the discussions are there!

Comments are closed.