Change (Not Really) To the RS Beta Program

Today Real Software announced to the Beta List that they were going to make some changes.  The notice, in part, says:

In the next few weeks, we will be updating the process that manages our mailing lists.  We will begin using a system that ensures everyone on this list has a current Real Studio license and that the email address subscribed to this list is one of the email addresses in your realsoftware.com account.

Of course, some on the existing beta list thought this was a horrible idea.  Doesn’t Real Software want beta testers?  Blah, blah.  It was just shy of outrage.

Well, here’s the deal:  Roughly half of the people on the beta list don’t have a current license for Real Studio.  This means that they can’t REPORT a bug even if they found something because the Feedback system requires a current license!

The Real Software beta list policy has been pretty wide open since I’ve been using REALbasic.  All you had to do was sign up for the beta program.  After that you could access the download URL’s for the beta versions.  It wasn’t until recently that you had to log in to the Real Software website to download beta versions.

Really all this change does is enforce a logical limitation.  Frankly, the beta list has needed to be pared for a long time.  There’s often a lot of inconsequential chatter that doesn’t help me as a beta tester nor helps Real Software fix anything.

Issuing beta builds is good for Real Software as we tend to use the product wildly different then they do.  Don’t get me wrong, they build the Real Studio IDE using Real Studio so they do eat their own dog food but they only have a handful of products in a limited number of use-cases.  We tend to use the product in abnormal ways either by design or by ignorance.

I wonder if even enforcing the current license restriction goes far enough.  The point of being part of the beta program is to report bugs and suggest changes.  I could probably name two dozen people that are vocal on the beta list that I’d want on MY beta list if I had choice.  Those are the people you want to keep.

Likewise, I know of a handful of people on the beta list that I’d find a way to politely leave off the new beta list.  Not that they’re wrong they just tend to be unreasonable in their reactions to beta software bugs.  When you’re unreasonable then engineers tend to ‘rooster up’ and resist the change.  Then we all lose and that’s no good.

Perhaps a good way to make a quality beta program is to start over with the list of current people that a) have a current license; and b) have submitted a Feedback report in the past 12 months.  I am almost afraid to see how low that number is.

I think this policy enforcement is probably a good thing before the 2012 Release 2 hits.  This is the release where the licensing gets simplified (for most of us).  So this means, potentially, a lot of new people using Real Studio for the first time.  So unless they get a paid license, they won’t be able to participate in the program.  Of course Real Software hasn’t announced anything on what happens to the beta program for R2 yet but I believe this to be a reasonable interpretation given the enforcement of the current policy, don’t you?

For me, this changes nothing.  Perhaps with less chatter RS can help focus our energies a bit better and if we do have something to report our voices don’t get drowned out.  I think this policy enforcement is a very good thing.

Anyway, what do all of you think about this change?  Is it fair or unfair?  Do you think it’s a good, bad, or do you even care?

21 thoughts on “Change (Not Really) To the RS Beta Program

  1. I think it is important to distinguish between feedback and beta program. The rationale behind requiring a current license for feedbacks has always been that only then can you run the current version and provide bug reports for the current version, and not clutter the list with reports on old bugs which might have been squashed already. With the new IDE being freely accessible everyone should then be able to contribute to the feedback and report bugs. Note that feedback is also an essential source of information on why something you are trying to do isn’t working (“it is a known bug”) and potential workarounds.

    RS has the right to run their beta program however they see fit. It could be invitation only. It could be open. However I would be worried about the shift in priorities. As an experienced developer said to me once when I complained about buggy in-build controls (movie player anyone?): “I’m not using any of the build-in controls, I just roll my own”. So will simple but very noticeable bugs (like the drawer shifting) which beginners can’t work around not even be reported because the top programmers don’t use them anyway? On the other hand will advanced features which are of no use to beginners (delegates, introspection, etc?) be pushed at the cost of fixes required by beginners?

    RS is making their IDE free. They hope to attract a lot of new customers. But have they prepared REALbasic properly for them, or are they going to be frustrated by bugs they don’t know how to work around? Or as I say in my signature: “Basic functionality must be rock solid.” – if you want to not only attract but retain new users to REALbasic then that is a must. Unfortunately I don’t see that being achieved yet.

  2. Markus Winter :
    I think it is important to distinguish between feedback and beta program. The rationale behind requiring a current license for feedbacks has always been that only then can you run the current version and provide bug reports for the current version, and not clutter the list with reports on old bugs which might have been squashed already. With the new IDE being freely accessible everyone should then be able to contribute to the feedback and report bugs. Note that feedback is also an essential source of information on why something you are trying to do isn’t working (“it is a known bug”) and potential workarounds.
    RS has the right to run their beta program however they see fit. It could be invitation only. It could be open. However I would be worried about the shift in priorities. As an experienced developer said to me once when I complained about buggy in-build controls (movie player anyone?): “I’m not using any of the build-in controls, I just roll my own”. So will simple but very noticeable bugs (like the drawer shifting) which beginners can’t work around not even be reported because the top programmers don’t use them anyway? On the other hand will advanced features which are of no use to beginners (delegates, introspection, etc?) be pushed at the cost of fixes required by beginners?
    RS is making their IDE free. They hope to attract a lot of new customers. But have they prepared REALbasic properly for them, or are they going to be frustrated by bugs they don’t know how to work around? Or as I say in my signature: “Basic functionality must be rock solid.” – if you want to not only attract but retain new users to REALbasic then that is a must. Unfortunately I don’t see that being achieved yet.

    Really? Could you name some basic functionality that isn’t rock solid? Keep in mind that if there’s a bug printing an odd-sized page in black and white on 64 bit versions of Linux, I would still consider printing rock solid. The bulk of our Feedback reports come from advanced users. For the most part, the basic functionality that beginners and the majority of users utilize is pretty solid.

  3. @Geoff Perlman
    Geoff, I’m not sure you want to open that flood gate. There are several controls and systems in Real Studio that we’ve just given up on because they are definitely not ‘rock solid’.

    Most of them are from you folks not using them in your own products. Outside of very simple demo apps, Name a Report you’ve made using your own reporting tools? Show me where you’re using TextField masks? Show me where you’re using a WebMoviePlayer that works properly in FireFox?

    This has been a common theme over my years using Real Studio: unless it’s a pain point for you and your team, it’s awful hard to get some things fixed. You have a mature product, the IDE, that does what it does very well. In my world view as a consultant, Real Studio is lacking some very basic fundamental controls (date, time, calendar to name a few) and abilities (true RTF) and there are controls that just don’t work like they are advertised (TextField masks for example). I believe this is fundamentally because you don’t use them the same way we do.

    I’d much rather you spend more your time fixing and expanding the basic functionality over creating a new IDE. Frankly, the new IDE won’t help me sell more consulting work but improved functionality will.

  4. Bob: All the feedback links have been stripped from my post – so here it is again without carets (or however you call those damn things):

    @Geoff: does this mean you have fixed my two pet peeves, the movieplayer and the drawer?

    Drawer:

    Submitted 6 years ago: Bizarre drawer behavior on Mactel iMac
    feedback://showreport?report_id=2109
    If you have a drawer on an Intel Mac, moving the parent window causes it to resize randomly. It makes the drawer useless. It does NOT happen on PPC Macs. One clue (maybe) — if you hold the parent window for a second or two before “dropping” it, no problem. But if you move it quickly, as in adjust the window on the screen, the drawer goes crazy.
    Bug first reported in February 2006 (yes, six!) and was verified, and resubmitted in May 2008 after the FogBugz transition. Bug report was still “open” in 2011, not even reviewed. Was reviewed at the end of 2011.

    StyledText / Label:

    For the first two: according to Norman “The mistake here seems to be that the StyledText operates in BYTES, not characters, for the lengths”

    Pasting into Styledtext box sometimes shifts styles by one character
    feedback://showreport?report_id=7537 1. Mai 2009

    Either the StyledText property or the SetTextAndStyle method doesn’t like extended characters
    feedback://showreport?report_id=13275 26. August 2010
    Under Cocoa build crashes with “CocoaTextArea:GetTextStyle was called and is not supported”.

    Submitted 3 years ago: Label.Multiline does not break a contiguous line of text on Windows
    feedback://showreport?report_id=7357
    Also true for Label (bug report 19140) – btw seems like a VERY bad idea to match the bug report for label 19140 into the bug report for StyledText – it should be the other way round. StyledText is deprecated, so this way the bug report is unlikely to ever get looked at!

    Submitted a year ago: Cocoa: TextArea.SelTextSize and other properties are missing when .SelLength =0
    feedback://showreport?report_id=17242

    Cocoa TextArea.BackColor doesn’t work
    feedback://showreport?report_id=19110

    MoviePlayer

    THE MOVIEPLAYER MUST BE THE MOST BROKEN CONTROL IN ANY LANGUAGE, and I feel RS has given up on it (and probably on the drawer as well) as the reports are not even being reviewed:

    MoviePlayer1 control ignores .SelStart and .SelLength properties
    feedback://showreport?report_id=3196 2. Juli 2008

    MoviePlayer1.Rate() only affects rate if currently playing; not next play.
    feedback://showreport?report_id=2094 1. Mai 2008

    Movieplayer control draws multiple position indicators on the slider during playback
    feedback://showreport?report_id=6152 9. Februar 2009

    Setting MoviePlayer.Height with no controller, sets wrong Height
    feedback://showreport?report_id=8598 29. Juli 2009

    MoviePlayer leaks memory each time .Play is called
    feedback://showreport?report_id=9251 17. September 2009

    Movieplayer control place on scrolling canvas does not clip properly at bounds of canvas
    feedback://showreport?report_id=6087 4. Februar 2009

    Can’t get volume for movieplayer controls on Intel builds
    feedback://showreport?report_id=4438 27. September 2008

    A movieplayer set to Looping=True has huge audio dropout during first repeat on Win32
    feedback://showreport?report_id=8827 19. August 2009

    Setting MoviePlayer.Movie = in the Stop() event causes a crash
    feedback://showreport?report_id=8668 5. August 2009

    MoviePlayer ControlBarHeight/Width incorrect with no Controller
    feedback://showreport?report_id=8597 29. Juli 2009

    ControllerHeight returns the height of the movie + the controller height
    feedback://showreport?report_id=4262 12. September 2008

    MoviePlayer.Volume always return 0 (with snow Leopard on MacbookPro)
    feedback://showreport?report_id=10362

    MoviePlayer Stop event does not get called on Windows when the movie finishes (QT only)
    feedback://showreport?report_id=18170

  5. @ Geoff: I would also propose: if you fix a bug then try to fix ALL bugs in the affected control. There isn’t much point just tinkering – you fix one bug out of ten and you still have an unusable control, plus you run the risk of introducing another bug or two in the same control. A control after being worked on should really be rock solid.

    It is like writing a method: you have to make sure the method does what it is supposed to do. There is no point in fixing just one bug, you fix until it is working. Otherwise you just go back and back and back, and the time it takes everytime to work yourself into the inner workings is being multiplied.

    I really feel doing each control properly and comprehensibly would massively speed up the number of bug fixes and result much faster in a more stable product.

  6. @Geoff: Really? Could you name some basic functionality that isn’t rock solid?

    1. The bookmarks on Windows. Trying to add them from the menu generates an error every time. I can drag-and-drop method names up to the bookmark bar, which sort of works, but there is a bug there that leads to double, triple, and even quadruple listings of the same method. I can go into customize and clean them up and exit the IDE and when I open it up again, the duplicates are there again. Your engineer can’t duplicate it, but I have no idea why I have duplicated this behavior on seven (7) different computers. In fact, I haven’t found a computer that they *do* work on. Without bookmarks getting around the code quickly is tedious.

    2. Transparancy problems on many controls on Windows. This renders many of the controls useless on Windows and I have had to buy third party replacements.

    I consider both of these basic functionality that is not rock solid.

  7. @Bob:

    🙂

    That’s not all of them, just a selection to prove my point, but yes, I would guess yours is longer than mine … and in this case longer is not better 😉

  8. P.S. And I only did it because Geoff asked: “Really? Could you name some basic functionality that isn’t rock solid?” … so I did.

  9. @Markus Winter
    Agreed. While there are many parts of Real Studio that *are* rock solid there are surprising gaps which is why I believe they don’t use them like we do or they’d have been fixed a long time ago.

  10. Geoff Perlman :

    Really? Could you name some basic functionality that isn’t rock solid? Keep in mind that if there’s a bug printing an odd-sized page in black and white on 64 bit versions of Linux, I would still consider printing rock solid. The bulk of our Feedback reports come from advanced users. For the most part, the basic functionality that beginners and the majority of users utilize is pretty solid.

    I believe I could.

  11. Bob Keeney :
    @Geoff Perlman
    In my world view as a consultant, Real Studio is lacking some very basic fundamental controls (date, time, calendar to name a few) and abilities (true RTF) and there are controls that just don’t work like they are advertised (TextField masks for example). I believe this is fundamentally because you don’t use them the same way we do.

    and a native TableView instead of Listbox control.

  12. My license expired 2 weeks ago. Originally, I wanted to renew, but after having had a look at the Cocoa fixes I saw the bug fixes that I need didn’t make it. As a result I won’t renew for now. So for me access to the beta was very good.

    And let’s not talk about problems with the html viewer on Carbon that wasn’t fixed for over 3 years because Cocoa was “coming soon”.

  13. Geoff Perlman :
    Really? Could you name some basic functionality that isn’t rock solid?

    Wow. Just Wow.

    Geoff, I’m afraid this disconnect with your own product explains a lot.

  14. @Geoff: as you consider printing rock solid, here are some verified bugs:

    5314: that you can’t cancel a print job (it might hang or even crash the app, requiring a force quit), reported 2008, ranked at number 17

    15804: Rotated Text, via stringshape, draws a black box instead of text when printing to the printer graphic

    11560: Usage of GDI Plus and printing a Reports ends up in an empty sheet of Paper

    10958: Printing more than one copy does not work on Windows 7

    12452: Printing html from a htmlviewer causes the system to hang on MAC OS X

    17232: Report.Document.PageCount() used in a ReportLabel.BeforePrinting-Event returns always NULL

    18514: one can’t print at higher than 72 dpi in Cocoa (ok, Cocoa is still beta so I give you that one)

    I wouldn’t call that rock solid.

    The problem is also that some bugs make other features useless – for example if you need printing then using rotated string shapes is a no go.

    And btw – with bug 5314 RS again merged in a LOT of similar but higher number bug reports. So what is the likelihood that this old bug will even be looked at again?

  15. Folks, I didn’t say that there are no bugs. Rock solid doesn’t mean bug-free. What I was saying is that the basic functionality is pretty solid. I’ve seen a few mentions of cases that are problematic (such as the printing DPI issue on Cocoa or the fact that you can’t print more than one copy on Windows 7).

    If the basic functionality was NOT pretty solid, no one would be using Real Studio. I look at our Feedback system every day. I interact with users every day. I use Real Studio myself. So there is no disconnect. We have users shipping products every day with Real Studio. Saying that it’s “solid” enough to do that doesn’t negate the need to fix bugs. Those are not mutually exclusive statements.

  16. @Markus Winter

    Markus, thanks for providing a list. Some people complain about bugs but when I ask them for a feedback case list, they don’t have one.

    FWIW:

    5314: This is still reproducible even in Cocoa (at least in the new IDE). So I’m assigning it to be fixed for R2.

    15804: I’m going to see if we can fix this one for R1 but if not, we will try to fix it for R2. There was another duplicate case which I have merged so the rank on this one just went from 118 to 59.

    11560: The notes are unclear on this one so I’m looking into it.

    10958: We will try to fix this for R1 but if not, we will see about fixing it for R2.

    12452: This one is no longer reproducible in 2012 r1b3 when building for Cocoa.

    17232: We will see about fixing this for r1 or r2.

    18514 is already scheduled to be fixed in R2.

  17. Geoff Perlman :Folks, I didn’t say that there are no bugs. Rock solid doesn’t mean bug-free. What I was saying is that the basic functionality is pretty solid.

    Sorry, Geoff, but my assessment of this situation is quite bit different. There are a number of basic UI controls that have a mix of problems on Windows related to transparency / double buffer / flicker that just make it a whole lot more work and expense than it should be. It’s a mixed bag some seem to work others don’t. Some have a double buffer property, others don’t. The Bevel button flickers with you run the mouse over it, the PushButton doesn’t. Some controls you can turn off the focus, others you can’t. Some look correct on a canvas, but not a Rectangle or PagePanel (or vice-versa). This is the most basic of functionality, that just has to work. When people try out Real Basic, they just expect stuff like this to work. Not to go online and see one of the issues listed above in the feedback system for nearly 4 years. I shudder to think how many potential VB6 converts you lost because you didn’t take care of the basic stuff like this when it was reported. The time would have paid for its self many times over.

  18. @Geoff Perlman
    Please don’t take this the wrong way, but as I stated above, you and your team don’t use your product like we do. Printing and reporting is a HUGE part of our real-world projects. Frankly, reporting and printing are big weak spots in Real Studio. Windows issues are also big deterrent to using Real Studio.

    I take some offense to you saying that people complain and don’t put them in Feedback. We certainly DO put them in Feedback and they languish for years. The fact that you, personally, looked at these reports and put them in the priority queue says that for the most part, our Feedback cases are ignored unless we publicly bitch about them. I don’t think you should be surprised.

    To me, it seems that unless you feel the pain it’s not a priority. I don’t know how to fix my opinion other than your team making full, complete, applications for clients like we do. Yes, you’d be competing against me and other consultants, but the number of bugs that would become ‘high priority’ would go up, IMO.

  19. Geoff Perlman :
    Folks, I didn’t say that there are no bugs. Rock solid doesn’t mean bug-free. What I was saying is that the basic functionality is pretty solid. I’ve seen a few mentions of cases that are problematic (such as the printing DPI issue on Cocoa or the fact that you can’t print more than one copy on Windows 7).
    If the basic functionality was NOT pretty solid, no one would be using Real Studio. I look at our Feedback system every day. I interact with users every day. I use Real Studio myself. So there is no disconnect. We have users shipping products every day with Real Studio. Saying that it’s “solid” enough to do that doesn’t negate the need to fix bugs. Those are not mutually exclusive statements.

    “Basic functionality” is not the same for all users. For example, REALSQLDatabase constitutes basic functionality for us. It is not “rock-solid”, and hasn’t been for a few years now.

    The fact is that Real Software has had serious quality problems since maybe 2008, and I can be more specific. Although I think that the situation has been improving over the last year or so, those of us who have been using RS for some time evaluate claims against our experience over this time.

  20. @Geoff: Doh! Serves me right for making a second list of bugs which are not my bugs. My list is further up ;-P

Comments are closed.