Why the REAL Studio Feedback Application is Bad

The number of bug reporting systems REAL Software has implemented in my time using REALbasic has been astounding.  Of course, some of the changes revolved around the emergence of the internet, but still, I’m amused at the iterations.

There was REALbugs (?) that was a standalone app that I never really used because I was a newbie. It memory serves, it was very much a Mac OS 9-type application.  As to how well it worked – I couldn’t tell you.

Then there was the feedback web site that allowed you to sign on to a bug to ‘vote’ for it.  It was nice, but people would literally sign on to EVERYTHING which defeated the purpose of the voting system to begin with.

Then RS switched to FogBugz and abandoned all of the old feedback items.  Supposedly, it would make life easier on them and they’d be able to fix bugs easier and faster.  Unfortunately, it made bug reporters life much harder since we couldn’t search the database.  RS was inundated with duplicate items and it was hard to tell the good reports from the bad.

So, now, in the current iteration we’ve come full circle and we’re back to an application called Feedback and is part of the download package.  It’s better in most respects to the previous system other than that it’s an application which just seems….odd in today’s internet world.  But I understand why they did it and for the most part it works.

It’s user interface forces you to search for an existing bug before you can add a new one (that’s good!).  You can see the newest cases (handy if you’re evaluating a new version of REAL Studio), My Cases, Participating In, and My Favorites.

You can add cases to your favorites list by clicking the star at the bottom of the cases listbox.  But a not so well documented feature is the Priority List.

The Priority List allows us to prioritize our bug fixes and feature requests.  This supposedly gives RS some feedback into what we want fixed or implemented sooner rather than later.  It’s also weighted in favor of the Professional and Enterprise users (we can argue whether that’s a good or bad thing in another post).  There is an algorithm to it but when I had my conversation with an RS employee I didn’t take notes so I might be off on the actual numbers.

A Personal license gets a weight of 1.  A professional user vote gets a weight of 10 and Enterprise has a weight of 20.  So Enterprise users are 20 times more powerful than Personal licenses.  That doesn’t strike me as too off base since a Personal license is $99 and the Enterprise is $995 (again we can argue the logic of the weighting another time).

Perhaps the smartest thing they did with the priority system is that you only get five votes and if my memory serves (forgive me, this is the day I got the flu), I believe a #1 rank gets 5 points and the last vote only gets 1.  So you have to be smart with your priority list and to edit it, go to My Cases and click the Edit Priorities button at the bottom of the list.  A separate list of all the cases you have some interest in are presented and you drag reorder the cases as you see fit but only the top five are counted and in the list are bold.

You can find the current list by clicking on Top Cases which brings up the current list.  If you double click on the list item you can see the internal rank in the upper left hand corner of the report.  And this, in my opinion, is where this system breaks down.

The current #1 item is case 9433 (iPhone support).  While it sounds good at first blush, iPhone development is a big break from how REAL Studio works currently, and it requires a lot of intermediate changes and there’s no evaluation from REAL Software on how long it would take to implement iPhone support or what we would have to give up during that time frame.

For example, the current #2 item is case 738 (Controls and Window Backgrounds Flicker in Windows) and is a huge problem for Windows application and my guess is that it hurts RS when Windows users evaluate REAL Studio more than they know.  Would you rather fix this gaping bug or have iPhone support?  Me, I want and need, this bug fixed.

Unfortunately, there’s no way to vote an item ‘down’.  The current system is rigged to only tell how many people think it’s a good idea.  There’s no way for RS to get a feel for how many think the top item is a BAD idea, or don’t think it should be a high priority item.

I personally think iPhone support is a rabbit hole of time and money that RS could use to adding some nice features and cleaning up long standing bugs.  To support the iPhone REAL Studio has to add a number of very important things before it can begin to *start* development:  Cocoa (currently in beta), LLVM compiler support (currently in beta for RBScript) only, new editors, and untold engineering problems on three platforms (is that even possible?).

Cocoa has taken close to two years to implement and ‘new features’ have slowed down (partially from Cocoa and partially from our requests for more bug fixes).  RS has talked about LLVM compiler support – first with RBScript and then for the entire app and who knows how long that will take to get working 100%.  The other items?  Only the Universe knows.

I urge all of you to play around with the Feedback application and find your high priority bugs and mark them as such.  This is the only way (currently) to tell REAL Software on what your priorities are.  If you want iPhone support, that’s great, but I caution you to be careful with what you wish for.

Happy coding!

18 thoughts on “Why the REAL Studio Feedback Application is Bad

  1. In retrospect ‘Bad’ was a bad term for the title, but what the heck, I had already published it. Perhaps, ‘Not Optimal’ would have been a better term.

  2. Always an interesting read. Thanks Bob.

    For the sake of accuracy, Enterprise users get a weight of 10, not 20. They pay 10x as much as a Personal user so that’s the logic. Regarding iPhone support versus flickering on Windows, believe it or not those two items require approximately the same amount of work to fix. Why? Because to really solve the flickering of controls on Windows will probably require creating a new platform layer for Windows using .Net. We can probably improve it some what without doing this but that’s ultimately the real solution and it has a lot of pluses and minuses.

    What we want to know from our customers is what they want. It is up to us to figure out how to deliver it. The cost of implementation is something we have to deal with. I don’t think it should be part of our customers decision making process.

    Regarding what Aaron said about Feedback, that’s certainly his opinion. However, I don’t think that a web app is inherently better than a desktop app. If you are going to rarely use it, a web app can make a lot of sense. But if it’s something you are going to use on a regular or even semi-regular basis, a desktop app is a fine solution and provides a richer and more intuitive user experience.

  3. @Geoff — that a feedback application is something requiring more than occasional use should really be worrisome. 😉

    Joking aside, I’m not interested in a rich, intuitive user experience when reporting a bug. I’m interested in the least amount of pain because I’m already upset that I’ve *found* a bug. Finding out that I have to download a completely separate application from the product I am using is a threshold that someone already upset should not have to cross. Either build the feedback application into the product (so there’s not extra work on the user’s part) or make an online system that can be accessed anywhere at any time. THAT provides the better user experience when the situation is already upsetting.

  4. One of the benefits of a desktop application is that we can automatically pass information to it making the process far easier than it would otherwise be with a web application.

  5. Geoff, thank you for the clarification on the weighting system!

    I think you’ve proved my point on the drawback to the Feedback system, though. If there was any sort of evaluation from RS on those two items I would have to weigh the merits of each. Knowing that bit of info *now* I would still rank the Windows flickering problem higher than the iPhone development simply because there are *WAY* more Windows machines out there than iPhone devices and despite the in-roads that Apple has made in the past five years, a vast majority of businesses run on Windows. Hence more bang for the buck and more likely for *me*, as a consultant, to sell RB to a client. One persons opinion and I know that there are many that feel differently.

    Prioritizing our bug reports and feature requests is a great idea, don’t get me wrong. But without guidance it’s merely a popularity contest. A plus/minus system (like Stack Overload, for example) might be a better solution. Otherwise, I can’t see how you could ever get a fair representation of what the community sees as worthwhile or not.

  6. @Aaron Ballman
    I’m with Aaron on this one. The feedback system should be in the application.

    Regarding .net. The sooner RealStudio supports .net on windows the better (for me). It’s one thing that would convince me to upgrade from RB5.5. My main development environment is now Visual Studio 2008. Being able to create presumably fast/compiled dlls from RealStudio into a Visual Studio project would be attractive for both internal projects and for marketing to other developers in the Windows environment.

  7. Well, there’s more to it than that. For example, you are assuming that all resources are equal. They are not. We have engineers that can work on Mac OS X, we have those that can work on Windows or Linux, still others that work on the iDE and portions of the framework written in REALbasic. So applying resources to the Windows flickering issue for example is not going to interfere with resources applied to Mac OS X or iPhone. That’s why having an estimate is going to be misleading. I understand what you are trying to suggest but I don’t think it’s possible at least not to do it well enough. And I question how many users would really take a lot of time to consider all this information even if we did provide it.

  8. No solution is going to be perfect — but the current situation has a fundamental flaw.

    Lacking the ability to down-vote something means that “zero” has multiple meanings. If something starts at zero with no no down-vote, then does that mean it’s uninteresting? Unimportant? Bad? There’s no way to tell. A feedback report saying “Every time you close REALbasic, a puppy should die” has exactly the same value as “In my particular odd situation, I get a crash.” Without downvotes, there’s basically no way to tell the difference between an unpopular idea and a downright bad idea.

  9. I will keep this short because I am typing on my iPhone.

    I would prefer being able to write iPhone apps over windows flickering.

    I use windows as my main machine. I hate the flickering on windows with rb apps. However if you do some work you can minimize the flickering. And programs not written in rb still flicker. That is windows.

    But the reason I prefer iPhone apps is becase I can’t write iPhone apps in realbasic right now. Plus if it is the same effort as windows flickering the I definately prefer it.

    The iPhone is a whole new platform I can code and sell my services on.

    This is much more valuable than just perfecting the windows flickering.

  10. @Jay
    Thanks for the comment. Interesting take. I don’t agree with it, but then we don’t have to. 😉

    As an aside, I wonder what else we would get as a side benefit from getting rid of flickering? Full COM support? Access to the .NET framework?

  11. I came across an interesting ‘feature’ of the Feedback app. While I like the fact that it forces me to search the database before I submit a bug, it only makes me search once. So once I’ve done a cursory search, there’s nothing to force me to search again. In the ideal world, it should disable the “Create New Case” button when I create a new case.

    Since I searched for it, I found it had been reported already:

  12. If you think an up/down vote would be a better way than the current priority system, please put as one of your top priorities.

    There’s some irony in this request, no?

  13. I think Bob has made a very valuable point and as a frequent StackOverflow user, one I think has been shown to work – if you’re going to have a voting mechanism you really MUST have a downvote!

  14. There was a lot of discussion about this in the notes of the feedback item. It appears that it’s already been dismissed. Oh well.

  15. @Aaron Ballman
    Case 11569, if anyone is interested. It is currently ranked 401.

    Nothing specific but there was a comment “we want feedback, but we won’t always agree with it.” To me that wording says it’s been reviewed and dismissed.

Comments are closed.