s = “Select field1, ” _ //1

+ “field2, ” _ //2

and so on but it’s still a pain.

Anyway, Christian S. posted a comment that got me thinking.  He said:

idxField is good. It can speedup applications a lot.

Is that really true?  I have never tested it so I created a little test application using an actual query from the project I’m working on.  I run the same query twice.  The first time I loop through the recordset using nothing but idxField and the second time I use nothing but field which uses the field name.  The results are interesting and confirmed my suspicions.

If I run my two functions just once and bring back only 1 record and do one read in the function the idx field is indeed faster by about 50%.  However my testing differences range from 6 to 300 microseconds faster and while I didn’t average out all the results, I’d say most times it was around 60 microseconds faster.  I ran this test a lot to see if I would see a wider variation but did not.  My 300 microseconds difference happened just once out of about 25 runs.

I changed my query so that instead of bringing back 1 record it brings back over 700 and I loop through the recordset and then do the same same test.  IdxField comes back about 15% faster.  When I loop through the same row 10,000 times there is no statistical difference which means that while Field is slower it’s a one time hit since my tests show that even doing the same pull 10,000 times the total difference between the two functions is still only about a 10-15% difference in overall speed.

My conclusion is that yes, idxField IS faster than using Field but to say it speeds up an application “a lot” is an exaggeration.  In 10 years of Real Studio consulting work I have rarely done an application that requires that much speed from the queries.  If you have any sort of graphical user interface I think it safe to say that updates to the controls on screen are significantly slower than pulling data out of the recordset.  That’s NOT to say that the query itself is fast because often times that is the most limiting factor.

I won’t discourage you from using idxField.  If you do, though, I really encourage you to use constants named after your fields.  So perhaps you create a constant named kField1 and the value is 1 for the first field in your recordset.  The only drawback to this approach is that you (or the developer after you) will change the SQL query and all the code using idxfield no longer works because your field order is different.

I much prefer the explicitness of using Field because I never have to worry if I change my field order.  It’s very readable.  It’s not perfect and there are plenty of reasons why it’s not the best (there’s a reason why we use ActiveRecord in a lot of projects).  Use what’s best for you and what’s best for the five years from now.

What say you, RB developers?

A lot of Real Studio developers avoid them because the term ‘database’ is scary and mysterious and brings up images of having to wrestle with huge problematic installations of MS SQL Server, PostgreSQL and MySQL.  While you might have to use them eventually, we do a lot of development on the lowly and surprisingly powerful SQLite.  It is a lightweight database that offers most of the features of the big database servers.  But for many, SQL, the language behind databases, is mysterious and arcane.  Not so!

At last weeks Real Studio Database Days training in Frankfurt, Simon Larkin of QiSQL gave a very interesting talk about SQL.  He also has a section on his website devoted to learning SQL.  His SQL School takes you through the basics of the terms and then starts introducing you to more and more complex situations.

Equally important is his Database Design section that walks you through the best ways of designing your database.  Learn about primary keys, entities, relationships and database normalization.  All in all the tutorials are very well done and if you’re just starting to use databases this is a good place to start.

Simon likes to use raw SQL, which is fine, but I’m lazy and SQL-challenged at times so I tend to use a number of SQLite utility applications.  Since each seems to have some compelling feature over the others I use the one that best fits the project.  I use Base, SQLiteManager, NaviCat, and the FireFox plugin SQLite Manager.  Of these, SQLiteManager and NaviCat can open encrypted databases and only Navicat being able to remember the encryption key.  Navicat also has a visual Query Builder that newbies to SQL might find attractive.

What SQLite tool are you using and why?

• Stéphane Pinel from Real Software
• Geoff Perlman from Real Software (via video chat)
• Jens Boschulte from DynaForms GmbH
• Simon Larkin from QiSQL
• Thomas Tempelmann
• Christian Schmitz from Monkeybread Software

Currently there are over 30 people registered from eight countries including Germany, Austria, Belgium, France, United Kingdom, Netherlands, and the USA.  This is a good time of the year to get away.

More info can be found at http://www.monkeybreadsoftware.de/realbasic/events/#database days

After checking flight costs there’s a possibility I’ll be there.  If I do, I’ll do a short presentation on our REALbasic implementation of ActiveRecord.  If you’ve not used it, ActiveRecord lets you use auto-complete of your tables/fields in the Real Studio IDE, takes advantage of the strong type-casting in Real Studio and, takes advantage of REALbasic events.

Real Studio doesn’t have database binding like Visual Basic 6 but it’s not a real big deal.  If anything, the lack of binding makes the code more explicit (i.e. easier to read) and you don’t have to go hunting through control properties to find table and field names.  The Real Studio database classes are generic so it doesn’t matter, generally, what database you’re connecting to.  The drawback to the lack of binding and the generic classes is that it does lend itself to creating the same code over and over and over again.

Because of the nature of Real Studio many users tend to put their db code into the form (window) and tie it to controls.  This leads to spaghetti code with the database specific code all over the place and makes changes to your database harder.  Seth has done two presentations at ARBP conferences 2009, 2011 and introduced attendees to ActiveRecord that we’ve used for years now.

Active Record is a very simple, and limited Object Relational Model (ORM) system.  It allows us to create REALbasic classes that the IDE knows about.  It’s not exceptionally strong with the relational data, or large blobs, but it can be programmed to handle it.

In a new project we’re converting an existing Visual Basic 6 project with roughly 25 tables and several tables have over a hundred fields each.  Using conventional means it would mean having a database editor open so I can copy and paste field names all the time.  However, using ActiveRecord we created the classes (we have a utility to do this) and now the IDE knows the table and field names.  This makes coding very fast and they’re is no worrying about spelling errors and there’s no longer any issue of what the data type is because the class knows what it is.  This is nice since the compiler will pick up any many errors that may not usually find until runtime.

The client was ecstatic after the conversion since he figured that would have taken about 20 hours to convert the VB6 code into something useable in RB.  Instead, between our utility and ActiveRecord it took me less than 4 hours.  So now instead of spending all the time getting classes ready, we’re doing the real work of connecting up the UI to a set of data aware classes.

Another feature that was added was to flag the developer if a field is in the database that isn’t in the class.  How many times do you add a field to the database (or a coworker does) and you forget to hook it up.  This doesn’t happen using ActiveRecord.  You can have class properties that aren’t a field, but if you delete a field property that’s been used in the application the compiler will flag you on it and that’s very useful too.

ActiveRecord makes extensive use of Shared Methods so that all of the database code for that table is access from that class and that class only.  It has a number of methods built-in such as getting a list of rows (in array form) and finding a record by the primary key.  It’s easily extensible.

Like I said earlier, it’s not perfect.  It doesn’t handle relational data at all, but it can be modified to do so.  Large blobs can slow it down, but in the few times this has been a big deal we’ve implemented ‘lazy loading’ where we don’t load that particular field until we ask for it.

We have a single tutorial page up for it now at the main website.  We’ll eventually turn this into video tutorials and we’ll demonstrate it in more video’s.  It’s an MIT style license so feel free to use it.  If you have additions and suggestions, please don’t hesitate to contact us.

More information, and downloadable classes can be found at http://www.bkeeney.com/realbasic/activerecord

It’s a good question in regards to Real Studio too as it makes you think about what you don’t use.  I’ve asked the question before on what’s the one thing you need above all else in Real Studio.  But removing something is a much harder question.  So it should probably be something that’s not very good, or makes things worse, or something made irrelevant by 3rd party tools.

After thinking about it for just a few seconds I came up with the one thing that I never use in Real Studio:  The Database Editor.  For me, it’s the one thing that is worse than useless since it makes the job of managing your databases harder.  I mean,it’s just not very good, in my opinion.  Based on my experience answering questions in the Real Software forums it’s not an uncommon experience.

In reality, the database editor experience is much like any generic tool:  it just doesn’t have the features that match up well to tools built for the specific database.  If you want a good SQLite tool there are some awesome commercial versions available.  Heck, there is a freeware version that works inside of FireFox that’s better than the DB editor, IMO.  The same goes with MySQL, PostgreSQL, Oracle and any other database that RS supports.

I’m a big fan of Navicat as they have versions of each of the aforementioned databases.  Granted, Navicat has a generic user interface and it’s a Java app (I think) but it’s the only thing that Navicat does (database admin tools).  It’s interface is consistent across all of their versions so it’s no big deal to move from the SQLite version to the PostgreSQL or the MySQL version.

If the Database Editor was removed from Real Studio would anyone really notice?  What would you remove from Real Studio if you had the chance?

This is not particularly good news for those that use REAL Server.  REAL Server was a product that was heavily hyped for several years and is no longer available for sale and support through REAL Software.

The good news is that the product isn’t going away.  REAL Server (or REAL SQL Server for those that have been around long enough) was purchased from SQLabs years ago and the primary developer, Marco Bambini, became a REAL Software employee and continued development of the product.

Marco and REAL Software parted ways several months ago.  It was not until this week that REAL Software acknowledged his departure and its effect.  It seems that as part of their separation deal, REAL Server will go back to SQLabs and be sold and supported by them.  More info here.

REAL Server, for those that don’t know the history, was originally written by Marco Bambini of SQLabs.  It was original called SQLite Server  and it made SQLite database files networkable and multi-user.

This was pretty exciting at the time because even though SQLite databases were lightweight and easy-to-use they didn’t work very well across network drives, it had none of the mechanisms required to handle multiple users, and it had no foreign key constraints.  From a certain perspective the acquisition and subsequent development of the product made sense from REAL Software’s perspective because it gave a migration path from the single-user SQLite database to an easy-to-use, install and administer database server.

Unfortunately, making SQLite into a database server proved to be difficult.  Until very recently, an SQLite database knew nothing about foreign key constraints (and even now you have to go out of your way to use them).  It also didn’t do any logic to handle concurrent users and all the headaches that go along with it (like record locking and user access control).  The fact the REAL Server did do some of this was a testament to how much work they put into it.  They fit the proverbial square peg into the proverbial round hole.

Unfortunately, REAL Server cost money and it was hard to compete against the MySQL and PostgreSQL database servers of the world which were mostly free.  For a while the license of MySQL was  a huge unknown mess (is it really any better now?) and REAL Server was marketed as a safe alternative to MySQL.

It’s hard to compete against free and well established database servers with hoards of developers contributing to it.  Foreign key constraints and stored procedures and all sorts of other goodies were already in MySQL and PostgreSQL and both continued to evolve with new features while REAL Server stagnated.

At REAL World 2008 Geoff Perlmann showed off a demo of a new version of REAL Software that allowed for plugins, written in c, to become its new pseudo stored procedures.  It was also supposed to show huge improvements in the number of concurrent users, have server side cursors, have client messaging and a host of other new features.

The demo was light on details but it was released later that year.  The last official release of REAL Server was in 2009.  However, many developers have found some of those new features to be buggy, and have stuck with the 2008 version.  Meanwhile, REAL Server has been stuck in perpetual beta since then with no appreciable work.  Now, users are stuck in limbo waiting for SQLabs to come up with a new release version.

Am I surprised by any of these developments?  No.  I always thought (http://www.bkeeneybriefs.com/2009/03/real-studio-the-good-the-bad-the-ugly/, http://www.bkeeneybriefs.com/2008/11/changes-at-real-software-part-deux/) that REAL Server was a product waiting for an audience.  And, as I said earlier, it’s very hard to compete with free, especially when you get into all of the drawbacks.  Heaven help you if you tried to get REAL Server inside of a corporate environment where database servers are specified to the nth degree and require dedicated support personnel.

This news sucks – especially if you had invested a lot of time and effort into REAL Server development in your projects.  The lack of new versions in the past year should have been a good clue, though.  Also, there were very few posts about REAL Server in the REAL Software forums.

I feel that the focus on the hobbyist developer blinds RS, sometimes, to what professional developers will gladly pay for.  Don’t get me wrong, there are a LOT of hobbyist REAL Studio developers and that’s great, but it’s been my experience that the hobbyist developers can’t pay for much and REAL Studio was a cost (even as inexpensive as it was) that most couldn’t afford.

It was a losing battle from the start, really.  It’s too bad that RS had to go through the painful realization that buying and building upon a product sometimes isn’t good enough.  It was too expensive for hobbyist developers and it wasn’t powerful enough for the professional developers.

In a nutshell it never took off and that’s sad.  It was a distraction and a drain on resources during a time when multiple developers were laid off due to the hard economic times.  Cocoa is now running into its third year of development and one has to seriously wonder if the distraction of REAL Server, even if it was just one developer, cost them some serious development time.  Certainly one could argue the money spent on development and marketing of REAL Server could have been better spent on other things.

Ultimately, the message this sends to the community is not a very good one.  From now on, we, as users, will have to weigh the impact of relying upon any tool from the company since it may or may not be supported years from now.  Granted, in this case, REAL Server is finding a new home and there will be support for the foreseeable future but what about the next new thing?  Do we have to wonder about Linux or Web App support five years from now?

What are your thoughts?

It’s very hard to evaluate and report on how well it works if I can’t even use it for more than 2 minutes (and no, I’m not running into the 10 minute demo limit).  It really makes me wonder what their QA process is or if we’re the QA  guinea pigs process.

There are a number of things to really like about the Valentina database.  It’s fast – there’s no doubt about it.  Valentina is truly a relational database (you cannot say that about SQLite or REAL Server) and it comes with a file (single connection) version as well as a server (multiple connections) version.  It’s been around a long time in REALbasic and you can use Valentina in a variety of ways including a class-way that is very object oriented.  It is also strongly type-cast meaning it won’t let you put a string into an integer field and so on (again SQLite and REAL Server are NOT like this).

There are a number of reasons why I stopped using Valentina but the biggest was bugs.  In one of my projects I got caught in a transition between db formats and because of my project deadline I was forced to use beta versions.  Which meant, of course, that they’d fix one bug and create another.  It was an ugly couple of months and it’s (obviously) left a bad taste in my mouth.  And yet I keep coming back for more….

It’s been a few years and I was really hoping that it was a nice stable environment.  Alas, I can’t say that it is.  I downloaded Valentina Studio 4.2 (their general admin tool) and decided to import my standard SQLite demo database for reporting tool comparisons.  Everything started off fine but when using their import wizard and hitting Finish it just goes *poof* (hard crashes) and a standard Apple error dialog comes up.

I suddenly remember why my eye tick comes back when I start working with Valentina.  Hard crashes were a common occurrence when I last used Valentina and is still obviously an issue (at least in Studio).  When you can’t perform a simple data import using a standard text file it makes you wonder what’s going to happen with your database.

It’s a trust issue, really, because the first experience with the product is the admin tool.  Instability in the showcase application (in my opinion), hints at deeper problems in the entire database.  Hard crashes are evil, bad things because it means the app quit in an uncontrolled fashion.  What happened to your database write just then?  Is the data good, bad or in some unknown state?  Either way I have a hard time telling a client that things will be okay if it happens to them.

If you are a Valentina and REALbasic user I’d like your opinion.  Am I just having a bad day or do you have these same frustrations as well?  Have you used the new reporting system?  Do you like it or not?

I don’t know if  RS marketing efforts are leading to these requests or if the ever-increasing Macintosh presense in the business world is making more people want cross-platform applications but something seems to have changed.  (Or it’s a combination of everything!)  Of course there’s nothing to say if the requests dry up in the next month or so.  Only time will tell.

I find it interesting that a lot of the recent projects are converting from either FileMaker or 4D.  RB can make a nice database application (assuming you know what you’re doing).  RS is always looking to enhance the user experience and I encourage RS to enhance their database tools.  I, however, do not want them to become another FileMaker or 4D because a generic application environment can do things better than a database-centric application, in my opinion.  Likewise there are things that FileMaker, 4D and MS Access do better than a generic application environment like RB.  It depends upon the requirements, in the long run.

Since I help run ARBP you can take this criticism with a grain of salt.  One of my biggest beefs with the RS developer program is that it’s simply a forwarded email.  If you signed up today you’d miss everything from last week (if you did sign up I’d certainly ask for all recent posts since $1k is nice chunk of change).  With ARBP we keep them around for a while (which is a problem all by itself that I’m trying to correct).  Another point is there isn’t a whole lot of information in the posts.  A contact information (name, email, phone) and a brief description of the project.  It would be nice to get more information.  What platforms?  Is it an existing project?  Is it a conversion project?  From what language?  Is it commercial or internal software?  I could probably continue on what I’d like to see, but I’d be repeating myself.

As a consultant, are you busy?  Better yet, are you as busy as you’d like to be?  Have you seen any trends in the past six months?