Nil Recordset

The question on how to deal with nil RecordSets is OFTEN asked in the Xojo forums.  Either the original poster or a well meaning responder will do something like this:

dim rs as recordset 
rs = db.sqlselect(sql)
If rs <> Nil Then
     If Not rs.EOF Then
       // No Records in Table or bad querry?
     End If
End If

It works, but it only masks the real issue.  Why is rs nil in the first place?  The answer is because the SQL statement to get the RecordSet had an error!  What is that error?  We have no idea because the code doesn’t ask for the error.

Most of the time the database will give you an error message that will point you in the right direction.  How do you tell that the database had an error?  You check the Error property.

dim rs as recordset 
rs = db.sqlselect(sql)
if db.error then
     //Do something
end


The database can give you an error code but this varies from database to database so I don’t find it exceptionally useful.  However, there is an ErrorMessage property that gives you a human readable message.  

dim rs as recordset 
rs = db.sqlselect(sql)
if db.error then
     MsgBox “DB Error: “ + db.ErrorMessage
     Return
end


In this example I’m showing this to the user but this is probably not a great idea for most uses because you don’t want the average user to know what happened.  However, it’s great while debugging.

The error you get back might be a little mysterious.  SQL is a language specific to databases and the error message returned expects you know a little bit about databases.

The second part of my message to you is that if you’re using SQLSelect or SQLExecute in your code you should look at PreparedStatements.  They’re safer in that PreparedStatement have some security features and does the heavy lifting of taking the input data and putting it into the right format.  What do I mean by this?

In a normal SQL statement you would have an SQL statement like this:

dim sql as string
sql = “Select * from users where last_name = ‘Keeney’;”


This selects all users whose last name is ‘Keeney’.  If you were going to allow the user to search for whatever name they want you would end up doing something like this:

dim sName as string
sName = txtSearchField.Text
sql = “Select * from users where last_name = '“ + sName + “’;”


Note that I had to manually put the single quotes in before and after the name since the database column is text.  This works great until you enter an Irish name like O’Neil.  The database will choke on this because it has an extra apostrophe in the statement.  You can escape this data yourself but what a pain.  Instead, use a PreparedStatement.

dim ps as SQLitePreparedStatement
dim sql as string
sql = “Select * from users where last_name = ?”
ps = db.prepare(sql)
ps.bindtype(0, SQLitePreparedStatement.SQLITE_TEXT
ps.bind(0, txtSearch.text)
dim rs as Recordset
rs = ps.SQLSelect
if db.error then
     MsgBox db.errormessage
     return
end


Nowhere in the code do I have to put in apostrophe’s or escape any data.  It’s just handled and this is true if you have database columns that are text, date, or numeric.

PreparedStatements have a number of advantages besides the automatic formatting.  They help with security and SQL Injection attacks.  If you don’t know what those are just do a web search to learn more.  You can reuse PreparedStatements in code but in my testing it doesn’t save much time as it appears as if the database plugin saves them internally.  Regardless, there are so many reasons to be using PreparedStatements over straight SQL.

To recap:  A nil recordset always mean a database error has happened.  You need to check for for the error after every operation whether that be a Select, Insert, Update, or Delete statement.  Basically after every SQLSelect and SQLExecute statement.  Ideally you want to start moving all of your code to use PreparedStatements because of all of its advantages.

Recordset’s are Funny Things

We have a new developer working at BKeeney Software and he’s in the learning phase of Xojo. Not only is he new to Xojo but he doesn’t have a ton of experience doing software development in general. As someone who’s been using Xojo for a long time it is interesting to see how people struggle with the language and the IDE.

Today we had one of those “teaching moments” as he was beating his head up against the wall with this bit of code in a save method. This is a simple Invoice application saving the invoice line item to an SQLite database that’s on the local machine. See if you can spot the mistake.

dim rs as recordset =db.sqlselect( "Select * From tinvoicelineitem where invoicelineitem_ID = " + str(iInvoiceLineItem_ID) )
if db.error then
   msgbox CurrentMethodName + " db error: " + db.ErrorMessage
   break
   return false
end
rs.Field("InvoiceID").IntegerValue = iInvoiceID
rs.Field("salesTax_Flag").BooleanValue = bSalesTax_Flag
rs.Update

if db.error then
   msgbox CurrentMethodName + " db error: " + db.ErrorMessage
   break
   return false
end


Have you spotted the error yet? I don’t blame you. It’s easy to miss and it took me a few minutes of staring at it to find it. The code runs with zero code errors and zero database errors. The only thing that’s wrong is that it doesn’t save any line item data.

Okay, I’ll tell you. I forgot to add the rs.edit command at the top of the method. Without telling the recordset that you’re editing it, it will never generate the SQL to update the RecordSet.



dim rs as recordset =db.sqlselect( "Select * From tinvoicelineitem where invoicelineitem_ID = " + str(iInvoiceLineItem_ID) )
if db.error then
   msgbox CurrentMethodName + " db error: " + db.ErrorMessage
   break
   return false
end
rs.Edit // IMPORTANT!
rs.Field("InvoiceID").IntegerValue = iInvoiceID
rs.Field("salesTax_Flag").BooleanValue = bSalesTax_Flag
rs.Update

if db.error then
   msgbox CurrentMethodName + " db error: " + db.ErrorMessage
   break
   return false
end


I can argue that the lack of an error is bad. Sure, I didn’t code it right, but you’d think there would be some sort of error if an update was called without a starting edit. I’ve added <feedback://showreport?report_id=35134> to the Feedback system.

To be honest, we don’t do straight Xojo database calls much any more simply because we’re using ActiveRecord for most projects. AR eliminates most of the tedious database coding and let’s you deal with the data and the UI. When you call the Save in ActiveRecord you don’t need to care if it’s a new record or not. AR handles all the gory (and boring) details for you.

What say you my Xojo friends? Do you get bit by this type of thing when doing database coding?

Recordset.idxField vs Recordset.Field

Yesterday I was complaining about my OPC project and I’ve added a few more bullet points in the comments section.  This morning I added a comment about how the idiot developer used Recordset.idxField(index) rather than Recordset.Field(“FieldName”).  I hate it when developers use idxField because it makes it tough to read and I constantly have to go back and look at the query and count fields.  In the short term I modify the query string in the IDE and put index comments in the in string.  Something like this:

s = “Select field1, ” _ //1

+ “field2, ” _ //2

and so on but it’s still a pain.

Anyway, Christian S. posted a comment that got me thinking.  He said:

idxField is good. It can speedup applications a lot.

Is that really true?  I have never tested it so I created a little test application using an actual query from the project I’m working on.  I run the same query twice.  The first time I loop through the recordset using nothing but idxField and the second time I use nothing but field which uses the field name.  The results are interesting and confirmed my suspicions.

If I run my two functions just once and bring back only 1 record and do one read in the function the idx field is indeed faster by about 50%.  However my testing differences range from 6 to 300 microseconds faster and while I didn’t average out all the results, I’d say most times it was around 60 microseconds faster.  I ran this test a lot to see if I would see a wider variation but did not.  My 300 microseconds difference happened just once out of about 25 runs.

I changed my query so that instead of bringing back 1 record it brings back over 700 and I loop through the recordset and then do the same same test.  IdxField comes back about 15% faster.  When I loop through the same row 10,000 times there is no statistical difference which means that while Field is slower it’s a one time hit since my tests show that even doing the same pull 10,000 times the total difference between the two functions is still only about a 10-15% difference in overall speed.

My conclusion is that yes, idxField IS faster than using Field but to say it speeds up an application “a lot” is an exaggeration.  In 10 years of Real Studio consulting work I have rarely done an application that requires that much speed from the queries.  If you have any sort of graphical user interface I think it safe to say that updates to the controls on screen are significantly slower than pulling data out of the recordset.  That’s NOT to say that the query itself is fast because often times that is the most limiting factor.

I won’t discourage you from using idxField.  If you do, though, I really encourage you to use constants named after your fields.  So perhaps you create a constant named kField1 and the value is 1 for the first field in your recordset.  The only drawback to this approach is that you (or the developer after you) will change the SQL query and all the code using idxfield no longer works because your field order is different.

I much prefer the explicitness of using Field because I never have to worry if I change my field order.  It’s very readable.  It’s not perfect and there are plenty of reasons why it’s not the best (there’s a reason why we use ActiveRecord in a lot of projects).  Use what’s best for you and what’s best for the five years from now.

What say you, RB developers?

Why Is My Recordset Nil?

This one comes up on the RB Forums a lot.  The post generally goes something like this:  “I have a database and everything works fine until here <insert code snippet> and I get a Nil Object Exception because rs is nil.”

If everything else is working fine then it must be the recordset.  If you’re not checking the database for an error you’ll never know!  The db.error property tells you that you had an error and the db.errormessage property does a pretty good job of telling you what the error is.

dim rs as recordset
rs = db.SQLSelect(“SELECT * FROM sometable WHERE somefield=avalue”)
if db.error then
msgbox “Error!” + db.ErrorMessage
return
end

//do something with the recordset here.

I find this happens a lot with developers that came from Visual Basic 6 where database errors throw an exception.  REALbasic doesn’t work the same way.  Get over it and start checking for database errors.

If you want to capture database errors you can do something like what we’ve done:

if gDB.Error then Raise new BK_Debug.BK_DatabaseException(gDB.ErrorMessage, s)

Where BK_DatabaseException is a RuntimeException subclass defined like this:

Sub Constructor(sMessage as string, sSQL as string)
If sMessage <> “” Then
Me.Message = sMessage
End If

If sSQL <> “” Then
Me.Message = Me.Message + EndOfLine +  EndOfLine + sSQL
End
End Sub

This can be useful if you have automated error handling and can receive this information.  Of course, if you’re creating a runtime exception you had better be handling them in the app.unhandledexception event otherwise your app will shutdown and you’ll have pissed off users.