In my previous blog, one of the commenters asked me about my thoughts on the new LLVM backend compiler that RS is implementing for REAL Studio. I’ll first start by saying that I am not a backend compiler expert, nor want to be. Much of what I’ll write about I’m getting from sources that are in the know, as they say.
What exactly is LLVM other than a fancy acronym? It stands for Low Level Virtual Machine and is a compiler infrastructure. It was started in 2000 at the University of Illinois and in 2005 Apple hired some of those original authors for their development system.
RB currently uses a custom written backend compiler. As you can imagine, maintaining a compiler is a lot of work and writing optimizations is not easy. LLVM was designed for multiple layers of optimization including compile-time, link-time, run-time and even idle-time.
What does LLVM mean for REALbasic applications? It means that applications should be smaller and faster. Code that isn’t used will be removed (dead-stripping code). Currently RB can do this but it’s not exceptionally efficient (but really not all that bad either) as it which leaves some unused code compiled into your application. For most of us this isn’t a big deal, but for many that start a project with a standard toolset, they might be introducing some inefficiencies into the final product.
Switching to LLVM is great news, though. Smaller, faster, better are all things to strive for. Without giving away too much information, the RBScript compiler is being reworked to use LLVM. It’s a good first step.
But that’s all it is. If you’ve not used RBScript, let me tell you, it’s not REALbasic. One of the most powerful features of REALbasic is the cross-platform debugger. Run in one environment while debugging in the other. Even when you are debugging locally (on the same machine in the same environment) you’re ‘remote debugging’. If my source is correct, switching to LLVM will require rewriting the entire debugger because the LLVM metadata system is different than RB’s debugger map system.
Since LLVM uses different metadata properties the current introspection system will break and have to rewritten. Plugins will also break. The IDE, the plugin system and even the reporting engine are a heavy consumers of introspection. So going to LLVM means a huge portion of the product needs to get updated.
Rewriting the debugger is not a trivial task. To do this the RIGHT way is a huge job that could take a couple of strong backend compiler developers 6 to 12 months to fully implement. Even hacking it together is probably a 4 to 8 month project and it would still take some time getting the remote debugger to work properly.
If all this sounds daunting and kind of scary, well, it is. Switching backend compilers is not something to do on a whim. It has to be well thought out and resources have to be allocated properly. The roadmap for features that will not be added during the transition time need to be thought out properly as well since limited resources are a fact of life in all organizations. A small organization needs smart long-term planning.
Make no mistake, though, the backend compiler switch promises to be take a while. It basically rewrites major portions of the product which will lead to some subtle and not-so-subtle changes. The risk is high and it will affect EVERYONE.
There’s been some talk about all the ‘freebies’ we will get when using LLVM (such as 64 bit support). Not true. Using LLVM gets you ‘closer’ but it still won’t be free. Switching to LLVM removes one of the many tasks involved with such support.
With all that said, I think moving to LLVM is a must. If RS wants to support different platforms it’s a smart move going forward. The optimization capabilities look very promising and will be welcome to many users. It IS the right thing to do. Just be prepared for some bumps along the way.
One final word. I’m a passionate RB user. I’ve also been a project manager where I think about the bumps in the road before they happen. I’m not an expert at this stuff, though, so any misconceptions or mistakes are entirely my own.