Lesson 01: Examples of AR apps and which AR library/SDK to use

Examples of Augmented Reality Apps on Android

I started off a month back by firstly looking to see what Android apps were already in the market claiming to be AR apps.

A rather basic one that shows the augmenting of 3D primitives (basic polygonal shapes) along with basic interaction by manipulating the objects onto the surrounding environment is Virtual Graffiti. This is an app that uses the black and white pattern markers to track the camera's movement. From this it calculates the camera's relative position in order to determine the correct location on the screen where to render the 3D objects.

Googling for Android AR apps will yield a tonne of sites all with their own ranking and list of the top XX apps. One comprehensive list I found with videos was a site that lists the 33 coolest apps.

One of the coolest, and from what I can gather, most popular AR apps is Wikitude. This app uses the GPS/location info services and other sensors available on your Android phone to overlay POI (points of interest) on your screen as you view the area where you are. Imagine simply holding the camera up in some random direction and nearby restaurants and shops etc popup on your screen letting you know where they are.

Another one that I particularly like is Google Sky Map which shows the real relative position of stars, planets and constellations by just pointing your camera in that direction.

In order to come up with your own idea for your first project, I suggest first watching lots of videos and trying out the ones that catch your eye. Then research into the technical aspects of what would be required to get an idea for the level of complexity. Don't go overkill with your first idea if your main aim is to learn the ropes - you can always create cooler apps down the line once you get more confidence. God knows I've already changed my mind about 3 times over the past few weeks!

Choosing an AR Library

Unless you are an overly eager beaver, don't believe in the concept of reusing code, do believe in reinventing the wheel (in which case you probably wouldn't bother reading my blog in the first place) then your next item of business is to choose an AR library to build your app upon. As somebody who has forget most of his high school linear and matrix algebra then using an AR library is an almost must. It will provide all the functions you require for tracking and funky dunky matrix calculations which are needed to place your objects in the correct position in 3D space. The inner workings of the computer vision algorithms is a topic for another day (ha! I recall my computer vision lecture from my compsci uni days and am glad that's over!).

A fellow blogging developer over at www.evilcodingmonkey.com has provided some similar articles about developing AR on Android which I referenced during my research phase. He has an excellent writeup including reviews and summaries of the various libraries available for AR on Android. I won't bother re-writing my own list here so please do check out his site.

Qualcomm's QCAR Platform

Suffice to say, I chose to go with Qualcomm's QCAR SDK. Some of the main reasons being:

Pros:

• Seems to be an active project with a big corporation behind it and lots of documents, references and lively forum available
• Available for both iPhone and iOS
• Videos show it to have pretty robust tracking (as compared to some AR apps I've seen with poor AR tracking where the objects are always disappearing off the screen as it fails to register the tracker in the screen)
• Has Virtual Buttons which allow for some cool looking interaction with your app
• Doesn't require the traditional black & white marker patterns for tracking; you can use photos

Cons:

• Registering photos as trackers must be done through Qualcomm's website's online tool (you upload the photo and then download some files to bundle with your application). The UI itself for the tool is pretty lame. Being able to do this programmatically would allow developers to add some cool functionality to their apps.
• The resource files containing your tracker data must be bundled with your app when you release it, i.e they cannot be dynamically loaded at run-time (although this feature is apparently going to be added in an upcoming version according to a post I read on Qualcomm's forums)
• The SDK bundled in your app will automatically sends anonymous data (usage statistics I suspect) to Qualcomm's servers. While this isn't a problem for me, I can imagine some situations (especially corporate ones) where the network access of an app must be strictly controlled to adhere to compliance regulations etc.
• The library is not open-source and written using the Android NDK (read more in the next post) which means moving from Java to C/C++ coding using JNI (bridge interface between Java and native level C/C++ code). This increases complexity of your app, the learning curve and also drastically decreases efficiency for debugging (which I have slightly improved on with my Wogles utility which I will write about in a future post).

Watch the intro video below that demonstrates the sample apps using QCAR that Qualcomm has released.

Or, alternatively, view the video on the Qualcomm website. When Qualcomm initially released QCAR they held a competition to encourage development with their platform (first price was something like over USD$100,000!). You can view a video from the winner and also a list of all participating apps under the "Featured Commercial AR Apps" section of the QCAR top page.


In conclusion


So now we have an idea of what can be achieved with AR apps on Android and furthermore what kind of functionality QCAR can offer as a versatile AR library. Sometime down the line I'm hoping to play with QCAR on iPhone and maybe one day experiment with other libraries out there, but for the meantime I've already got a lot on my plate.