Sergei Volodin
Imagine a typical augmented reality app. It would probably use regular qr-code-like markers. They are easily detectable and that’s why they are so detrimental to the scene. For qml-ar we used seamless markers not disturbing the user and achieved great performance allowing for a smooth experience.
Augmented reality is becoming more and more widespread and has a great range of applications. One of them is education. Augmented reality allows to greatly increase the cognitive ease of learning by creating interactive scenes explaining a given topic. However, the currently widespread AR applications use markers which would significantly reduce the aesthetical beauty of the scene and would distract the user and thus decrease the cognitive ease. Specifically, the QR-code-like markers are designed specifically to be easily detectable. This is what makes them so distracting: contrasting colors and sharp edges.
There exist multiple marker types which are seamless, or which do not add any negative impact to the scene. One of them is the Random Dot markers developed by Hideaki Uchiyama. They consist of a number of dots places to the scene at random places. Using perspective-invariant features it is possible to detect them given a picture of an object. The advantage of this project is that, compared to others, it has its code available.
This code, however, does not run smoothly on portable devices because of performance issues. We have optimized it using a parallel pipeline and GPU processing and have achieved great framerate and latency on Android. Furthermore, the library is easily usable in Qt/QML which makes it applicable for multiple use cases.
The talk will go in details in the specifics of seamless markers available and in the details of the implementation of the library which resulted in the speedups. This path involves extensive profiling, using Qt parallelism primitives, writing shaders, and hacking Android.
The code of the library with full details is available at https://github.com/chili-epfl/qml-ar