A long time ago, when augmented reality (AR) was not yet known worldwide (so long before Pokémon GO), in the summer of 2013 we already had a love-hate relationship with a thing known as a tracking target. But since we prefer love over hate any day, we got together with the Graz University of Technology later that very year to develop a tool for the local improvement of these tracking targets . What’s a target, why did it need improving and why is that relevant for today, you may ask. Well, read on to find out.
Since the first AR applications were developed, this innovative technology has quickly developed in many different directions. Firstly, that is due to the rise of smartphones and the fact that almost everyone has now got one. Another reason is the technology’s commercialization – there’s hardly a big player (i.e. Google, Facebook, Microsoft, Apple and so on) left who is not doing research and development in the field of augmented reality (if you’re new to the topic though, you can find a short introduction here). Basically, AR is always about displaying digital content in the physical world. In order for this to look realistic to the viewer, it is important that the positioning of the virtual camera matches the viewer’s perspective. The closer the match, the better and more natural the AR experience.
In order to determine the perspective of the virtual camera, irregularly tiled patterns were being used until a few years ago. These patterns or tracking targets are known to the AR application. When the target is being viewed through the camera stream, the app recognizes the target’s position and can place the desired content accordingly. As you can easily imagine, that isn’t particularly pretty or practical. Our first CodeFlügel business cards had such a black and white pattern on them, so we could easily hand our latest AR showcase to our customers – but we weren’t exactly happy with this solution.
Pretty soon, AR research was trying to tackle this exact problem and developed the so called natural feature tracking. That may sound complicated to you, but ‘natural feature tracking target’ is just a fancy way of saying picture or photo. Thus, we can now take normal pictures and integrate them into our AR app as targets. So when our app ‘sees’ the picture, it can calculate the virtual camera position and knows where and how to place the content. Which means that, for our new business cards, we could simply use our logo which then, for example, lets you watch our CEO show you around our office.
Nowadays, photos are most often used as targets for mobile AR applications. And that’s where we’ve had a few discussions with our clients – because from an AR developer’s point of view, one photo isn’t as good as another. Since the picture is used to determine the camera position, a lot of the accuracy and thus the awesomeness of the AR experience depends on it. As the name suggests, natural feature tracking uses naturally occurring features in the picture to determine its rotation. These are basically prominent details, edges, corners and contrasts. Round forms are completely unsuitable, since they contain no or extremely few natural feature points.
Ideally, there should be as many feature points as possible, which appear in many different compositions and are distributed evenly across the picture. As a rule of thumb: photos usually work better than artificially created graphics, large uniform surfaces and regularly tiled or repetitive patterns are problematic.
As you can see in the above example, most minimalist company logos are unfortunately not well suited for use as targets. The pattern overlay on the right has increased the number of feature points and distributed them more evenly, which makes for a better target. Especially in the case of company logos there’s a question of how much you can alter the image to make it a more suitable target. If the number of feature points is very low, it is usually better to use the logo in combination with one or more additional photos and some text, since a target does not necessarily have to consist of a single image. If you do use a single image, you can improve its tracking quality with some minimal local changes. Here are some examples for such minimal improvements – on the left are the original images and on the right you can see the image’s feature points in yellow.
Such small changes are barely noticeable and can dramatically improve the tracking performance – in some cases though, choosing a different image altogether may be the better solution. These individual decisions always depend on the application and the context in which it should be used.
Of course, AR tracking frameworks have developed a lot since we did our study on target improvement. In addition to natural features, they can now also recognize lines and fonts and use them to correctly position the camera. Further, additional features in the physical environment of the target are tracked during runtime in order to actively improve stability. Still, the initial positioning relies on the tracking target alone, which means that a single circle or line as well as a regular pattern continue to be bad targets. All in all, a close collaboration between customer/designer and developer will produce the most successful results – not only concerning the target quality, of course.
Elias Tappeiner, Dieter Schmalstieg, and Tobias Langlotz. Local Optimization for Natural Feature Tracking Targets. In Proc. IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2014, Munich, Germany, September 2014.