Sony UK presents its markerless tracking technology Magnat for PS Vita at GDC 2012

At GDC 2012 Sony UK has presented its new markerless tracking technology for PS Vita called Magnat. The technique allows to track basically any textured 3D environment.

The algorithm is able to initialize itself without any need of markers or pre-learning phase. It is also able to detect planar surfaces in the cloud of reconstructed 3D points. This allows the user to add 3D objects interactively.

Since those planar surfaces can easily be triangulated and the corresponding real video texture is known, they are able to deform these surfaces and render them for example liquid. See the second video for this nice effect.

While I have no further information about the actual algorithm that is used by Magnat, my first guess would be a variation of the popular Parallel Tracking and Mapping approach presented by Klein in 2007.

more videos after the break

Qualcomm presenting mobile 3D tracking at MWC 2012

"Sesame Street Augmented Reality Dolls Take AR to the Next Level"

Metaio Markerless 3D Tracking using Gravity Aware Algorithms

Metaio has announced to release in mid mars 2012 its new Juneio SDK, which will include a feature to track 3D objects and sceneries without markers. Metaio could win last tracking contest at ISMAR 2012 using this technology. The first video shows some insights of the gravity aware algorithms that were implemented to allow stable mobile markerless 3D tracking.

Markerless 3D Tracking Technology Video

Dense Tracking and Mapping in Real-Time

Great ICCV paper by Richard Newcomb about tracking and mapping techniques with a single camera that even calculates a deth map in real time.

"DTAM is a system for real-time camera tracking and reconstruction which relies not on feature extraction but dense, every pixel methods. As a single hand-held RGB camera flies over a static scene, we estimate detailed textured depth maps at selected keyframes to produce a surface patchwork with millions of vertices. We use the hundreds of images available in a video stream to improve the quality of a simple photometric data term, and minimise a global spatially regularised energy functional in a novel non-convex optimisation framework. Interleaved, we track the camera 6DOF motion precisely by frame-rate whole image alignment against the entire dense model."

ISMAR 2011 - Tracking Contest Impressions

Some impressions of the Metaio team during the tracking competition at ISMAR 2011 in Basel. Finally they got the best score and won the contest. The contestants had to go through a parcour with various challenges for their tracking system.

The goal was to determine exact positions in the real world from single 3D world coordinates. The parcour contained relatively complex environments, like a car or specular setups with mirrors.

Metaio used their Juneio system on a Samsung Galaxy S with Android.

via Junaio Blog

Javascript based Natural Feature Tracking enables markerless AR in the browser

Coming back from ISMAR 2011 in Basel, I could see a very interesting project by developed by Christoph Oberhofer, Jens Grubert, Gerhard Reitmayr (ICG Graz University of Technology - Austria). It proofs that a reliable and performant natural feature based tracking can actually be implemented in Javascript, HTML5 and WebGL.

There are still a lot of issues especially with the access to live video, but results are already impressive. One has to imagine that this runs entirely on a single website.

The demo runs also on an Android phone with Firefox Beta. Of course not as fluide as on a PC, but still with reasonable interactive framerates. Chrome and Opera alpha were used for the desktop demo. Interestingly it was not running on Firefox on PC.

This work is a very good example of the pace of innovation regarding cutting edge augmented reality applications. There is no need for additional plugins, like FLARToolkit. See my post on that. It runs directly in your browser. It is clearly another possibility to bring augmented reality closer to the a broader public.

We can only imagine what will be possible, when the browser support of HTML5, WebGL and Javascript will be significantly improved and technologies like WebCL will come in play.

Great work