Gentle Introduction to Tango

Tango has created quite buzz since its initial release. And I must say it is a powerful augmented reality platform and first of its kind for a mobile phone.

What exactly is Tango – Tango is a platform that uses Computer Vision and certain hardware(more info below), that gives the device the ability to understand its position relative or we can say with respect to the world. This platform gives sensing abilities to the mobile phone by telling whether a wall is in front or not, have you been in this room or area before or where exactly you are going. Impressive Right!!

Computer Vision is a method for acquiring, processing, analyzing and understanding digital images and extraction of high-dimensional data from the real world to produce numerical or symbolic information in form of decisions.

Now the question arises if it is so powerful and impressive platform, why don’t mobile devices in the market get this feature enabled in an upcoming update? Well, they can’t as it won’t make your device a Tango ready smartphone, as each tango enabled device must have –

  1. Wide angle camera
  2. Depth sensing camera
  3. Accurate sensor for time-stamping
  4. Software stack to use motion-tracking, area-learning, and depth sensing. (Phew…)

Currently, there are only two devices available with Tango platform enabled.

Asus Zenfone AR
Lenovo Phab 2 Pro

 

 

 

 

 

 

 

 

 

As mentioned above Tango gives mobile devices awareness about the environment by using three core technologies:

Motion Tracking:

Hardware on mobile that enables this technology are – Fish Eye Lens (Wide angle camera) + Inertial Measurement Unit (Time stamping). So what this hardware do that helps in motion tracking. Well, the camera helps detect corners and edges per frame and how they have moved in frames to determine how much it has traveled this is called Feature Tracking. Inertial measurement unit i.e IMU use accelerometer and gyroscope to determine how fast device accelerates and directions are turned.
The images are then fused from camera to IMU sensor data to calculate how much the device is moved, this happens many times per frame how many? MANY!!!

 

Depth Perception:

There is a depth perception camera behind phone which throws infrared (therefore does not work in sunlight). It works great indoors where there are no textures on the wall. Also, it has certain range up to which it works. Depth is calculated with the help of Structured light, Time of flight and Stereo. structures light and time of flight uses IR receiver and IR sensor whereas stereo uses the camera.

 

Area Learning:

OK, we are going to spend some time on this. Imagine you are walking in a room with your tango device putting virtual object at different places with the help of Depth Perception and Motion tracking your devices calculates the position of the virtual object, but suddenly you stumble now when you look at the phone you will find that all the items that you have placed at different locations are not exactly where you left them. This is called Drift.

Area learning solves this issue. It saves the area where you have been in its memory on file called ADF – Area Description File. It stores visual references in this file coupling it with motion tracking increases the accuracy. Its success totally depends on how many reference images tango has and stationary object that does not change over time. Light also affects area learning capability of the device. The best part about area learning is that you can load existing ADF file and match it against the environment that you are currently in.
When Tango device has learned an area there are two things it can do to improve the information provided by motion tracking.

  1. Improve the accuracy of the trajectory by performing ‘Drift corrections’.
  2. Orient and position itself with a previously learned are by performing ‘Localization’.

 

This is just an introduction post of Tango and what exactly it is capable of, but to see it in action we are going to do code in future posts, We will start with very simple Tango APIs and slowly turn up the notch. So stay tuned for the upcoming post on Tango.

 

If you are curious about Tango and want to know about it then leave a comment or email me at contact@nipundavid.com

If you are interested to know more about the work I have done than do the same as above 😀