What is Augmented Reality?

Augmented reality is using technology to superimpose information on the world we see. For example, images and sounds are superimposed over what the user sees and hears. Picture the "Minority Report" or "Iron Man" style of interactivity. 

This is rather different from virtual reality. Virtual reality means computer-generated environments for you to interact with, and being immersed in. Augmented reality (also known as AR), adds to the reality you would ordinarily see rather than replacing it. 

Augmented reality is often presented as a kind of futuristic technology, but it's been around in some form for years, if your definition is loose. For example, the heads-up displays in many fighter aircraft as far back as the 1990s would show information about the attitude, direction and speed of the plane, and only a few years later they could show which objects in the field of view were targets. 

In the last several years various labs and companies have tried to build devices that give us augmented reality. In 2009, the MIT Media Lab’s Fluid Interfaces Group presented SixthSense, a device that combined the use of a camera, small projector, smartphone and mirror. The device hangs from the user’s chest in a lanyard fashion from the neck. Four sensor devices on the user's fingers can be used to manipulate the images projected by SixthSense. 

Google rolled out Google Glass in 2013, moving augmented reality to a more wearable interface, in this case glasses. It displays on the user’s lens screen via a small projector and responds to voice commands, overlaying images, videos and sounds onto the screen. Google pulled Google Glass at the end of December 2015 but plans to make a new version. 

As it happens, phones and tablets might be the way augmented reality gets into most people's lives, at least at first. Vito Technology's Star Walk app, for instance, allows a user to point the camera in their tablet or phone at the sky and see the names of stars and planets superimposed on the image. Another app called Layar uses the smartphone’s GPS and its camera to collect information about the user’s surroundings. It then displays information about nearby restaurants, stores and points of interest. 

Some apps for tablets and phones work with other objects as well. Disney Research developed an AR coloring book, in which you color in a character in a conventional (though app-compatible) book and launch the app on the device. The app accesses the camera and uses it to detect which character you are coloring, and uses software to re-create the character in 3D character on the screen. 

The future of augmented reality

This doesn't mean that phones and tablets will be the only venue for AR. For example there's work being done on augmented reality earbuds, which allow you to adjust the sounds that come in from your surroundings. Research continues apace on including AR functionality in contact lenses and other wearable devices that would operate by themselves. The ultimate goal of augmented reality is to create a convenient and natural immersion, so there's a sense that phones and tablets will get replaced, though it isn't clear what those replacements will be. Even glasses might take on a new form, as "smart glasses" are developed for blind people. 

Like any new technology AR has a lot of political and ethical issues. Google Glass, for example, raised a privacy concerns. Some worried that conversations might be surreptitiously recorded or pictures snapped, or thought that they might be identified by face recognition software — though Google officially said it wasn't allowing face recognition apps onto Google Glass' version of an App Store, a startup called Lambda Labs said they were making one anyway.