At this year's WWDC, Apple was excited to announce ARKit 2.
Apple's Senior Vice President of Engineering, Craig Federighi demonstrated a very pithy example of an application of augmented reality (AR) with an app called Measure. He measured the dimensions of an old trunk from his college days, as well as some personal belongings, using his phone.
The measurements were overlaid over the video view, marking the length and width of the truck as you would with a tape measure, but all in a digital space. This is a demonstration of the power of augmented reality (cue guitar solo).
Image credit: Apple
At the intersection of the real world and the digital exists a curious meta-reality blurring the lines of reality. Technology marketers have teased us with augmented reality for a while, but with the increased support from mobile device vendors, we're primed to see a whole lot more.
Wait. What is AR?
Augmented reality is an interactive experience that bridges the real and digital worlds, perceptually enhancing reality with computer-generated information. Simply: it's experiencing digital stuff in a way that feels natural in the real world.
Although similar in name, augmented reality is not be confused with virtual reality, which is an experience that replaces your perception of reality with one of an entirely computer-generated simulation. To use film references: think Iron Man's J.A.R.V.I.S. HUD vs The Matrix.
AR vs VR. Image credit: Marvel Studios/Jayse Hansen | Warner Bros
Visual AR is probably the most impressive expression of AR, but it's also the most technically difficult. Whilst the concept of AR was invented in the 90s, it's taken a while for the computational power needed to be available for domestic consumers. Apps like Google Translate, Pokémon Go, and IKEA Place offer ways to see digital information in the context of the real world.
Louis Rosenberg testing Virtual Fixtures, one of the first augmented reality systems ever developed (1992). Image credit: WCS100
Other kinds of AR are less demanding: auditory and haptic supplement what you hear and feel. Good examples of these are apps like Zombie Run or Apple Maps directions for the Apple Watch.
Why use AR?
It can be difficult to understand the digital world. We've grown up interacting with real things, and it can be cognitively challenging to frame digital concepts in terms we understand. In the digital space, we're not even limited to representations of physical things, we can model things that are impossible to realise physically.
Pokémon Go uses AR to make the fictional characters feel more in the world with you. In addition to geographical features (like gathering at PokéGyms), you can also visualise Pokémon as if they were really present.
Image credit: Niantic/Pokémon Company
IKEA has been using AR for a number of years to help visualise products in your home, and with the release of ARKit, IKEA Place offers an easy and realistic representation of IKEA products that look natural in-situ. Not only does this increase the convenience of buying products for the home, it actively encourages exploring their catalogue to find alternative products that would fit (saving the buy-decide-return loop).
Image credit: IKEA
Museums can use AR to contextualise and enhance the pieces on display with rich media. Pokémon Go nuisances aside, museums such as the Royal Ontario Museum are using AR to reimagine how they're are experienced and make the museum experience available to more people.
When to use AR
As with many new technologies, it's easy to get sucked up in the hype and miss the opportunity to make meaningful experiences. It's a technology that can help people understand intangible concepts.
Things to think about when considering AR:
1. Do you have digital content that is non-intuitive to understand?
2. Can you explain why AR will help users to understand better?
3. Are there more traditional methods of realising this explanation?
Augmented reality isn't for everyone. Visual AR won't help the visually impaired decide on new furniture. Auditory AR won't motivate the hard of hearing to run faster, and so on. But used appropriately, it can be very inclusive: visual AR can help the hard of hearing, or non-native-speakers to understand signs or labels. Auditory AR can help the blind to experience content traditionally only experienced by well-sighted people.
Contextual haptic feedback is perhaps more available to everyone, but doesn't come with a common language of use, and requires training every time (and by extension, won't be 'easy-to-use').
So when designing, we have to be aware of these limitations. If we're designing for an augmented experience, we not only need to inform the user about how to get the best from the technology (use in bright light areas, calibrate before use, etc.), but also why the experience is important to be enhanced.
Another point to note is that your users will need a device that supports AR, not just smartphone users, but relatively modern devices. It's also very processor demanding, and will consume a fair amount of power, and won't be usable for extended periods of time.
With a believable experience, augmented reality can close the mental gap between the abstract and the real, helping us to create real experiences that incorporate digital concepts. As with any design, it's important to understand 'the why' as well as 'the what' of any new technology to help inform us of when it's appropriate to use.