The names we give our technologies are often ... aspirational. Automobiles can move on their own—meaning without horses—but they still can't steer themselves reliably. Visiting via telepresence is nothing like being there. Smartphones have very low IQs. Augmented reality (AR) has been in this same hopeful category since the term was coined in the early 1990s. It was meant to describe screen-mediated experiences that add information to one's surroundings rather than replacing those surroundings, as virtual reality does. The problem turns out to be very hard to solve.
For one thing, AR objects need to look real, to stick solidly in place, and to stay there in shared environments (the “localization” problem). Creators need tools to build them, and users need ways to find and interact with them. And all of it needs to happen without blocking out the actual world; otherwise AR feels more like degraded reality. One by one, however, innovators have been removing those roadblocks. An era when we can all use AR for work, learning and entertainment is coming into view:
In 2016 Microsoft introduced HoloLens, a self-contained AR headset, and last November it started selling an improved version to business users for $3,500 per unit. Consumers can now buy a similar headset for $2,300 from the lavishly funded and much hyped start-up Magic Leap. Both devices use see-through lenses called waveguides to create a 3-D effect.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
I tried the Magic Leap One glasses recently. Used indoors, they generate remarkably bright, solid-looking objects, such as animated animals or robots, that register perfectly with their surroundings. For the moment, though, both headsets suffer from a limited field of view: about 50 degrees diagonally. That's less than half the arc visible to the human eye, meaning that not much of your surrounding reality gets augmented.
•Last November, San Francisco–based start-up Ubiquity6 released Display.land, an app that lets users capture, annotate, decorate and share photorealistic 3-D models of real-world places using the cameras on their smartphones. Phone-sensor data and computer-vision techniques allow Ubiquity6 to pin these models to the real world with an accuracy of centimeters.
Anjney Midha, co-founder and CEO of Ubiquity6, says he thinks of Display.land as the grown-up version of Minecraft, Roblox, Fortnite Creative and other virtual worlds that allow kids to author and share their own creations. “The worlds they're building have Lego-like aesthetics, but once you bring in photorealism, you ‘age up,’” Midha says. “Adults also have a desire to express themselves and be creative in immersive ways. They just haven't had an easy-enough tool to do it.”
•Also in November, a start-up called Sturfee in Milpitas, Calif., emerged from stealth mode with a technique that helps to solve the localization problem outdoors. It uses satellite imagery and computer vision to figure out what a smartphone camera is pointing at and to retrieve the right underlying 3-D mesh to anchor shared, persistent AR objects. But even before this was available, producers were going beyond games to create compelling outdoor AR projects. One example is the interactive Museum of the Hidden City, a tour from Walking Cinema that shows visitors to San Francisco powerful visual evidence of the way racist “slum clearance” projects transformed a minority neighborhood.
•Tech Web sites are abuzz with rumors that Apple will place 3-D distance sensors alongside the back-facing cameras of new iPhones in 2020, allowing developers to build more powerful AR apps. The company is also thought to be designing its own AR glasses, with a release date as soon as this year or as late as 2023.
AR has already reached consumers through smartphone-based games such as Pokémon Go and Harry Potter: Wizards Unite, both from Niantic. Such experiences will gain fidelity and become more collaborative—but they will run mostly on phones because AR headsets are still bulky and expensive. “If I want to allow real-time collaboration for people in my warehouse, I'm happy to pay for $2,000-plus hardware,” Midha says. “But consumers are going to be on smartphones for a while.” Which leaves us plenty to aspire to.
