Cover Image: December 2011 Scientific American Magazine See Inside

How to See the Invisible

Augumented-reality apps uncover the hidden reality all around you















Share on Tumblr

Everybody’s amazed by touch-screen phones. They’re so thin, so powerful, so beautiful!

But this revolution is just getting under way. Can you imagine what these phones will be like in 20 years? Today’s iPhones and Android phones will seem like the Commodore 64. “Why, when I was your age,” we’ll tell our grandchildren, “phones were a third of an inch thick!”

Then there are the apps. Right now we’re all delighted to do simple things on our phones, like watch videos and play games. But the ingredients in the modern app phone—camera, GPS, compass, accelerometer, gyroscope, Internet connection—make it the perfect device for the next wave of software. Get ready for augmented reality (AR).

That term usually refers to a live-camera view with superimposed informational graphics. The phone becomes a magic looking glass, identifying physical objects in the world around you.

If you’re color-blind like me, then apps like Say Color or Color ID represent a classic example of what augmented reality can do. You hold up the phone to a piece of clothing or a paint swatch—and it tells you by name what color the object is, like dark green or vivid red. You’ve gone to your last party wearing mismatched clothes.

Other apps change what you see. When a reader sent me a link to a YouTube video promoting Word Lens, I wrote back, “Ha-ha, very funny.” It looked so magical, I thought it was fake.

But it’s not. You point the iPhone’s camera at a sign or headline in Spanish. The app magically replaces the original text with an English translation, right there in the video image, in real time—same angle, color, background material, lighting. Somehow the app erases the original text and replaces it with new lettering. (There’s an English-to-Spanish mode, too.)

Some of the most promising AR apps are meant to help you when you’re out and about. Apps like New York Nearest Subway and Metro AR let you look down at the ground and see colorful arrows that show you which subway lines are underneath your feet. Raise the phone perpendicular to the ground, and you’ll see signs for the subway stations—how far away they are and which subway lines they serve.

When you’re in a big city, apps like Layar and Wikitude let you peer through the phone at the world around you. They overlay icons for information of your choice: real estate listings, ATM locations, places with Wikipedia entries, public works of art, and so on. Layar boasts thousands of such overlays.

There are AR apps that show you where the hazards are on golf courses (Golfscape GPS Rangefinder), where you parked your car (Augmented Car Finder), who’s using Twitter in the buildings around you (Tweet360), what houses are for sale near you and for how much (ZipRealty Real Estate), how good and how expensive a restaurant is before you even go inside (Yelp), the names of the stars and constellations over your head (Star Walk, Star Chart), the names and details of the mountains in front of you (Panoramascope, Peaks), what crimes have recently been committed in the neighborhoods around you (SpotCrime), and dozens more.

Several of these apps are not, ahem, paragons of software stability. And many, like Layar, are pointless outside of big cities because there aren’t enough data points to overlay.

As much fun as they are to use, AR apps mean walking through your environment with your eyes on your phone, held at arm’s length—a posture with unfortunate implications for social interaction, serendipitous discovery and avoiding bus traffic.

Furthermore, there’s already been much bemoaning of our society’s decreasing reliance on memory; in the age of Google, nobody needs to learn the presidents, the state capitals or the periodic table. AR apps are only going to make things worse. Next thing you know, AR apps will identify our friends using facial recognition. Can’t you just see it? You’ll be at a party, and someone will come up to you and say, “Hey, how are you—” (consulting the phone) “—David?”



5 Comments

Add Comment
View
  1. 1. k6dgw 04:56 PM 11/18/11

    Thank you David!!! With no color vision, I've spent my life asking others ... wife, friends, salespeople, and [gasp!] other customers ... the color of the shirt I'm holding. Reading TechnoFiles, I immediately downloaded ColorID to my Android and now I can join the 90% of other people holding their phones while they shop, only mine tells me the answer to my continual question, "What color is this." Thanks a bunch.

    Reply | Report Abuse | Link to this
  2. 2. candide 10:30 AM 12/8/11

    A current iPad 2 is about equivalent in processing power to the worlds most powerful computer in 1985, the Cray 2.

    In 1985 nobody would have dared contemplate that a Cray 2 could be the size of a stack of paper and would run 10 hours on batteries.

    The same way now we cannot accurately predict 25 years from now. The interesting part, to me, is how predictions, except for flying cars, have been short or less than the reality.

    Reply | Report Abuse | Link to this
  3. 3. BillR 10:55 AM 12/8/11

    Pretty soon we will have the phone screen imbedded in our glasses or contacts and we will not need to hold the "phone" out at arms length. The phone will probably be imbedded in the earpiece which will be tiny and the contacts will be an enhanced bluetooth device. The cheap phones will look like glasses with everything embedded in the frames.

    Reply | Report Abuse | Link to this
  4. 4. jtdwyer in reply to candide 03:33 PM 12/8/11

    Nearly all of the most impactful technological developments over the past 50+ years are based on very specific discoveries in quantum physics and resulting developments in electronics. Fortunately, we're not only waiting for flying cars but also artificial intelligence - always 'just around the corner'.

    Unfortunately, the pace of these new developments' impact of humanity and society are increasing at a accelerating rate while critical ancient infrastructure is left to disintegrate, unable to support ever increasing populations. Not to promote autocratic central planning, but surely there are critical issues that must be considered other than gee-whiz market potential and short term profits...

    Reply | Report Abuse | Link to this
  5. 5. kyle stanley 09:55 AM 12/9/11

    I agree... wearable and or implanted systems are the next step most likely using subliminal prompts which means you will still need to learn however data retrieval is much quicker.

    Reply | Report Abuse | Link to this
Leave this field empty

Add a Comment

You must sign in or register as a ScientificAmerican.com member to submit a comment.
Click one of the buttons below to register using an existing Social Account.

More from Scientific American

See what we're tweeting about

Scientific American Editors

More »

Free Newsletters


Get the best from Scientific American in your inbox

Solve Innovation Challenges

Powered By: Innocentive

  SA Digital
  SA Digital

Email this Article

How to See the Invisible: Scientific American Magazine

X
Scientific American Magazine

Subscribe Today

Save 66% off the cover price and get a free gift!

Learn More >>

X

Please Log In

Forgot: Password

X

Account Linking

Welcome, . Do you have an existing ScientificAmerican.com account?

Yes, please link my existing account with for quick, secure access.



Forgot Password?

No, I would like to create a new account with my profile information.

Create Account
X

Report Abuse

Are you sure?

X

Institutional Access

It has been identified that the institution you are trying to access this article from has institutional site license access to Scientific American on nature.com. To access this article in its entirety through site license access, click below.

Site license access
X

Error

X

Share this Article

X