How close are engineers to achieving this Terminator-like view?
There are a few devices out there that are okay for a 30- or 40-degree field of view, but when you think about what it will take for consumers to be interested, they’re going to want about 60, 70 or more degrees [in the] field of view. You don’t want this thing to be this little box in the center of your field of view. It could get very annoying, very quickly.
At what point does augmented-reality technology become integrated into the body itself?
One of our university labs actually did some work on an implanted neural interface chip—[which interacts with the brain]—that was powered wirelessly. It could be placed subcutaneously [under the skin] and never had to be removed because the power came from a small induction coil that was on the outside of the person’s skin. These things were fractions of an inch apart, and you could power what was designed as a very low-power device. But that’s just nibbling at the edges of it.
In addition to figuring out the display technology needed for wearable computing, what other challenges are there?
There are other issues, particularly if you want the virtual world and the real world to move together. That’s basically a latency problem. The graphics you see on a computer screen and television screen are running in a deeply pipelined design—[there’s a pipeline of continuous images that flow to the screen]. Frames are being processed at 30 or 60 per second to deliver moving images. But what happens when your viewing angle changes, as it would when you’re wearing [an augmented-reality] headset? Now you have to drain the pipeline of the previous images from your previous perspective and start refilling the pipeline with images from your new point of view. When this isn’t done fast enough, you get an effect known as “jutter”. You can see it sometimes in the movies or on TV when a camera moves quickly—the images on screen seem to stutter through a series of visual jumps before landing in the right position.
So, not only do you have to have a new display, you have to invent a new graphics architecture that doesn’t have this latency problem. For example, if I look at one person, I’m collecting data about what I’m seeing. When I turn my head to look at someone or something else, all of the data related to that first person is gone, and I’m gathering data on the new image I’m seeing. One solution is to shut off the current image as soon as the headset detects that the wearer’s head is moving. The display returns when the new angle of view is reached. The problem is that we move our heads so often the user would have to learn to tolerate such a display. Otherwise, they’d be sick in an instant.
Hopefully before the decade is out we will get a solution to the graphics architecture that makes it jutter-free. In some sense I think the rest of the electronics is almost trivial. Building a smartphone-glass computing system for such a headset is really not going to be that difficult. It’s well within what we can do today.
People will argue whether the Google approach has consumer potential or not. I think it’s a far cry from what I think people really, really want.
Are there wearable systems that are delivering what people want?
One of our favorite ones is Looxcie, which is a little clip on video camera. That notion of continuously recording your life is a pretty interesting one. It’s a way to document your day.
What you really want, though, is good face recognition so when you go to that cocktail party or you walk into a bar, a little thing in your ear says, “Hey, that’s Bob Jones, your high school buddy.” You can walk right up to that person and say, “Oh, Bob, hey, how are you doing? So good to see you.” I guarantee you’ll sell a million of those in about a week.
What are some other uses for these portable augmented-reality systems?
You begin to think of using augmented reality with biosensors, something like what the Tricorder XPRIZE is pursuing. Although everybody gets excited when they hear the word “tricorder,” [the scanner device featured in Star Trek that doctors used to instantly diagnose patients, the show’s characters] could have had their clothing instrumented to collect all of their vital signs with something like a smartphone [with an augmented-reality app]. They wouldn’t need to wave some magic wand over somebody to tell them what was wrong. Your Star Trek jumpsuit should have been reporting that information [to your phone] in real time.