JEFFERSON HAN: Han displays his "interface-free" touch-driven computer screen at the 2006 Technology, Entertainment, Design (TED) conference. Image: © JURVETSON, VIA FLICKR
More In This Article
At Perceptive Pixel's offices on Manhattan's West Side, Jefferson Han stands in front of a megasize multi-touch screen and runs his fingertips across the display. Each finger leaves a trail of colored pixels in its wake, causing the display to look, briefly, like it has been scratched by a set of digital claws.
Han, who founded Perceptive Pixel in 2006 and serves as the company's chief scientist, next uses his index finger to draw a loop on the 100-inch display. It causes a menu of options to appear on the screen, not all that different from using a mouse to click on a drop-down menu. Of course, Han doesn't need a mouse, or a keyboard for that matter. He selects one of his menu options, which are arranged in a loop resembling the one he drew to pull up the menu, and away he goes.
Perceptive Pixel's technology is best known for the role it played in helping cable news network CNN create its "Magic Wall," a dynamic, on-air graphical representation of voting results during the network's coverage of the 2008 presidential election. Although Han acknowledges that multi-touch is "interesting" at the level of gadgets such as Apple's iPhone, the technology comes into its own only when you're able to manipulate objects on a display screen using both hands.
Creating software to make multi-touch work properly is the hardest part of developing these systems. "The interface is such a paradigm shift," he says. "You can't just bolt the software onto existing hardware." For this reason, Han and his colleagues have essentially created new operating system for touch-sensitive technology that recognizes an infinite number of touch points—ideally, allowing multiple users to manipulate the same display simultaneously.
We sat down with Han to talk about his approach to multi-touch, how it differs from offerings by consumer electronic device-makers, and where the technology is headed.
[An edited transcript of the interview follows.]
What prompted you to start researching and developing multi-touch sensing technology?
The user's interface with the computer, not the back-end processing horsepower, has become the bottleneck to the user, who accesses information via a straw, in the form of a mouse and keyboard. The input needs more attention, but it's very difficult to impose a new interface on people. Still, as you see computing in more places people realize you can ask for more than just the keyboard. As much as I hate [the movie] Minority Report, it had some impact on people realizing that there's more to computing that simply doing one thing at a time, you can bring your other hand around and use that too. One way to approach a new interface is to ask, "What would a child do with this?"
How does your multi-touch work build on your previous research in the areas of computer graphics, machine learning, real-time computer vision and human-computer interfaces?
My interest is in the visual, that's really why I got into programming and in particular, computer graphics. Graphics has lately made a great shift towards machine learning, which itself is about understanding data. That's where the computer vision work came from. Computer vision is the inverse problem of computer graphics. Instead of telling the computer what the state of the world is, the computer has to figure it out [using e.g. visual cues]. How do you get computers to understand a scene? How do you clean up the sensor information [that the computers are receiving]? Doing this involves using strong engineering and math components.