ADVERTISEMENT

Talking to Computers

A keyboard, monitor and mouse are now standard devices on every personal computer, but that may change¿somewhat¿in the near future
aging



J. W. Stewart
ALTO
Image: XEROX PARC

THE ORIGINAL BEIGE BOX, the ALTO, was the first modern PC, introduced in 1973.

A short black line blinks in front of you on the screen. One hand rests on a keyboard, the other on a mouse. You stand poised to scroll, reload, double click¿perhaps to open a spreadsheet or the newest installment of Tomb Raider. So ubiquitous are the computer monitor, keyboard and mouse that hardly a person alive in the industrialized world cannot relate to that setup. And yet a host of new technologies that promise to make personal computing even easier¿among them voice recognition, handwriting recognition and touch screens¿are emerging.

There is certainly room for improving how we talk to computers. "I don't think anybody is in love with the PC here," says Rickson Sun, director of research and development at IDEO, a product-design consulting firm based in Palo Alto, Calif. "We've spent millions of years evolving to this point, sort of hunting and gathering, moving around, not sitting down in front of screens." But not everyone believes that new technologies will completely replace existing standards. Whereas the mouse may soon be a thing of the past, the traditional keyboard is most likely here to stay.

During the 1960s, the first keyboards and monitors were nothing short of revolutionary. To interact with the earlier computers¿machines such as the Mark1, the ENIAC and the UNIVAC¿users had to rely on punch cards. The original monitors were based on cathode-ray tubes (CRT), a technology borrowed from TV screens. The familiar QWERTY keyboard¿named after the order of the five keys in the top left-hand corner of the keyboard¿hailed from mechanical typewriters. These CRT monitors and QWERTY keyboards were far from user-friendly in today's sense, but before the graphical user interface (GUI) came on to the scene, they faced no competition or complement.

Mouse Evolution
Images: STANFORD RESEARCH INSTITUTE (left), LOGITECH (right)

PROTOTYPE MOUSE, developed by Doug Engelbart in 1963, stands in sharp contrast to today's wireless version.

"[At that time] the mouse would have been absolutely useless because everything worked from the keys of the keyboard," explains Alan Hedge, a professor of ergonomics at Cornell University. "So the advent of the GUI led to this sort of requirement for different kinds of input devices." The first truly modern PC was the ALTO, developed at Xerox's Palo Alto Research Center (PARC) in 1973. This computer, introduced 11 years before the Apple Macintosh, had a GUI and a mouse. The Macintosh took the mouse mainstream, and it became standard on all PCs when IBM-compatible computers switched from MS-DOS, a key-based operating system, to Windows.

What followed was a lot of refinement but few radical changes. The mechanical mouse became optical and wireless, and scroll wheels were added. Manufacturers introduced the trackball, based on the same technology, as an alternative to the mouse on laptop computers. The LCD screen¿a thinner and lighter version of the CRT monitor¿emerged. And some computers incorporated more ergonomical versions of the old QWERTY keyboard.

True innovation was, as usual, born out of necessity when computers shrank further in the form of Personal Digital Assistants (PDAs). Early PDAs looked like mini-laptops, and their lilliputian keyboards were hard to use¿especially if you had to hold the device in one hand as you typed with the other. To get around this problem, PDA designers took on handwriting recognition technologies. The earliest versions failed. But when US Robotics developed their popular PDA, the Palm Pilot, they introduced Graffiti, a shorthand notation that the device could read more easily than actual cursive.

"The advantage of Graffiti was they could do it with a small processor, so they could get it on the market cheap," says Terry Winograd, a professor of computer science at Stanford University. Graffiti required users to learn the computer's writing (not the other way around, as originally planned), but it had clear advantages over the keyboard on tiny devices. Today virtually all PDAs use some form of it.

Cellular phones now face the same miniaturization dilemma¿especially as they try to merge with PDAs. In lieu of handwriting recognition technology, many phones employ voice recognition, also in its infancy but with much potential. "My intuition is that it [voice recognition] is going to hit more as the power to do voice processing becomes doable on small machines, which it really isn't yet," Winograd says. "You don't want a little keyboard on your cell phone because it's horrible to try to type on a little keyboard and you're already using a voice device, so for short things you'd rather say them."

Cellular phone-PDA convergence
Image: MAGCOM

THE CONVERGENCE of the cell phone and the PDA as seen by Magcom. The device on the far right is already on the market.

But voice control has its limitations: "The problem with auditory information is you have to keep the information really simple because it's a stream of information that the brain is processing serially," Hedge says. "So you really couldn't stand a computer that would be speaking to you at the rate that we put visual information on the computer screen." He also sees shortcomings with handwriting recognition. "Any notion that we can replace keyboards with pen-based computing is wildly misguided. The fastest you can go with a pen is about a third the speed that you can type. It's a single channel output compared to multiple channel outputs when you type, and that's the same limitation you have with voice as well. Those technologies just have a basic fundamental human limitation that usually gets overlooked."

"You can do a lot with 10 fingers," Winograd adds. "Your bandwidth out of 10 fingers working at once is larger than any other output device you have, which music is a great example for. I don't think the keyboard will go away, but many of the tasks don't need it, and it will go away for those." And he doesn't expect a profound redesign of the keyboard because any improvements would have to warrant learning a new layout. "Why aren't we using metric in this country? I don't think anybody would disagree that metric is better," he says. "There's not a lot of leverage unless you do things like a one-handed keyboard."


Image: Courtesy of ALAN HEDGE

TOUCH SURFACES are one way in which the keyboard and other input devices may merge. Shown here is a working prototype.

For the mouse, it's a different story. "The mouse is the fax machine of this decade," Hedge remarks. "By the end of the decade it will be relatively scarce. You're likely to see a lot of the input technologies converging on one form of technology." The result may be a form of touch pad. "The surface itself acts as both a keyboard and as an input device; anywhere on the surface is active for you to move the cursor around," Hedge notes. "But also it has the advantage of allowing you to perform a variety of new gestural inputs that take care of operations that used to require a series of mouse clicks. For example, the simple act of pulling your thumb and finger together over a surface in a sort of nipping gesture to cut a piece of text out. So the whole thing becomes a very different kind of working."

Working prototypes of such touch pads exist and are expected to enter the market in the fall. The next step may be integrating a lightweight display with this touch surface technology, uniting all three original components. "The displays are headed to areas where they're thinner, lighter; they're like pieces of paper," Sun says, referring to e-ink, a new display technology under development at several companies. "You'll have very high resolution, so you can have lots of small type on a small piece of paper. You'll have a stack of papers and they can update easily. Paper is thin, flexible and lightweight."

Glasses as a display
Image: IDE

THE NEXT STEP? Display glasses could be part of the future of mobile computing.

Display glasses, too, are already a reality, though often as part of wearable computers, which themselves are rare. According to Winograd, they probably won't become popular in the short term. "You don't want to wear special glasses," he says. "You'll be carrying your cell phone anyway. If you're talking long term, your cell phone will be replaced by something like a hearing aid. If everything else goes away and the only reason you're still carrying something is to look at it, then it makes sense to mount it on your face."

So is the end of the current PC near? "The standard screen plus keyboard plus pointing device will die out," Winograd concludes, but adds, "I think you're still going to want to have some times when you sit down, use your full attention and maximize throughput. Not that the current screen and keyboards are optimal, but it'll be approximately that configuration. There are some kinds of things for which workstations are very highly optimized, and you will probably never find anything that will be much better.

Rights & Permissions
Share this Article:

Comments

You must sign in or register as a ScientificAmerican.com member to submit a comment.
Scientific American MIND iPad

Give a Gift & Get a Gift - Free!

Give a 1 year subscription as low as $14.99

Subscribe Now >>

X

Email this Article

X