Vocal Joysticks Control Cursors With Sounds
Mouth sounds, rather than whole words or sentences, control vocal joysticks, enabling people without the use of their hands to deftly use computers. Cynthia Graber reports.
SUBSCRIBE TO Science Quickly
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
The internet has changed the world. But the information and resources it provides can be out of reach to those with physical disabilities that affect their hands and arms. Now scientists at the University of Washington are developing a solution using the human voice. It’s called a Vocal Joystick. You might be thinking, okay, the software recognizes speech. So someone says, “scroll down the page,” and the cursor responds. But saying that sentence takes too long—people without physical disabilities navigate the web much faster than that.
Instead of full sentences, vowels move the Vocal Joystick cursor in eight directions. Sounds like ah, ee and oo, k or ch, release the mouse. Raising the pitch of the voice speeds the cursor. The Vocal Joystick has been developed for the internet, for playing video games, even for operating a robotic arm. Developers believe that it could eventually allow people to drive a wheelchair using only sounds. They also plan on incorporating other vocal expressions such as trills and vibrato into later versions. Which eventually could sound something like singing to a computer.
It’s Time to Stand Up for Science
If you enjoyed this article, I’d like to ask for your support. Scientific American has served as an advocate for science and industry for 180 years, and right now may be the most critical moment in that two-century history.
I’ve been a Scientific American subscriber since I was 12 years old, and it helped shape the way I look at the world. SciAm always educates and delights me, and inspires a sense of awe for our vast, beautiful universe. I hope it does that for you, too.
If you subscribe to Scientific American, you help ensure that our coverage is centered on meaningful research and discovery; that we have the resources to report on the decisions that threaten labs across the U.S.; and that we support both budding and working scientists at a time when the value of science itself too often goes unrecognized.
In return, you get essential news, captivating podcasts, brilliant infographics, can't-miss newsletters, must-watch videos, challenging games, and the science world's best writing and reporting. You can even gift someone a subscription.
There has never been a more important time for us to stand up and show why science matters. I hope you’ll support us in that mission.
