Researchers in Greece have developed a new system that converts video into virtual, touchable maps for the blind. The three-dimensional maps use force fields to represent walls and roads so the visually impaired can better understand the layout of buildings and cities.
"Imagine I'm blind and I want to come to New York," says Konstantinos Moustakas, lead researcher on the virtual mapping project and a graduate student at Aristotle University of Thessaloníki in Greece. "I should have a map."
Architects sometimes create three-dimensional models for the blind, but these replicas can only be used by one person at a time. Paper maps with ridges signifying roads are not ideal either, because they cannot convey enough information. With Moustakas' system, a digital version of a diorama can be accessed simultaneously by people around the world. Extra information is presented in audio clips.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
To build the virtual dioramas, the researchers first shoot video of an architectural model. The video is then processed frame by frame using software developed by Moustakas' team. As the camera angle changes, the software tracks each structure and determines its shape and location. That data is used to create a three-dimensional grid of force fields for each structure. "Each point on the grid has an associated force value," Moustakas says.
Two common-touch interfaces simulate the force fields by applying pressure to the user's hand: the CyberGrasp glove, which pulls on individual fingers, and the Phantom Desktop, which applies a single force to the hand via a wand. Moustakas said the process is somewhat like trying to identify an object by running a finger or wand along its surface.
Virtual, touchable maps, also known as haptic maps, have been created before, but they were made using stereoscopic movies, which require special cameras. Moustakas' system works with a standard video camera.
Moustakas also developed a system that converts pictures of traditional paper maps into a three-dimensional street map. Users run a finger or wand down the grooved roads of the virtual map, while street names are automatically read aloud.
Moustakas tested both systems on 19 visually impaired people. During the tests, subjects were asked to identify buildings in the virtual scene and travel from one location to another.
According to the study, published in the journal IEEE MultiMedia, the subjects preferred the virtual street maps for navigating large areas, such as cities, and the virtual dioramas for assessing small groups of buildings. Moustakas is currently working on integrating the two systems.
Reginald Golledge, a professor of geography at the University of California, Santa Barbara, says he believes Moustakas' research is "a good step in the right direction." Golledge, who has conducted research on other virtual mapping systems for more than 15 years, notes that blind users would still need a guide dog or cane to navigate potholes in the real world.
Dan Jacobson, co-chair of the International Cartographic Association's commission on maps and graphics for the visually impaired, says Moustakas' technology could be useful for the sighted as well as the blind. A haptic map could be helpful in situations where a sighted user is visually distracted, for example. It could also convey information about things that are not in view. "In a virtual world … you could feel your way around a building to see what's behind," Jacobson says.
Regardless of the intended user, Golledge says the system will have to become more portable to be widely accepted. The Phantom Desktop, for example, has to be plugged into an outlet. Although it could be handy for travel planning, it could not be used en route.