Many machines can recognize humans by their fingerprints or facial features. These biometric traits are not the only ones that set individuals apart, however. Each person’s walking gait is unique—and it can serve not only as an identifier but also as an indicator of mood and health. A team of researchers has developed remote sensors that analyze footsteps by measuring tiny floor vibrations. They used this technology to identify specific individuals walking through a building and to test a new method of hands-off wellness monitoring.
The way you walk is “like a fingerprint—it’s like a very unique signature of yourself,” says Hae Young Noh, who initially performed the research as a civil and environmental engineer at Carnegie Mellon University and has since moved to Stanford University. Gait can reveal “who you are, where you are, what kinds of activities you’re doing or even your cognitive state.” If hardware sensors detect a pattern of footsteps, software can analyze them to verify an individual’s identity. Earlier systems have done so with 95 percent accuracy, says Vir Phoha, a professor of electrical engineering and computer science at Syracuse University, who was not involved in the new work.
And walking patterns can provide more than a simple ID, Phoha adds: “There is a lot of information you can learn from a person’s gait—specifically, health-related information.” If somebody starts placing more weight on one side or another, for example, the change in balance might indicate a neurological problem. This information could help doctors monitor seniors and other at-risk people who want to live independently: tracking subjects’ gait could keep tabs on their health without directly impinging on their space.
To measure this data-rich signature, researchers previously had to outfit subjects with wearable devices or have them walk on special mats or altered flooring. But Noh, along with electrical and computer engineer Pei Zhang of Carnegie Mellon and their colleagues, wanted to develop portable footstep sensors that would work remotely. The scientists took advantage of the fact that typical walls and floors pick up even faint vibrations from activity in the space they contain. “We call this ‘structures as sensors,’ where we’re using these big physical structures like buildings and bridges as a sensing system to indirectly monitor humans and surrounding environments,” Noh says.
Sensing vibrations from a mere footstep requires extremely sharp detectors. “To give you an idea of how sensitive our sensors are: when you sit in the chair a meter away, we put the sensor on the ground,” Zhang says, and “we can sense your heartbeat.” Each sensor—a cylindrical device with height of a few centimeters—sits on the floor and can monitor a walker at a distance of up to 20 meters, Noh says. The researchers can distribute such sensors as an array throughout an area where they want to detect footsteps. But first, the team had to “teach” the new system to distinguish these signals from the background of sounds heard in any busy building.
“Fighting the noise is the biggest challenge we have,” Noh says, and addressing it required both hardware and software solutions. On the hardware side, each sensor has an amplifier that automatically changes how much it boosts a footstep vibration. When a vibration seems to be coming from farther away, the amplifier turns it up. As the signal gets stronger and threatens to overwhelm the sensor, the amplifier decreases its sensitivity. Noh likens this process to remotely controlling the volume of a speaker: listeners make it louder when they are farther away in order to hear better, but as they get closer, and the sound becomes too intense, they dial it down.
Once the sensors have picked up a footstep, the software takes over. “We do various signal-processing and machine-learning [techniques] to learn what is the human-related signal versus other noise that we’re not interested in,” Noh says. Like the data from other footstep-detection methods, such as wearables or pressure mats, walking patterns measured with the new sensors can be used to determine an individual’s identity and some kinds of potential health issues. The team has presented its work at several conferences and seminars—most recently in February at the Society for Experimental Mechanics’ International Modal Analysis Conference.
The way the system displays walkers’ behavior live on a computer monitor made one researcher think of a more fantastical device. Eve Schooler, principal engineer and director of emerging Internet of Things networks at Intel, says she was interested in creating a technological version of the “Marauder’s Map”—a magical floor plan in the Harry Potter book and film series that “uses footsteps to visually portray where people are.” In the real world, such a device might track a building’s occupants and other objects in real time. Schooler was not involved in the project but has previously worked with the researchers. Inspired by Schooler’s suggestion, the Carnegie Mellon team made its own iteration, creating a digital display that shows footprints appearing on a floor plan with the appearance of the magical paper map.
The fictional Marauder’s Map only portrayed one location, but the researchers’ portable footstep sensors could be used in any building, Schooler notes. “Some of the algorithms that they’ve developed make the result transferable, which is what’s so interesting,” she says. “You don’t have to do all this calibration to figure out people’s signature across buildings—they have the techniques to do that for you.” Once the experimental system “learns” a person’s signature gait, the sensor array can recognize that individual whether at the office or home. Given the devices’ affordability—Noh estimates each would cost about $10 to $20 to produce—and the fact that they can be placed every 20 meters to create an image of an entire floor—the wide range of applications indicated by Schooler indeed seems possible.
The ability to conduct this kind of monitoring raises obvious privacy concerns, and the researchers suggest their technology should be used only for consensual health care applications. Such monitoring systems, they note, can help caregivers who need to know when elderly people might be likely to fall. They could also alert or children’s hospitals to symptoms of chronic diseases, such as muscular dystrophy, as early as possible. The developers contend that in such cases, footstep sensors would preserve privacy better than, say, a camera that also captures visual information. “This [was] actually created because of the privacy concerns of the other type of monitoring mechanisms,” Zhang says. And in health-related scenarios, he adds, “I’m willing to trade off a little bit of my data to prevent falls and to detect diseases.”
A version of this article with the title “Step Spy” was adapted for inclusion in the July 2020 issue of Scientific American.