Augmenting the Mind and Body
With real time feedback, scientists are training accident victims to reuse paralyzed limbs and soldiers to stay cool, even under fire
By Mike May, December 19, 2016
I watch a computer screen in Lynne Gauthier’s lab. As an assistant professor of physical medicine and rehabilitation at Ohio State, she searches for ways to treat people after injuries or during illnesses, such as strokes and multiple sclerosis. To help me understand one of her tools, a movement-driven video game called Recovery Rapids, she moves her arm in a rowing motion, and a kayak on the screen moves down a river. It looks easy enough, until I try—and nothing happens.
Although the Microsoft Kinect system locks onto me, putting me in control of the kayak, I’m going nowhere.
“You need to make bigger motions because the software knows you are capable of bigger movements,” Gauthier says. “It pushes you to your max performance.”
When I do, the kayak starts to move. As I try to keep the kayak moving down the river, Gauthier explains that she and her colleagues developed this system to study how video games might provide constraint-induced movement therapy (CI therapy). This therapy can be used in patients who have a weak arm, for example, as often happens after a stroke or as multiple sclerosis progresses. However, CI therapy takes many hours every week—more than a therapist can provide or a patient can afford.
Video games might also slow down the cognitive effects of aging. At the University of California, San Francisco, neuroscientist Adam Gazzaley’s team developed NeuroRacer to test cognitive abilities. In the September 5, 2013, issue of Nature, Gazzaley and his colleagues reported that using NeuroRacer “resulted in performance benefits that extended to untrained cognitive control abilities,” such as improved attention and memory even in people 60–85 years old. Based on results like that, Gazzaley says, “Experience is the gateway to brain plasticity.”
To turn brain plasticity into better performance, Gauthier hopes that her system will be widely used in patients’ homes. Her team is running a randomized controlled trial involving multiple clinics and hundreds of patients to see if this method is as effective as the therapist-driven one. Their work represents one of several intriguing examples of how data from human performance is being used to create new methods for augmenting that performance.
At the Air Force Research Laboratory at Wright-Patterson Air Force Base in Dayton, Ohio, I observe another approach to augmentation. Here, a military operator focuses intently on screens that show the location of two drones being directed to specific targets. A range of devices—including heart rate–variability monitors, eye-movement detectors, flexible electronic sensors on the skin, and more—keep track of the operator’s physical and mental status in real time. Meanwhile, a computer takes in all of the data, analyzes it with machine-learning algorithms and generates a 0–100 indicator of the operator’s ability to continue. From this combination of sensing and assessing, scientists there are searching for new ways to augment the operator’s mental performance. “The more that I can personalize the assessment,” explains Scott Galster, chief of the applied neuroscience branch of the AFRL, “the more I can learn what it looks like when the operator fails.” That knowledge of failure enables the lab to create personalized augmentations to boost the mental performance of each individual. And what’s perhaps most exciting is that these personalized forms of augmentation may be adapted to help people in countless other circumstances who also are facing a challenge.
For example, similar combinations of sensors and assessment can be used to benefit athletes. At Ohio State, staff members for the Buckeye football program collect data on players from training sessions, team runs, and practices. “We look at one specific player and compare his data to his own from previous days,” says Phil Matusz, the team’s associate director of strength and conditioning. Matusz and his colleagues also contrast each team member’s data with others in his unit, say, fellow receivers. Players take part in the process as well, making self-assessments for each practice that complement their coaches’ evaluations. “All this goes in our performance logs,” Matusz explains.
Achieving the best results from this strategy depends on how well these elite athletes and military personnel understand the value of the augmentation. “We show and educate the athlete on daily reports and show comparisons between performance and preparation leading up to practice,” Matusz says. “We then educate them on why they should take time to prepare prior to practice: hydration, nutrition, soft-tissue work, float tanks, et cetera.”
Although Buckeye football has just begun using aspects of the Sense-Assess-Augment platform, Matusz already sees its value. When asked if and how he thinks it has been working, he quickly replies, “Yes—very well.”
Beyond the Body
The Sense-Assess-Augment framework extends beyond what is real to the virtual. At the Ecole Polytechnique Fédérale de Lausanne in Switzerland, cognitive neuroscientist Olaf Blanke uses virtual reality to treat chronic pain. According to the 2012 National Health Interview Survey conducted by the U.S. Centers for Disease Control and Prevention, about 50 million Americans suffer from ongoing pain. In describing this finding in The Journal of Pain, Richard Nahin—lead epidemiologist for the U.S. National Institutes of Health’s National Center for Complementary and Integrative Health—wrote that individuals with serious pain “were likely to have worse health status, to use more health care, and to suffer from more disability than those with less severe pain.”
Many factors, including the challenges of managing pain medication, drive the need for new treatments like those incorporating virtual reality. By creating an out-of-body experience—an illusion—for someone suffering from chronic pain, Blanke has found a way to influence the pain experience. For example, if a patient’s right hand hurts, that pain decreases when the person sees that hand through virtual reality. In addition, making the virtual limb bigger reduces the pain even more. In March 2016, Blanke and his colleagues reported in The Journal of Pain that: “Our data reveal novel links between pain and self and suggest that embodied virtual bodies are a promising technique for future pain treatments.”
Back in Gauthier’s lab, she explains that although more data remain to be collected and analyzed to validate her video- game therapy, she is already getting good feedback. “I’ve had multiple sclerosis patients use this and tell me that they felt empowered by it,” she says. But she wants even more. As I struggle to pick up a piece of fruit on the screen, she says, “We want to change clinical practice.” It’s clear that her compelling work has the potential to do just that, transforming tomorrow’s guidelines for healthcare practice and rehabilitation.
When I finally grab a cherry in the rehabilitation video game, I realize that the augmentations currently under way have the capacity to improve the lives of elite military forces, top-level athletes, recovering patients, and even me.
Better Brains, Better Bodies was created by Scientific American Custom Media, a division separate from its board of editors, working in partnership with Ohio State University.