One of the distinguishing features of machines is that they don’t need to sleep, unlike humans and any other creature with a central nervous system. Someday though, your toaster might need a nap from time to time, as may your car, fridge and anything else that is revolutionized with the advent of practical artificial intelligence technologies.
The change will come when (and if) AI systems that mimic living brains are incorporated into the wide range of devices that currently rely on conventional computers and microprocessors to help us through the day. At least that’s the implication of new research that we are conducting in Los Alamos National Laboratory to understand systems that operate much like the neurons inside living brains.
Our realization came about as we worked to develop neural networks that closely approximate how humans and other biological systems learn to see. We were investigating the way that these simulated networks respond to unsupervised dictionary training. In this sort of activity, networks set about classifying objects without having prior examples with which to compare them. Imagine handing many images of exotic animals to a child, and asking them to group similar ones together. The child might not know what an antelope is, but they would place them in a separate pile from the lions or penguins, for example.
On supporting science journalism
If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
It likely would come as no surprise to any teacher of young children that we found that our networks became unstable after continuous periods of learning. However, when we exposed the networks to states that are analogous to the waves that living brains experience during sleep, stability was restored. It was as though we were giving the neural networks the equivalent of a good, long nap.
This sort of instability is not a characteristic of all AI networks. The issue only arises when training biologically realistic processors, or when trying to understand biology itself. The vast majority of researchers on machine learning, deep learning and AI never encounter this instability because, in the very artificial systems they study, they have the luxury of performing mathematical operations that have no equivalent in living neurons.
Our decision to expose our biologically realistic networks to an artificial analogue of sleep was nearly a last-ditch effort to stabilize them. They were spontaneously generating images that were analogous to hallucinations. We experimented with various types of numerical noise, roughly comparable to the static you might encounter between stations while tuning a radio. The best results came when we used noise with a wide range of frequencies and amplitudes. The noise mimics the input received by the neurons in your brain during slow-wave sleep, which is the deep sleep we can’t live without.
The results suggest that in both artificial and natural intelligence systems slow-wave sleep may act to ensure that neurons maintain their stability and do not hallucinate.
Sleeplike states in neural networks are very different from the mode your PC enters after some set period of inactivity. A conventional computer that has gone to “sleep” is effectively in suspended animation, with all computational activity frozen in time. And the age-old advice from the IT department to try “turning your computer off and then on again” when a PC gets glitchy is tantamount to exposing your machine to a brief period of brain death.
That kind of sleep mode would do nothing to settle an unstable neural network. And power cycling would simply reset the network and undo any prior training, effectively giving the network a severe case of amnesia. In neural networks as well as living creatures, a sleeplike state is not inactivity, but a different kind of activity that is crucial to the proper functioning of neurons.
We are just starting to investigate an additional benefit of artificial sleep in our simulations. Often, a few neurons in a simulated network fail to function at all when a simulation is started. We have found that applying artificial sleep states seems to reset idle neurons to ensure they become functioning components in the network.
As researchers build networks that increasingly resemble living nervous systems, it should probably come as little surprise that they seem to need sleep as much as we do. Similarly, we expect that sophisticated AI systems will help us to more fully understand sleep and other characteristics in biological systems. The napping toaster of the future may provide novel insights into the workings of our brains—in addition to a warm and crispy breakfast food.