Neural Networks Need To Rest Just Like Us
We want AI that can help us with pretty much everything. Neural networks are the closest we have gotten to AI so far. And they have been great. But, they aren’t as untiring as we thought.
Image by Sabine Zierer from Pixabay
- Be also sure to check out my other posts and follow me @kralizec and subscribe to my Youtube channel at Kralizec Gaming Youtube Channel
Neural network-based AIs are capable of doing a lot of work especially analyzing large sets of data to find interesting things in them. They can find tumors, diagnose COVID-19, track fast-radio bursts, or even simulate the Universe. And we have always thought that one of the key benefits of AIs is the fact that they don’t tire. They are computers, right?
But, it seems we have made a colossal mistake. Neural networks need to have a rest just like us. On one hand, this is somewhat scary but it also suggests we are heading in the correct direction when it comes to developing true general AIs.
Facts seem to suggest that neural networks need a bit of a holiday from time to time. In between all the face recognition, spam filtering, researching cancer they also need to relax. Computer scientists Yijing Watkins from the Los Alamos National Laboratory (LANL) and her colleagues concluded neural network-based AIs do need to rest from time to time.
If advanced neural networks work for a long time constantly searching for the prescribed patterns in the data sets they can become unstable. But the solution could be as simple as letting them rest. This should bring the whole system back to normal.
But, at the same time, it is not just as easy as shutting down the neural networks. They need to sleep, not “die”. So, Watkins and her team created a specific type of noise that corresponds to the deep sleep state in humans – NREM (No Rapid Eye Movement). This noise is called Gaussian Noise and it includes a set of different wave frequencies and amplitudes.
The researchers believe that their new algorithm that creates the noise for AI can successfully replace the NREM sleep phases of human sleep. This part of sleep renews the biological neurons in our brain and the algorithm should do the same for the electronic neurons of an AI.
Watkins and her team would soon like to test their sleep algorithm on a neuromorphic chip called Loihi from Intel that is trying to imitate the neural structure of the human brain. It includes 130 thousand artificial neurons and is ideal for use in AIs for analyzing videos and other applications that are similar to human senses.
Sources:
- If you like the content I’m producing about science maybe you will like the content I produce about gaming as well! Be sure to check out my other posts!