Self-organization toward 1/f noise in deep neural networks

Area of Science:

• Computational neuroscience
• Artificial intelligence
• Machine learning

Background:

• Biological neural networks demonstrate 1/f noise patterns, indicative of healthy brain function.
• The presence of 1/f noise in artificial neural networks (ANNs) has not been widely established, despite their advanced cognitive capabilities.

Purpose of the Study:

• To investigate the existence of 1/f noise patterns in ANNs during time series classification tasks.
• To determine the relationship between neuron utilization and the emergence of 1/f noise in ANNs.

Main Methods:

• Training ANNs on time series classification datasets.
• Analyzing the noise patterns in neuron activations.
• Correlating neuron activity with utilization levels.

Main Results:

• 1/f noise patterns were observed in ANNs trained on time series data.
• Neuron activations most closely approximated 1/f noise when neurons were highly utilized.
• Underutilized neurons in large networks showed deviations from 1/f noise towards white noise patterns.

Conclusions:

• ANNs can exhibit 1/f noise, similar to biological neural networks.
• Neuron utilization is a critical factor influencing the presence of 1/f noise in ANNs.
• Network architecture and neuron activity levels impact the deviation from 1/f noise towards white noise.