Information theoretical approach to the storage capacity of neural networks with binary weights

H Suyari1, I Matsuba

  • 1Department of Information and Image Sciences, Faculty of Engineering, Chiba University 1-33, Yayoi-cho, Inage-ku, Chiba-shi, Chiba 263-8522 Japan.

Related Concept Videos

Storage01:23

Storage

A schema is a mental framework that helps individuals organize and interpret information. Schemata, formed from previous experiences, influence how we process new information: how we encode it, the inferences we make, and how we retrieve it. For instance, a schema for what a typical classroom looks like might include desks, a teacher's desk, a whiteboard, and students in such an environment. This expectation helps us quickly understand and navigate new classrooms without needing to analyze each...
Cognitive Neuroscience Approach01:21

Cognitive Neuroscience Approach

The cognitive neuroscience approach to development focuses on how the central nervous system supports emerging cognitive abilities. Neuroimaging and behavioral studies link specific neural systems to memory and cognitive processing in infancy.Distinct Memory Systems in the Developing BrainInfant long-term memory includes two distinct systems: implicit and explicit memory. Implicit, or nondeclarative, memory involves unconscious recall of skills and routines. It develops early and appears in...
Information Processing Approach01:30

Information Processing Approach

The information-processing theory of cognitive development centers on fundamental mental processes, including attention, memory, and problem-solving skills. Researchers in this field examine how cognitive abilities, such as working memory, evolve and influence children's overall development. Studies indicate that children with stronger working memory tend to excel in reading comprehension, math, and problem-solving compared to peers with less efficient memory skills. Low working memory is also...
Neuroplasticity01:01

Neuroplasticity

Neuroplasticity reflects the brain's remarkable capacity to adapt and evolve, responding dynamically to learning, experiences, or injury by reorganizing its neural circuitry. This reorganization involves creating new neural connections and refining old ones through a series of biological processes that contribute to the brain's lifelong development and adaptability.
Understanding Memory01:19

Understanding Memory

Memory is the retention of information or experiences over time, facilitated through three main processes: encoding, storage, and retrieval. Encoding is the process of inputting information into the memory system. For instance, when listening to a lecture, watching a play, reading a book, or having a conversation, the brain is actively encoding information. This initial stage involves transforming sensory input into a form that can be processed and stored by the brain. Various factors, such as...
Chunking01:12

Chunking

Chunking is a powerful cognitive technique that improves short-term memory retention by organizing information into smaller, more manageable units. The brain, limited by working memory capacity, can more easily process and store information when it is divided into "chunks" rather than presented as discrete, unrelated elements. Chunking is especially useful when dealing with large amounts of information, such as numerical sequences, words, or complex ideas.
The principle behind chunking is...