Distributed associative memory network with memory refreshing loss.

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Neuroscience

Background:

• Memory augmented neural networks (MANN) struggle with complex relational reasoning due to limitations in single external memory encoding.
• Content-based addressable memory networks often fail to create rich representations for relational reasoning in long temporal sequences.

Purpose of the Study:

• To introduce a novel Distributed Associative Memory (DAM) architecture with Memory Refreshing Loss (MRL) to enhance MANN performance.
• To improve the relational reasoning and memorization capabilities of MANNs for complex tasks.

Main Methods:

• Implemented a DAM architecture using multiple smaller, independently updated associative memory blocks for distributed data representation.
• Introduced MRL to reinforce associations between input data and task objectives by reproducing data from memory.
• Applied the DAM and MRL approach to the Differential Neural Computer (DNC) model.

Main Results:

• The proposed DAM architecture with MRL significantly enhances relational reasoning performance in MANNs.
• Achieved state-of-the-art results on both memorization and relational reasoning tasks when applied to the DNC model.
• Distributed representation and memory refreshing improved the encoding of input data for complex reasoning.

Conclusions:

• The novel DAM architecture with MRL offers a significant advancement in MANN capabilities for complex relational reasoning.
• The findings suggest that distributed memory representations and memory refreshing are crucial for enhancing memorization and reasoning in artificial neural networks.
• This approach provides a promising direction for developing more sophisticated AI systems capable of human-like memory and reasoning.