Determinantal point process attention over grid cell code supports out of distribution generalization.

Area of Science:

• Computational Neuroscience
• Artificial Intelligence
• Cognitive Science

Background:

• Deep neural networks excel at human-like intelligence but struggle with out-of-distribution (OOD) generalization.
• Human generalization abilities, particularly OOD, remain a challenge for current artificial intelligence models.
• Understanding brain mechanisms for generalization can inform AI advancements.

Purpose of the Study:

• To identify brain processing properties contributing to human OOD generalization.
• To develop a novel algorithm for achieving strong OOD generalization in artificial neural networks.
• To provide insights into the role of grid cell codes in generalization.

Main Methods:

• Developed a two-part algorithm leveraging mammalian brain computation features.
• Incorporated grid cell code representations of metric spaces.
• Implemented an attentional mechanism using determinantal point process (DPP) called DPP attention (DPP-A) for sparse coverage.
• Combined task-optimized error with DPP-A in a novel loss function.

Main Results:

• The algorithm achieved strong OOD generalization performance on analogy and arithmetic tasks.
• The proposed method successfully exploited recurring motifs in grid cell codes.
• Integration with common neural network architectures demonstrated effectiveness.

Conclusions:

• Grid cell codes in the mammalian brain may significantly contribute to generalization performance.
• The DPP-A mechanism offers a potential method for improving OOD generalization in artificial neural networks.
• This work bridges computational neuroscience and AI by offering a biologically inspired approach to generalization.