CSLM: Convertible Short-Term and Long-Term Memory in Differential Neural Computers

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Conventional neural networks struggle with complex sequential learning tasks.
• External memory networks like Differentiable Neural Computers (DNCs) show promise but often suffer from low memory utilization efficiency.

Purpose of the Study:

• To introduce a novel DNC architecture, the Convertible Short-Term and Long-Term Memory DNC (CSLM-DNC).
• To improve memory utilization efficiency and learning performance in sequential tasks by enabling memory conversion.

Main Methods:

• Developed a new memory scheme with convertible short-term and long-term memory components.
• Implemented a learning mechanism for memory conversion based on read/write frequency.
• Evaluated the CSLM-DNC on question answering and copy/repeat copy tasks.

Main Results:

• The CSLM-DNC architecture demonstrated significantly improved memory efficiency.
• The model showed enhanced learning performance on complex sequential tasks compared to conventional DNCs.
• Qualitative and quantitative evaluations confirmed the effectiveness of the proposed memory conversion mechanism.

Conclusions:

• The CSLM-DNC offers a more efficient approach to external memory utilization in neural networks.
• The brain-inspired memory conversion mechanism is key to improving performance on sequential learning.
• This architecture holds potential for advancing AI in complex data processing and memory management.