对层次时间记忆空间聚合器算法的信息理论分析,为标准信息瓶方法设置了一个新的上限
Shiva Sanati1, Modjtaba Rouhani1, Ghosheh Abed Hodtani2
1Department of Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran.
Frontiers in computational neuroscience
|June 23, 2023
概括
层次时间记忆的空间聚合器 (SP) 算法显示了抗噪声和更高稀疏度的性能改善. 最佳的重建是在2%的稀疏度下实现的,证明了其在编码数据方面的有效性.
科学领域:
- 机器学习 机器学习
- 计算神经科学是一种神经科学.
- 信息理论 信息理论
背景情况:
- 层次时间记忆 (Hierarchical Temporal Memory,简称HTM) 是一种不受监督的机器学习算法,其灵感来源于新皮层原理.
- 空间聚合器 (SP) 是一个核心的HTM组件,它从二进制输入流中生成稀疏分布式表示.
- 使用信息理论指标评估SP的散散化对于理解其性能至关重要.
研究的目的:
- 评估层次时间记忆 (HTM) 中空间聚合器 (SP) 算法的散散化效率.
- 引入和使用修改的信息瓶 (IB) 测量来评估在不同稀疏和噪声条件下SP的性能.
- 以数学证明稀疏性和SP算法性能之间的正相关性.
主要方法:
- 使用信息瓶 (IB),克拉默-拉奥下界和费舍尔信息矩阵进行信息理论分析.
- 引入了IB关系的一个新的上限,称为modified-IB.
- 使用MNIST,时尚-MNIST和纽约出租车数据集测试SP算法,包括引入高达40%的噪音.
- 使用概率映射和隐藏马尔科夫模型重建稀疏的SP输出表示.
- 克拉默-拉奥下界与考西分布数据的数学分析.
主要成果:
- SP算法表现出显著的对噪声的抵抗力,高达40%的输入噪声不会导致明显的输出变化.
- 使用修改IB的数值计算表明,较低的噪音和更高的稀疏度水平提高了重建的有效性.
- 该SP算法以2%的稀疏度实现了最佳结果.
- 数学证明证实,增加稀疏性可以提高SP算法性能.
结论:
- 空间聚合器 (SP) 算法表现出强大的性能和抗噪能力,特别是在更高的稀疏度水平.
- 修改后的IB测量为评估不同稀疏度和噪声条件下的SP性能提供了有价值的工具.
- 稀疏性是优化SP算法的效率和在层次时间内存中的表示能力的关键因素.
相关概念视频
01:24Working Memory
387
Working memory refers to a combination of components, including short-term memory and attention, that allow an individual to hold information temporarily as we perform cognitive tasks. It is an essential cognitive function that enables the execution of complex tasks such as problem-solving, comprehension, and reasoning. Unlike short-term memory, which simply involves the storage of information for a brief period, working memory involves the active manipulation and processing of this...
387
01:23Storage
107
A schema is a mental framework that helps individuals organize and interpret information. Schemata, formed from previous experiences, influence how we process new information: how we encode it, the inferences we make, and how we retrieve it. For instance, a schema for what a typical classroom looks like might include desks, a teacher's desk, a whiteboard, and students in such an environment. This expectation helps us quickly understand and navigate new classrooms without needing to analyze...
107
01:22Chunking and Rehearsal in Sensory Memory
250
Improving short-term memory can be achieved through techniques like chunking and rehearsal. Chunking involves organizing information into larger, more manageable units. This technique is particularly useful for information that exceeds the typical memory span of between five and nine items. For instance, logging into an online account with a password like "ta89vq0179gz" involves grouping letters and numbers into three chunks—ta89, vq01, and 79gz. It makes large amounts of...
250
01:12Chunking
148
Chunking is a powerful cognitive technique that improves short-term memory retention by organizing information into smaller, more manageable units. The brain, limited by working memory capacity, can more easily process and store information when it is divided into "chunks" rather than presented as discrete, unrelated elements. Chunking is especially useful when dealing with large amounts of information, such as numerical sequences, words, or complex ideas.
The principle behind chunking...
The principle behind chunking...
148
01:19Role of Hippocampus in Memory
352
The hippocampus, a critical brain structure, plays an essential role in memory processing, particularly in the formation and retrieval of memory. This small, seahorse-shaped region is located within the medial temporal lobe, with one hippocampus in each brain hemisphere. Experimental studies involving lesions in the hippocampi of rats have demonstrated significant impairments in tasks such as object recognition and maze navigation, indicating the hippocampus involvement in both recognition and...
352