确定点过程注意力超过网格细胞代码支持分布外的泛化.
Shanka Subhra Mondal1, Steven Frankland2, Taylor W Webb3
1Department of Electrical and Computer Engineering, Princeton University, Princeton, United States.
eLife
|August 1, 2024
概括
这项研究引入了一种由哺乳动物大脑计算启发的新算法,以增强人工神经网络中的分布外 (OOD) 泛化. 该方法利用网格细胞代码和确定点过程注意力 (DPP-A) 来提高复杂任务的性能.
科学领域:
- 计算神经科学是一种神经科学.
- 人工智能的人工智能
- 认知科学 认知科学
背景情况:
- 深度神经网络在类似人类的智能方面表现出色,但在分布外 (OOD) 泛化方面扎.
- 人类的概括能力,特别是OOD,仍然是当前人工智能模型的挑战.
- 了解大脑的概括机制可以为AI的进步提供信息.
研究的目的:
- 识别有助于人类OOD泛化的大脑处理属性.
- 开发一种新的算法,以在人工神经网络中实现强大的OOD泛化.
- 提供对网格单元代码在概括中的作用的见解.
主要方法:
- 开发了一个利用哺乳动物大脑计算特征的两部分算法.
- 嵌入的网格单元代码表示了米数空间.
- 实施了一个使用确定点过程 (DPP) 的注意力机制,称为DPP注意力 (DPP-A),用于稀疏覆盖.
- 结合任务优化错误与DPP-A在一个新的损失函数.
主要成果:
- 该算法在类比和算术任务上实现了强大的OOD泛化性能.
- 提出的方法成功地利用了网格电池代码中反复出现的动机.
- 与常见的神经网络架构的集成证明了有效性.
结论:
- 哺乳动物大脑中的网格细胞代码可能会对概括性能做出重大贡献.
- DPP-A机制提供了一种改善人工神经网络中OOD概括的潜在方法.
- 这项工作将计算神经科学和人工智能联系起来,通过提供一种以生物学为灵感的概括方法.
相关概念视频
Quantifying and Rejecting Outliers: The Grubbs Test
1.5K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
1.5K
Distributions to Estimate Population Parameter
4.1K
The accurate values of population parameters such as population proportion, population mean, and population standard deviation (or variance) are usually unknown. These are fixed values that can only be estimated from the data collected from the samples. The estimates of each of these parameters are sample proportion, the sample mean, and sample standard deviation (or variance). To obtain the values of these sample statistics, data are required that have particular distribution and central...
4.1K
Sampling Distribution
12.3K
Given simple random samples of size n from a given population with a measured characteristic such as mean, proportion, or standard deviation for each sample, the probability distribution of all the measured characteristics is called a sampling distribution. How much the statistic varies from one sample to another is known as the sampling variability of a statistic. You typically measure the sampling variability of a statistic by its standard error. The standard error of the mean is an example...
12.3K
End Point Prediction: Gran Plot
305
A Gran plot is used to predict the equivalence volume or endpoint of a potentiometric or acid-base titration without reaching the endpoint. Typically, titration data is collected as a function of the titrant's volume up to a point less than the equivalence volume and then transformed into a linear format. The straight line is extended to the x-axis, indicating the necessary titrant volume to achieve the equivalence point.
For potentiometric titration, the Gran plot is created by plotting...
For potentiometric titration, the Gran plot is created by plotting...
305
What Are Outliers?
3.7K
Outliers are observed data points that are far from the least squares line. They have unusual values and need to be examined carefully. Though an outlier may result from erroneous data, at other times, it may hold valuable information about the population under study and should be included in the data. Hence, it is crucial to examine what causes a data point to be an outlier.
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
The z score is used to find outliers or unusual values. It should be noted that any values beyond -2 and +2 are...
3.7K
Area Computation by the Alternative Coordinate Method
50
The alternative coordinate method, also known as the Shoelace Formula, is a technique for determining the area of a traverse using Cartesian coordinates. This method relies on the sequential arrangement of x and y coordinates for each point of the shape, ensuring accuracy and ease of application.In this approach, each corner's x and y coordinates are listed as fractions, with the x-coordinate as the numerator and the y-coordinate as the denominator. These coordinates are arranged sequentially...
50


