有效代码的几何学:速率扭曲权衡如何扭曲生成模型的潜在表示
Leo D'Amato1,2, Gian Luca Lancia2,3, Giovanni Pezzulo2
1Department of Control and Computer Engineering, Polytechnic University of Turin, Turin, Italy.
PLoS computational biology
|May 12, 2025
概括
生成模型由于信息压缩而扭曲潜在的表示,受容量和数据等因素的影响. 发现的关键扭曲是原型化,专业化和正交,塑造模型行为.
科学领域:
- 计算神经科学是一种神经科学.
- 机器学习 机器学习
- 信息理论 信息理论
背景情况:
- 生物利用内部世界模型进行适应性行为,因资源限制而需要信息压缩.
- 认知系统中的信息压缩可能导致隐性表示的扭曲,偏离外部现实.
- 速率扭曲理论为最佳信息压缩提供了一个框架,但没有指定导致的扭曲.
研究的目的:
- 调查速度扭曲权衡如何影响生成模型中的潜在表示.
- 为了识别不同约束下信息压缩产生的特定扭曲.
- 了解通过压缩形成的潜空间的几何性质.
主要方法:
- 使用Beta变量自编码器 (VAE) 作为生成模型.
- 系统地改变模型容量,数据分布和任务目标.
- 分析潜伏的表征来识别模式和扭曲.
主要成果:
- 确定了三个主要的扭曲:原型化,专业化和正交.
- 这些扭曲是信息压缩的特征,适应模型约束.
- 扭曲可以共存,创造各种潜在空间几何形状.
结论:
- 速率扭曲理论的约束在生成模型中塑造了潜在的表示几何.
- 研究结果提供了关于人工和生物系统如何压缩信息的见解.
- 了解这些扭曲对于开发更有效,更准确的内部模型至关重要.
相关概念视频
Choosing Between z and t Distribution
2.7K
The z and the Student t distribution estimate the population mean using the sample mean and standard deviation. However, to decide which distribution to use for a calculation, one needs to determine the sample size, the nature of the distribution, and whether the population standard deviation is known. If the population standard deviation is known and the population is normally distributed, or if the sample size is greater than 30, the z distribution is preferred. The Student t distribution is...
2.7K
Improving Translational Accuracy
2.5K
2.5K
Residuals and Least-Squares Property
7.2K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.2K
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K
Sampling Methods: Overview
248
A sample refers to a smaller subset representative of a larger population. In analytical chemistry, studying or analyzing an entire population is often impractical or impossible. Therefore, samples are used to draw inferences and generalize the whole population. The sampling method selects individuals or items from a population to create a sample. Standard sampling methods include random, judgemental, systematic, stratified, and cluster sampling.
In analytical chemistry, the choice of...
In analytical chemistry, the choice of...
248
Reducing Line Loss
135
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
135


