深度预测编码与双向传播用于分类和重建
Senhui Qiu1, Saugat Bhattacharyya1, Damien Coyle2
1Intelligent Systems Research Centre, School of Computing, Engineering and Intelligent Systems, Ulster University, Londonderry, BT48 7JL, UK.
概括
深度双向预测编码 (DBPC) 使神经网络能够高效地执行分类和重建任务. 这种新的学习算法在较小的网络和并行学习中实现了高精度,超过了现有的方法.
科学领域:
- 计算神经科学是一种神经科学.
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 预测编码 (PC) 是一种大脑信息处理理论,其中层预测先前层的活动,用于本地错误计算和并行学习.
- 现有的PC方法提供了基本的学习原则,但可以用于同时执行任务的增强.
研究的目的:
- 引入深度双向预测编码 (DBPC) 作为神经网络的新型学习算法.
- 为了使同时分类和重建任务使用单一一组学习的权重.
- 通过本地信息利用和跨网络层并行培训来提高学习效率.
主要方法:
- DBPC通过让每个层预测前一层和下一层的活动来训练网络,从而促进前和反传播.
- 该算法支持完全连接和卷积神经网络的训练.
- 学习依赖于本地可用的信息,使所有网络层实现并行计算.
主要成果:
- 在MNIST (99.58%),时尚-MNIST (92.42%) 和CIFAR-10 (74.29%) 上,DBPC实现了高分类准确度,超过了已建立的PC基准.
- 在多个数据集上,性能与最先进的错误反向传播方法具有竞争力.
- 与基准数据相比,DBPC使用的网络要小得多,同时可以从所有学习的表示中进行输入重建.
结论:
- 通过利用本地信息和并行学习机制,DBPC提供了一个高效的培训协议.
- 该算法有效地同时执行分类和重建任务.
- DBPC提出了一种更有效的方法来训练多功能神经网络.
相关概念视频
01:23Propagation of Action Potentials
7.0K
The propagation of an action potential refers to the process by which a nerve impulse, or "action potential," travels along a neuron.
Neurons (nerve cells) have a resting membrane potential, with a slightly negative charge inside compared to outside. This is maintained by ion channels, such as sodium (Na+) and potassium (K+) channels, which control the flow of ions. When a stimulus, like a touch or a signal from another neuron, triggers the neuron, sodium channels open, allowing sodium ions to...
Neurons (nerve cells) have a resting membrane potential, with a slightly negative charge inside compared to outside. This is maintained by ion channels, such as sodium (Na+) and potassium (K+) channels, which control the flow of ions. When a stimulus, like a touch or a signal from another neuron, triggers the neuron, sodium channels open, allowing sodium ions to...
7.0K
01:30Classification of Signals
908
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
908
01:25Neural Circuits
1.6K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.6K
01:10Reconstruction of Signal using Interpolation
351
Signal processing techniques are essential for accurately converting continuous signals to digital formats and vice versa. When a continuous signal is sampled with a period T, the resulting sampled signal exhibits replicas of the original spectrum in the frequency domain, spaced at intervals equal to the sampling frequency. To handle this sampled signal, a zero-order hold method can be applied, which creates a piecewise constant signal by retaining each sample's value until the next...
351
01:29Aggregates Classification
387
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
387
01:31Classification of Systems-II
242
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
242