Memory-efficient fully coupled filtering approach for observational model building.

Area of Science:

• Computational neuroscience
• Machine learning algorithms
• Optimization techniques

Background:

• Global Extended Kalman Filter (GEKF) offers high performance for neural network training.
• GEKF's primary limitation is its significant computational cost, hindering its application to larger networks.
• Previous attempts to mitigate cost involved weight decoupling, which reduced network accuracy.

Purpose of the Study:

• To develop a novel algorithm that achieves GEKF's accuracy without its computational burden.
• To maintain full synaptic weight connectivity while reducing memory requirements.
• To provide an efficient alternative for training neural networks.

Main Methods:

• An algorithm is proposed that bypasses the construction of the state covariance matrix, the bottleneck in GEKF.
• The method ensures all synaptic weights remain connected.
• Memory requirements are comparable to or less than heuristic decoupling methods.

Main Results:

• The new algorithm successfully emulates the accuracy of GEKF.
• It significantly reduces the computational cost associated with GEKF.
• The memory footprint is comparable to or more efficient than existing decoupling schemes.

Conclusions:

• The proposed algorithm offers a computationally efficient and accurate method for training neural networks.
• It overcomes the limitations of GEKF without sacrificing performance.
• The method is extendable to other derivative-free nonlinear Kalman filters, broadening its applicability.