Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations

Area of Science:

• Computational neuroscience
• Numerical analysis
• High-performance computing

Background:

• Double-precision floating-point arithmetic is standard in high-performance computing but demands significant energy and memory.
• Reduced-precision arithmetic offers potential gains in efficiency but often introduces numerical errors.
• Accurate numerical solutions are critical in fields like neuroscience for modeling complex systems.

Purpose of the Study:

• To investigate methods for enhancing the accuracy of reduced-precision fixed-point arithmetic.
• To evaluate these methods in the context of solving ordinary differential equations (ODEs) for neuroscience applications.
• To explore computationally cheaper alternatives inspired by digital signal processing techniques.

Main Methods:

• Utilizing the Izhikevich neuron model to simulate neuronal behavior and test ODE solvers.
• Comparing fixed-point arithmetic with stochastic rounding against single-precision floating-point and round-to-nearest fixed-point methods.
• Investigating a dither-inspired technique for improved resolution in fixed-point arithmetic.

Main Results:

• Fixed-point arithmetic with stochastic rounding demonstrated superior accuracy in spike timing compared to single-precision floating-point and round-to-nearest fixed-point arithmetic.
• The observed improvements held across various neuron behaviors and ordinary differential equation solvers.
• A computationally inexpensive dither-based method also showed promise for enhancing accuracy.

Conclusions:

• Stochastic rounding is a key factor in achieving accurate results with reduced-precision fixed-point arithmetic for ODE solutions.
• These findings have broad implications for solving ODEs in neuroscience and other scientific domains, including those involving partial differential equations.
• Reduced-precision arithmetic, when carefully implemented, can provide an energy-efficient and accurate alternative for scientific computation.