Stochastic rounding: implementation, error analysis and applications

Area of Science:

• Computer Science
• Numerical Analysis
• Applied Mathematics

Background:

• Finite precision arithmetic poses challenges in scientific computing.
• Traditional rounding methods like round-to-nearest (RN) can introduce significant errors and suffer from stagnation.
• Stochastic rounding (SR), a probabilistic rounding technique, is gaining renewed interest for its potential to mitigate these issues.

Purpose of the Study:

• To survey the mathematical properties, probabilistic error analysis, and implementation of stochastic rounding.
• To highlight the advantages of SR over traditional rounding methods, particularly in computational accuracy.
• To explore the applications of SR in machine learning and the numerical solution of differential equations.

Main Methods:

• Probabilistic error analysis of stochastic rounding.
• Comparison of error bounds between SR and round-to-nearest (RN).
• Implementation strategies for SR in computer arithmetic.

Main Results:

• SR provides a high-probability error bound with a constant factor of $$ for inner product computations, outperforming RN's worst-case bound of $$.
• SR is immune to stagnation, a critical issue where small updates are lost in RN.
• SR demonstrates practical utility in machine learning and numerical differential equation solving.

Conclusions:

• Stochastic rounding offers a robust alternative to traditional rounding methods in floating-point arithmetic.
• Its probabilistic nature and immunity to stagnation lead to improved numerical stability and accuracy.
• SR is a valuable technique for enhancing computations in demanding applications like machine learning and scientific simulations.