Related papers: Bounds on non-linear errors for variance computati…
Stochastic rounding (SR) offers an alternative to the deterministic IEEE-754 floating-point rounding modes. In some applications such as PDEs, ODEs and neural networks, SR empirically improves the numerical behavior and convergence to…
Recently, stochastic rounding (SR) has been implemented in specialized hardware but most current computing nodes do not yet support this rounding mode. Several works empirically illustrate the benefit of stochastic rounding in various…
The quality of numerical computations can be measured through their forward error, for which finding good error bounds is challenging in general. For several algorithms and using stochastic rounding (SR), probabilistic analysis has been…
Classical probabilistic rounding error analysis is particularly well suited to stochastic rounding (SR), and it yields strong results when dealing with floating-point algorithms that rely heavily on summation. For many numerical linear…
Due to the limited number of bits in floating-point or fixed-point arithmetic, rounding is a necessary step in many computations. Although rounding methods can be tailored for different applications, round-off errors are generally…
Stochastic rounding (SR) is a probabilistic rounding mode that mitigates errors in large-scale numerical computations, especially when prone to stagnation effects. Beyond numerical analysis, SR has shown significant benefits in practical…
Stochastic rounding (SR) is a probabilistic method used to round numbers to floating-point and fixed-point representations. In length $n$ summation, the worst-case error of SR grows as $\sqrt{n}$ with high probability, unlike for standard…
We derive two probabilistic bounds for the relative forward error in the floating point summation of $n$ real numbers, by representing the roundoffs as independent, zero-mean, bounded random variables. The first probabilistic bound is based…
The conventional rounding error analysis provides worst-case bounds with an associated failure probability and ignores the statistical property of the rounding errors. In this paper, we develop a new statistical rounding error analysis for…
Stochastic iterative algorithms, including stochastic gradient descent (SGD) and stochastic gradient Langevin dynamics (SGLD), are widely utilized for optimization and sampling in large-scale and high-dimensional problems in machine…
Stochastic Rounding is a probabilistic rounding mode that is surprisingly effective in large-scale computations and low-precision arithmetic. Its random nature promotes error cancellation rather than error accumulation, resulting in slower…
This paper considers a probabilistic model for floating-point computation in which the roundoff errors are represented by bounded random variables with mean zero. Using this model, a probabilistic bound is derived for the forward error of…
Probabilistic models are proposed for bounding the forward error in the numerically computed inner product (dot product, scalar product) between of two real $n$-vectors. We derive probabilistic perturbation bounds, as well as probabilistic…
Probabilistic rounding error analysis can yield much sharper bounds than classical worst-case theory, but existing results typically rely on zero-mean rounding errors and often leave the confidence parameter implicit. This work revisits…
We consider the problem of online linear regression in the stochastic setting. We derive high probability regret bounds for online ridge regression and the forward algorithm. This enables us to compare online regression algorithms more…
We propose a stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs. Our approach is based on a bi-objective viewpoint of chance-constrained programs that seeks solutions on the…
We analyze the forward error in the floating point summation of real numbers, for computations in low precision or extreme-scale problem dimensions that push the limits of the precision. We present a systematic recurrence for a martingale…
In this paper, we introduce a new stochastic approximation (SA) type algorithm, namely the randomized stochastic gradient (RSG) method, for solving an important class of nonlinear (possibly nonconvex) stochastic programming (SP) problems.…
Large-scale numerical computations make increasing use of low-precision (LP) floating point formats and mixed precision arithmetic, which can be enhanced by the technique of stochastic rounding (SR), that is, rounding an intermediate…
When randomized ensembles such as bagging or random forests are used for binary classification, the prediction error of the ensemble tends to decrease and stabilize as the number of classifiers increases. However, the precise relationship…