Related papers: Effects of number of digits in large-scale multila…
Significant inaccuracy often occurs during the process of mathematical calculation due to the digit limitation of floating point, which may lead to catastrophic loss. Normally, people believe that adjustment of floating-point precision is…
The study addresses the problem of precision in floating-point (FP) computations. A method for estimating the errors which affect intermediate and final results is proposed and a summary of many software simulations is discussed. The basic…
All but a few digital computers used for scientific computations have supported floating-point and digital arithmetic of rather limited numerical precision. The underlying assumptions were that the systems being studied were basically…
In recent years, half precision floating-point arithmetic has gained wide support in hardware and software stack thanks to the advance of artificial intelligence and machine learning applications. Operating at half precision can…
Deep neural networks have enabled progress in a wide variety of applications. Growing the size of the neural network typically results in improved accuracy. As model sizes grow, the memory and compute requirements for training these models…
Support for arithmetic in multiple precisions and number formats is becoming increasingly common in emerging high-performance architectures. From a computational scientist's perspective, our goal is to determine how and where we can safely…
Finite precision computations using digital computers involve the following inherent errors: (1) Round-off error of finite precision computations (2) Binary computer arithmetic precludes exact number representation of traditional decimal…
Mixed-precision computing has become increasingly important in modern high-performance computing and machine learning applications. When implementing custom mixed-precision functions -- such as fused operators, optimized GPU kernels, or…
Floating-point computations are quickly finding their way in the design of safety- and mission-critical systems, despite the fact that designing floating-point algorithms is significantly more difficult than designing integer algorithms.…
For scientific computations on a digital computer the set of real number is usually approximated by a finite set F of "floating-point" numbers. We compare the numerical accuracy possible with difference choices of F having approximately the…
Forthcoming large-scale surveys will soon attempt to measure cosmic shear to an unprecedented level of accuracy, requiring a similarly high level of accuracy in the shear measurements of galaxies. Factors such as pixelisation, imperfect…
Quantum steering, as a manifestation of nonlocal quantum correlations, plays a crucial role in enabling various quantum information processing tasks. However, practical implementations are often hindered by significant challenges arising…
A new deterministic floating-point arithmetic called precision arithmetic is developed to track precision for arithmetic calculations. It uses a novel rounding scheme to avoid excessive rounding error propagation of conventional…
The typical processors used for scientific computing have fixed-width data-paths. This implies that mathematical libraries were specifically developed to target each of these fixed precisions (binary16, binary32, binary64). However, to…
Many algorithms feature an iterative loop that converges to the result of interest. The numerical operations in such algorithms are generally implemented using finite-precision arithmetic, either fixed- or floating-point, most of which…
This paper presents the analysis of the impact of a floating-point number precision reduction on the quality of text classification. The precision reduction of the vectors representing the data (e.g. TF-IDF representation in our case)…
The vast use of computers on scientific numerical computation makes the awareness of the limited precision that these machines are able to provide us an essential matter. A limited and insufficient precision allied to the truncation and…
In this paper, we investigate the impact of higher-order distortions on the precise measurement of weak gravitational lensing shear and flexion. We begin by defining generalized higher-order distortions and outlining methods for measuring…
In basic computational physics classes, students often raise the question of how to compute a number that exceeds the numerical limit of the machine. While technique of avoiding overflow/underflow has practical application in the electrical…
Reduced numerical precision is a common technique to reduce computational cost in many Deep Neural Networks (DNNs). While it has been observed that DNNs are resilient to small errors and noise, no general result exists that is capable of…