English
Related papers

Related papers: Empirical Research and Automatic Processing Method…

200 papers

Floating-point arithmetic performance determines the overall performance of important applications, from graphics to AI. Meeting the IEEE-754 specification for floating-point requires that final results of addition, subtraction,…

Mathematical Software · Computer Science 2024-04-02 Lucas M. Dutton , Christopher Kumar Anand , Robert Enenkel , Silvia Melitta Müller

Since many years ago, multilateration has been used in precision engineering notably in machine tool and coordinate measuring machine calibration. This technique needs, first, the use of laser trackers or tracking interferometers, and…

Signal Processing · Electrical Eng. & Systems 2020-03-27 Jean Marc Linares , Santiago Arroyave-Tobón , José Pires , Jean Michel Sprauel

In studying the complexity of iterative processes it is usually assumed that the arithmetic operations of addition, multiplication, and division can be performed in certain constant times. This assumption is invalid if the precision…

Computational Complexity · Computer Science 2021-03-22 Richard P. Brent

Traditional optimization methods rely on the use of single-precision floating point arithmetic, which can be costly in terms of memory size and computing power. However, mixed precision optimization techniques leverage the use of both…

Machine Learning · Computer Science 2023-09-25 Basile Lewandowski , Atli Kosson

Debugging accumulation of floating-point errors is hard; ideally, computer should track it automatically. Here we consider twofold approximation of an exact real with value + error pair of floating-point numbers. Normally, value + error sum…

Numerical Analysis · Computer Science 2014-01-06 Evgeny Latkin

Probabilistic model checking computes probabilities and expected values related to designated behaviours of interest in Markov models. As a formal verification approach, it is applied to critical systems; thus we trust that probabilistic…

Logic in Computer Science · Computer Science 2021-10-19 Arnd Hartmanns

Our goal is to find accurate and efficient algorithms, when they exist, for evaluating rational expressions containing floating point numbers, and for computing matrix factorizations (like LU and the SVD) of matrices with rational…

Numerical Analysis · Mathematics 2025-10-20 James Demmel

The amount of data generated and gathered in scientific simulations and data collection applications is continuously growing, putting mounting pressure on storage and bandwidth concerns. A means of reducing such issues is data compression;…

Numerical Analysis · Mathematics 2025-05-15 Alyson Fox , Peter Lindstrom

In recent years, half precision floating-point arithmetic has gained wide support in hardware and software stack thanks to the advance of artificial intelligence and machine learning applications. Operating at half precision can…

Numerical Analysis · Mathematics 2024-09-19 Longfei Gao , Kevin Harms

We present a scheme to automatically set the precision of floating point variables in an application. We design a framework that profiles applications to measure undesirable numerical behavior at the floating point operation level. We use…

Numerical Analysis · Computer Science 2016-06-02 Ralph Nathan , Helia Naeimi , Daniel J. Sorin , Xiaobai Sun

Mixed-precision computations are a hallmark of the current stage of AI, driving the progress in large language models towards efficient, locally deployable solutions. This article addresses the floating-point computation of…

Machine Learning · Computer Science 2026-05-08 Stanislav Budzinskiy , Marian Gloser , Tolunay Yilmaz , Ying Hong Tham , Yuanyi Lin , Wenyi Fang , Fan Wu , Philipp Petersen

Generating 2-by-2 unitary matrices in floating-precision arithmetic is a delicate task. One way to reduce the accumulation error is to use less floating-point operations to compute each of the entries in the 2-by-2 unitary matrix. This…

Numerical Analysis · Mathematics 2022-11-09 Weslley da Silva Pereira , Ali Lotfi , Julien Langou

We describe an approximate rational arithmetic with round-off errors (both absolute and relative) controlled by the user. The rounding procedure is based on the continued fraction expansion of real numbers. Results of computer experiments…

Numerical Analysis · Mathematics 2025-10-20 Grigori Litvinov , Anatoli Rodionov , Andrei Chourkin

Current critical systems commonly use a lot of floating-point computations, and thus the testing or static analysis of programs containing floating-point operators has become a priority. However, correctly defining the semantics of common…

Programming Languages · Computer Science 2025-10-20 David Monniaux

Floating-point arithmetic plays a central role in science, engineering, and finance by enabling developers to approximate real arithmetic. To address numerical issues in large floating-point applications, developers must identify root…

Programming Languages · Computer Science 2018-07-02 Alex Sanchez-Stern , Pavel Panchekha , Sorin Lerner , Zachary Tatlock

Customizing the precision of data can provide attractive trade-offs between accuracy and hardware resources. We propose a novel form of vector computing aimed at arrays of custom-precision floating point data. We represent these vectors in…

Other Computer Science · Computer Science 2016-02-16 Shixiong Xu , David Gregg

Renewed interest in mixed-precision algorithms has emerged due to growing data capacity and bandwidth concerns, as well as the advancement of GPUs, which enable significant speedup for low precision arithmetic. In light of this, we propose…

Numerical Analysis · Mathematics 2020-12-14 Alec Michael Dunton , Alyson Fox

Verification of programs using floating-point arithmetic is challenging on several accounts. One of the difficulties of reasoning about such programs is due to the peculiarities of floating-point arithmetic: rounding errors, infinities,…

Programming Languages · Computer Science 2022-06-23 Roberto Bagnara , Abramo Bagnara , Fabio Biselli , Michele Chiari , Roberta Gori

We describe a procedure based on the iteration of an initial function by an appropriated operator, acting on continuous functions, in order to get a fixed point. This fixed point will be a calibrated subaction for the doubling map on the…

Dynamical Systems · Mathematics 2020-10-26 Hermes H. Ferreira , Artur O. Lopes , Elismar R. Oliveira

We analyze the forward error in the floating point summation of real numbers, for computations in low precision or extreme-scale problem dimensions that push the limits of the precision. We present a systematic recurrence for a martingale…

Numerical Analysis · Mathematics 2022-03-31 Eric Hallman , Ilse C. F. Ipsen