Related papers: Floating-Point Arithmetic on Round-to-Nearest Repr…
Binary embedding is a nonlinear dimension reduction methodology where high dimensional data are embedded into the Hamming cube while preserving the structure of the original space. Specifically, for an arbitrary $N$ distinct points in…
In this paper we investigate encoding the bit-stream resulting from coarse Sigma-Delta quantization of finite frame expansions (i.e., overdetermined representations) of vectors. We show that for a wide range of finite-frames, including…
A simple method to generate a two-dimensional binary grid pattern, which allows for absolute and accurate self-location in a finite planar region, is proposed. The pattern encodes position information in a local way so that reading a small…
We develop new approximation algorithms and data structures for representing and computing with multivariate functions using the functional tensor-train (FT), a continuous extension of the tensor-train (TT) decomposition. The FT represents…
Quantization based model compression serves as high performing and fast approach for inference that yields models which are highly compressed when compared to their full-precision floating point counterparts. The most extreme quantization…
We illustrate the power of partial-encapsulate, showing how it is used in the implementation of floating-point operations in ACL2.
In basic computational physics classes, students often raise the question of how to compute a number that exceeds the numerical limit of the machine. While technique of avoiding overflow/underflow has practical application in the electrical…
The novel concept of entanglement renormalization and its corresponding tensor network renormalization technique have been highly successful in developing a controlled real space renormalization group (RG) scheme. Numerically approximate…
We summarize recent progress in the understanding of fixed point resolution for conformal field theories. Fixed points in both coset conformal field theories and non-diagonal modular invariants which describe simple current extensions of…
This article traces a brief history of the use of single electron spins to compute. In classical computing schemes, a binary bit is represented by the spin polarization of a single electron confined in a quantum dot. If a weak magnetic…
Rotary Position Embedding (RoPE) is the de facto positional encoding in large language models due to its ability to encode relative positions and support length extrapolation. When adapted to vision transformers, the standard axial…
A stationary random sequence admits under some assumptions a representation as the sum of two others: one of them is a martingale difference sequence, and another is a so-called coboundary. Such a representation can be used for proving some…
Floating-point addition on a finite-precision machine is not associative, so not all mathematically equivalent summations are computationally equivalent. Making this assumption can lead to numerical error in computations. Proper ordering…
Numerical data processing is a key task across different fields of computer technology use. However, even simple summation of values is not precise due to the floating point representation use. This paper presents a practical algorithm for…
Every Transformer architecture dedicates enormous capacity to learning rich representations in semantic embedding space -- yet the rotation manifold acted upon by Rotary Positional Embeddings (RoPE) has been treated as a fixed, hand-crafted…
Given a real dataset and a computation family, we wish to encode and store the dataset in a distributed system so that any computation from the family can be performed by accessing a small number of nodes. In this work, we focus on the…
This paper presents a saddlepoint approximation of the random-coding union bound of Polyanskiy et al. for i.i.d. random coding over discrete memoryless channels. The approximation is single-letter, and can thus be computed efficiently.…
We present a method for randomizing formulas for bilinear computation of matrix products. We consider the implications of such randomization when there are two sources of error: One due to the formula itself only being approximately…
We propose a new complex block floating-point format to reduce implementation complexity. The new format achieves wordlength reduction by sharing an exponent across the block of samples, and uses box encoding for the shared exponent to…
It is often useful to perform integration over learned functions represented by neural networks. However, this integration is usually performed numerically, as analytical integration over learned functions (especially neural networks) is…