Related papers: On Precision - Redundancy Relation in the Design o…
Let $x \in \R$ be given. As we know the, amount of bits needed to binary code $x$ with given accuracy ($h \in \R$) is approximately $ \m_{h}(x) \approx \log_{2}(\max {1, |\frac{x}{h}|}). $ We consider the problem where we should translate…
The persistent storage of big data requires advanced error correction schemes. The classical approach is to use error correcting codes (ECCs). This work studies an alternative approach, which uses the redundancy inherent in data itself for…
Inexact computing aims to compute good solutions that require considerably less resource -- typically energy -- compared to computing exact solutions. While inexactness is motivated by concerns derived from technology scaling and Moore's…
This paper deals with the problem of universal lossless coding on a countable infinite alphabet. It focuses on some classes of sources defined by an envelope condition on the marginal distribution, namely exponentially decreasing envelope…
Adaptive coding faces the following problem: given a collection of source classes such that each class in the collection has non-trivial minimax redundancy rate, can we design a single code which is asymptotically minimax over each class in…
It is known that for memoryless sources, the average and maximal redundancy of fixed-to-variable length codes, such as the Shannon and Huffman codes, exhibit two modes of behavior for long blocks. It either converges to a limit or it has an…
Numerical software, common in scientific computing or embedded systems, inevitably uses an approximation of the real arithmetic in which most algorithms are designed. In many domains, roundoff errors are not the only source of inaccuracy…
Function-correcting codes, introduced by Lenz, Bitar, Wachter-Zeh, and Yaakobi, protect specific function values of a message rather than the entire message. A central challenge is determining the optimal redundancy -- the minimum…
This work proves new results on the ability of binary Reed-Muller codes to decode from random errors and erasures. We obtain these results by proving improved bounds on the weight distribution of Reed-Muller codes of high degrees.…
Machine learning (ML) is probably the first and foremost used technique to deal with the size and complexity of the new generation of data. In this paper, we analyze one of the means to increase the performances of ML algorithms which is…
This paper considers the problem of maintaining statistic aggregates over the last W elements of a data stream. First, the problem of counting the number of 1's in the last W bits of a binary stream is considered. A lower bound of…
In the fillable array problem one must maintain an array A[1..n] of $w$-bit entries subject to random access reads and writes, and also a $\texttt{fill}(\Delta)$ operation which sets every entry of to some $\Delta\in\{0,\ldots,2^w-1\}$. We…
We consider the problem of constructing binary codes to recover from $k$-bit deletions with efficient encoding/decoding, for a fixed $k$. The single deletion case is well understood, with the Varshamov-Tenengolts-Levenshtein code from 1965…
In this paper we provide a method to obtain tight lower bounds on the minimum redundancy achievable by a Huffman code when the probability distribution underlying an alphabet is only partially known. In particular, we address the case where…
Computing the rate-distortion function for continuous sources is commonly regarded as a standard continuous optimization problem. When numerically addressing this problem, a typical approach involves discretizing the source space and…
Shannon's seminal 1948 work gave rise to two distinct areas of research: information theory and mathematical coding theory. While information theory has had a strong influence on theoretical neuroscience, ideas from mathematical coding…
Memorization in large language models poses critical risks for privacy and fairness as these systems scale to billions of parameters. While previous studies established correlations between memorization and factors like token frequency and…
In this paper, we study the problem of lossless universal source coding for stationary memoryless sources on countably infinite alphabets. This task is generally not achievable without restricting the class of sources over which…
This paper deals with rate distortion or source coding with fidelity criterion, in measure spaces, for a class of source distributions. The class of source distributions is described by a relative entropy constraint set between the true and…
We introduce new definitions of universal and superuniversal computable codes, which are based on a code's ability to approximate Kolmogorov complexity within the prescribed margin for all individual sequences from a given set. Such sets of…