Related papers: Entropy Distance
We design a heuristic method, a genetic algorithm, for the computation of an upper bound of the minimum distance of a linear code over a finite field. By the use of the row reduced echelon form, we obtain a permutation encoding of the…
New lower bounds on the minimum average Hamming distance of binary codes are derived. The bounds are obtained using linear programming approach.
We consider the problem of constructing prefix-free codes in which a designated symbol, a space, can only appear at the end of codewords. We provide a linear-time algorithm to construct almost-optimal codes with this property, meaning that…
We study entropy-bounded computational geometry, that is, geometric algorithms whose running times depend on a given measure of the input entropy. Specifically, we introduce a measure that we call range-partition entropy, which unifies and…
The number-theoretic codes are a class of codes defined by single or multiple congruences and are mainly used for correcting insertion and deletion errors. Since the number-theoretic codes are generally non-linear, the analysis method for…
The asymptotic rate vs. distance problem is a long-standing fundamental problem in coding theory. The best upper bound to date was given in 1977 and has received since then numerous proofs and interpretations. Here we provide a new,…
Inspired by the recent theory of Entropy-Transport problems and by the $\mathbf{D}$-distance of Sturm on normalised metric measure spaces, we define a new class of complete and separable distances between metric measure spaces of possibly…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
We study a quantity called discrete layered entropy, which approximates the Shannon entropy within a logarithmic gap. Compared to the Shannon entropy, the discrete layered entropy is piecewise linear, approximates the expected length of the…
Bounds on the entropy of patterns of sequences generated by independently identically distributed (i.i.d.) sources are derived. A pattern is a sequence of indices that contains all consecutive integer indices in increasing order of first…
We introduce distance entropy as a measure of homogeneity in the distribution of path lengths between a given node and its neighbours in a complex network. Distance entropy defines a new centrality measure whose properties are investigated…
An equidistant code is a code in the Hamming space such that two distinct codewords have the same Hamming distance. This paper investigates the bounds for equidistant codes in Hamming spaces.
We establish a connection between barcode entropy and metric entropy. Namely, we show that the barcode entropy bounds the metric entropy from below for a measure from a specific class of invariant measures associated with a pair of…
In many high-impact applications, it is important to ensure the quality of output of a machine learning algorithm as well as its reliability in comparison with the complexity of the algorithm used. In this paper, we have initiated a…
In many problems in data mining and machine learning, data items that need to be clustered or classified are not points in a high-dimensional space, but are distributions (points on a high dimensional simplex). For distributions, natural…
This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal…
The entropy region is a fundamental object in information theory. An outer bound for the entropy region is defined by a minimal set of Shannon-type inequalities called elemental inequalities also referred to as the Shannon region. This…
We consider the problem of describing the typical (possibly) non-linear code of minimum distance bounded from below over a large alphabet. We concentrate on block codes with the Hamming metric and on subspace codes with the injection…
We study the notion of approximate entropy within the framework of network theory. Approximate entropy is an uncertainty measure originally proposed in the context of dynamical systems and time series. We firstly define a purely structural…
Entanglement entropy appears as a central property of quantum systems in broad areas of physics. However, its precise value is often sensitive to unknown microphysics, rendering it incalculable. By considering parametric dependence on…