Related papers: A Scale-Invariant Entropy Statistic for Distance D…

In a paper [8] the authors classify entropy into three categories, as a thermodynamics quantity, as a measure of information production, and as a means of statistical inference. An entropy measure introduced by Mathai falls into the second…

We lift important results about universally typical sets, typically sampled sets, and empirical entropy estimation in the theory of samplings of discrete ergodic information sources from the usual one-dimensional discrete-time setting to a…

Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…

In this paper we generalize the concept of random networks to describe networks with non trivial features by a statistical mechanics approach. This framework is able to describe ensembles of undirected, directed as well as weighted…

A pattern of a sequence is a sequence of integer indices with each index describing the order of first occurrence of the respective symbol in the original sequence. In a recent paper, tight general bounds on the block entropy of patterns of…

Products between phase-type distributed random variables and any independent, positive and continuous random variable are studied. Their asymptotic properties are established, and an expectation-maximization algorithm for their effective…

We study entropy-bounded computational geometry, that is, geometric algorithms whose running times depend on a given measure of the input entropy. Specifically, we introduce a measure that we call range-partition entropy, which unifies and…

Systematic generalization remains challenging for current language models, which are known to be both sensitive to semantically similar permutations of the input and to struggle with known concepts presented in novel contexts. Although…

The profile of a sample is the multiset of its symbol frequencies. We show that for samples of discrete distributions, profile entropy is a fundamental measure unifying the concepts of estimation, inference, and compression. Specifically,…

We consider distributions of ordered random vectors with given one-dimensional marginal distributions. We give an elementary necessary and sufficient condition for the existence of such a distribution with finite entropy. In this case, we…

We study the notion of approximate entropy within the framework of network theory. Approximate entropy is an uncertainty measure originally proposed in the context of dynamical systems and time series. We firstly define a purely structural…

This paper deals with the construction of the multiparticle correlation expansion of relative entropy for lattice systems. Thanks to this analysis we are able to express the statistical distance between two systems as a series built over…

In this paper non-asymptotic exact rearrangement invariant norm estimates are derived for the maximum distribution of the family elements of some rearrangement invariant (r.i.) space over unbounded measure in the entropy terms and in the…

Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…

We introduce a new family of shift spaces --- the subordinate shifts. Using subordinate shifts we prove in an elementary way that for every nonnegative real number $t$ there is a shift space with entropy $t$.

The Shannon entropy of a random variable has much behaviour analogous to a signed measure. Previous work has explored this connection by defining a signed measure on abstract sets, which are taken to represent the information that different…

The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant "additivity" properties: (i) existence of a…

After a brief historical survey, the paper introduces the notion of entropic model sets (cut and project sets), and, more generally, the notion of diffractive point sets with entropy. Such sets may be thought of as generalizations of…

We consider an isomorphism invariant for measure-preserving systems - types of generalized entropy convergence rates. We show the connections of this invariant with the types of Shannon entropy convergence rates. In the case when they…

We introduce a notion called entropic independence that is an entropic analog of spectral notions of high-dimensional expansion. Informally, entropic independence of a background distribution $\mu$ on $k$-sized subsets of a ground set of…