Related papers: Range-Renewal Speed and Entropy for I.I.D Models

We apply a common measure of randomness, the entropy, in the context of iterated functions on a finite set with n elements. For a permutation, it turns out that this entropy is asymptotically (for a growing number of iterations) close to…

Various properties of relative entropy have led to its widespread use in information theory. These properties suggest that relative entropy has a role to play in systems that attempt to perform inference in terms of probability…

Certain renewal theorems are extended to the case that the rate of the renewal process goes to 0 and, more generally, to the case that the drift of the random walk goes to infinity. These extensions are motivated by and applied to the…

In the context of control and estimation under information constraints, restoration entropy measures the minimal required data rate above which the state of a system can be estimated so that the estimation quality does not degrade over time…

A definition of entropy via the Kolmogorov algorithmic complexity is discussed. As examples, we show how the meanfield theory for the Ising model, and the entropy of a perfect gas can be recovered. The connection with computations are…

We study the unitary time evolution of the entropy of entanglement of a one-dimensional system between the degrees of freedom in an interval of length l and its complement, starting from a pure state which is not an eigenstate of the…

This is a survey paper on the dimension theory of self-similar measures on the real line focusing on the role of entropy rates.

We introduce the entropy rate of multidimensional cellular automata. This number is invariant under shift-commuting isomorphisms; as opposed to the entropy of such CA, it is always finite. The invariance property and the finiteness of the…

The routine definitions of both entropy, and differential entropy show inconsistencies that make them not reciprocally coherent. We propose a few possible modifications of these quantities so that 1) they no longer show incongruities, 2)…

We study the exponential decay of relative entropy functionals for zero-range processes on the complete graph. For the standard model with rates increasing at infinity we prove entropy dissipation estimates, uniformly over the number of…

Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building…

We investigate bounds on speed, non-adiabatic entropy production and trade-off relation between them for classical stochastic processes with time-independent transition rates. Our results show that the time required to evolve from an…

In this article we study a diophantine property of probability measures on R. We will always assume that the considered measures have an exponential moment and a drift. We link this property to the points in C close to the imaginary axis…

We consider in this paper the problem of computing the entropy of a braid. We recall its definition and construct, for each braid, a sequence of real numbers, whose limit is its entropy. We state one conjecture about the convergence speed,…

Intermolecular correlations lower values of both diffusion and entropy. We present an analysis of the existing relations between long-time diffusion (D) and entropy. S. A recently proposed inequality, a lower bound, by Sorkin et al.,…

We propose utilizing entropy as a diagnostic tool to distinguish between constant and dynamical dark energy models. Entropy, a measure of the system's disorder or information content, captures the complexity and evolution of the universe.…

The entropy rates of the Wright-Fisher process, the Moran process, and generalizations are computed and used to compare these processes and their dependence on standard evolutionary parameters. Entropy rates are measures of the variation…

The entanglement evolution after a quantum quench became one of the tools to distinguish integrable versus chaotic (non-integrable) quantum many-body dynamics. Following this line of thoughts, here we propose that the revivals in the…

Deep Reinforcement Learning (Deep RL) has had incredible achievements on high dimensional problems, yet its learning process remains unstable even on the simplest tasks. Deep RL uses neural networks as function approximators. These neural…

The mutual information between stimulus and spike-train response is commonly used to monitor neural coding efficiency, but neuronal computation broadly conceived requires more refined and targeted information measures of input-output joint…