Related papers: Empirical entropy in context
Some aspects of the predictability problem in dynamical systems are reviewed. The deep relation among Lyapunov exponents, Kolmogorov-Sinai entropy, Shannon entropy and algorithmic complexity is discussed. In particular, we emphasize how a…
We establish a general class of entropy inequalities that take the concise form of Gaussian comparisons. The main result unifies many classical and recent results, including the Shannon-Stam inequality, the Brunn-Minkowski inequality, the…
There is no single universally accepted definition of "Complexity". There are several perspectives on complexity and what constitutes complex behaviour or complex systems, as opposed to regular, predictable behaviour and simple systems. In…
A number of recent works have sought to generalize the Kolmogorov-Sinai entropy of probability-preserving transformations to the setting of Markov operators acting on the integrable functions on a probability space $(X,\mu)$. These have…
According to a previous conjecture, spatial and temporal Lyapunov exponents of chaotic extended systems can be obtained from derivatives of a suitable function: the entropy potential. The validity and the consequences of this hypothesis are…
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a…
In this paper we will give a short presentation of the quantum Levy-Khinchin formula and of the formulation of quantum continual measurements based on stochastic differential equations, matters which we had the pleasure to work on in…
By 'informational entropy', we understand an inherent boundary to knowability, due e.g. to perceptual, theoretical, evidential or linguistic limits. In this paper, we discuss a logical framework in which this boundary is incorporated into…
We define a general notion of entropy in elementary, algebraic terms. Based on that, weak forms of a scalar product and a distance measure are derived. We give basic properties of these quantities, generalize the Cauchy-Schwarz inequality,…
We study the entropy rate of pattern sequences of stochastic processes, and its relationship to the entropy rate of the original process. We give a complete characterization of this relationship for i.i.d. processes over arbitrary…
Stochastic chains represent a wide and key variety of phenomena in many branches of science within the context of Information Theory and Thermodynamics. They are typically approached by a sequence of independent events or by a memoryless…
We describe some basic results for Quantum Stochastic Processes and present some new results about a certain class of processes which are associated to Quantum Iterated Function Systems (QIFS). We discuss questions related to the Markov…
We derive upper and lower bounds on the overall compression ratio of the 1978 Lempel-Ziv (LZ78) algorithm, applied independently to $k$-blocks of a finite individual sequence. Both bounds are given in terms of normalized empirical entropies…
In this paper, some general properties of Shannon information measures are investigated over sets of probability distributions with restricted marginals. Certain optimization problems associated with these functionals are shown to be…
Different aspects of the predictability problem in dynamical systems are reviewed. The deep relation among Lyapunov exponents, Kolmogorov-Sinai entropy, Shannon entropy and algorithmic complexity is discussed. In particular, we emphasize…
These expository notes propose to follow, across fields, some aspects of the concept of entropy. Starting from the work of Boltzmann in the kinetic theory of gases, various universes are visited, including Markov processes and their…
Many of the traditional results in information theory, such as the channel coding theorem or the source coding theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a…
The evolution of entanglement entropy in quantum circuits composed of Haar-random gates and projective measurements shows versatile behavior, with connections to phase transitions and complexity theory. We reformulate the problem in terms…
This is a review on entropy in various fields of mathematics and science. Its scope is to convey a unified vision of the classical as well as some newer entropy notions to a broad audience with an intermediate background in dynamical…
We survey diverse approaches to the notion of information: from Shannon entropy to Kolmogorov complexity. Two of the main applications of Kolmogorov complexity are presented: randomness and classification. The survey is divided in two parts…