Related papers: Statistical Criticality arises in Most Informative…
Zipf's law, which states that the probability of an observation is inversely proportional to its rank, has been observed in many domains. While there are models that explain Zipf's law in each of them, those explanations are typically…
Entropy production (EP) is known as a fundamental quantity for measuring the irreversibility of processes in thermal equilibrium and states far from equilibrium. In stochastic thermodynamics, the EP becomes more visible in terms of the…
A new framework is introduced for examining and evaluating the fundamental limits of lossless data compression, that emphasizes genuinely non-asymptotic results. The {\em sample complexity} of compressing a given source is defined as the…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
Random matrix ensembles (RME) of quantum statistical Hamiltonian operators, {\em e.g.} Gaussian random matrix ensembles (GRME) and Ginibre random matrix ensembles (Ginibre RME), found applications in literature in study of following quantum…
We investigate the critical behavior of continuous phase transitions in the context of Ginzburg Landau models with a double well effective potential. In particular, we show that the recently proposed configurational entropy, a measure of…
The Renyi entropy with a free Renyi parameter $q$ is the most justified form of information entropy, and the Tsallis entropy may be regarded as a linear approximation to the Renyi entropy when $q\simeq 1$. When $q\to 1$, both entropies go…
Modern generative models exhibit unprecedented capabilities to generate extremely realistic data. However, given the inherent compositionality of the real world, reliable use of these models in practical applications requires that they…
In this work, we investigate the relation between the concept of ``information rate'', an information geometric method for measuring the speed of the time evolution of the statistical states of a stochastic process, and stochastic…
There is a consensus in science that information theory and statistical physics have a close relationship but the literary proofs of the equivalence between most of the conceptions in the two disciplines are still missing. In this work,…
We obtain the posterior distribution of a random process conditioned on observing the empirical frequencies of a finite sample path. We find under a rather broad assumption on the "dependence structure" of the process, {\em c.f.}…
A mathematical framework for information-theoretic analysis is established, with a new viewpoint of describing transmitted messages and communication channels by the nonlinear expectation theory, beyond the framework of classical…
In this article a classification of some proposed macroscopic entropy production (MEP) principles is given. With the help of simple electrical network models, at least six interesting and most used principles are distinguished: the least…
Maximum entropy principle (MEP) offers an effective and unbiased approach to inferring unknown probability distributions when faced with incomplete information, while neural networks provide the flexibility to learn complex distributions…
The maximum entropy principle can be used to assign utility values when only partial information is available about the decision maker's preferences. In order to obtain such utility values it is necessary to establish an analogy between…
We apply the Principle of Maximum Entropy to the study of a general class of deterministic fractal sets. The scaling laws peculiar to these objects are accounted for by means of a constraint concerning the average content of information in…
Finding interdependency relations between (possibly multivariate) time series provides valuable knowledge about the processes that generate the signals. Information theory sets a natural framework for non-parametric measures of several…
As experiments advance to record from tens of thousands of neurons, statistical physics provides a framework for understanding how collective activity emerges from networks of fine-scale correlations. While modeling these populations is…
We study information processing in populations of Boolean networks with evolving connectivity and systematically explore the interplay between the learning capability, robustness, the network topology, and the task complexity. We solve a…
We study the entropy production of a system with a finite number of states connected by random transition rates. The stationary entropy production, driven out of equilibrium both by asymmetric transition rates and by an external probability…