Related papers: Statistical Criticality arises in Most Informative…
A recurrent idea in the study of complex systems is that optimal information processing is to be found near bifurcation points or phase transitions. However, this heuristic hypothesis has few (if any) concrete realizations where a standard…
An explosive percolation transition is the abrupt emergence of a giant cluster at a threshold caused by a suppression of the growth of large clusters. In this paper, we consider the information entropy of the cluster size distribution,…
A sufficient statistic is a significant concept in statistics, which means a probability variable that has sufficient information required for an inference task. We investigate the roles of sufficient statistics and related quantities in…
The problem of determining the best achievable performance of arbitrary lossless compression algorithms is examined, when correlated side information is available at both the encoder and decoder. For arbitrary source-side information pairs,…
The output distribution, when rate is above capacity, is investigated. It is shown that there is an asymptotic equipartition property (AEP) of the typical output sequences, independently of the specific codebook used, as long as the…
The use of the von Neumann entropy in formulating the laws of thermodynamics has recently been challenged. It is associated with the average work whereas the work guaranteed to be extracted in any single run of an experiment is the more…
This paper introduces a statistical and other analysis of peer reviewers in order to approach their "quality" through some quantification measure, thereby leading to some quality metrics. Peer reviewer reports for the Journal of the Serbian…
The characterisation of information processing is an important task in complex systems science. Information dynamics is a quantitative methodology for modelling the intrinsic information processing conducted by a process represented as a…
Stochastic thermodynamics gives universal relations for microscopic entropy production, yet its critical behavior at macroscopic nonequilibrium transitions remains unclassified. We study well-mixed reversible chemical reaction networks in…
We address the fundamental limits of learning unknown parameters of any stochastic process from time-series data, and discover exact closed-form expressions for how optimal inference scales with observation length. Given a parametrized…
Zipf's law states that sequential frequencies of words in a text correspond to a power function. Its probabilistic model is an infinite urn scheme with asymptotically power distribution. The exponent of this distribution must be estimated.…
Information theoretic measures (entropies, entropy rates, mutual information) are nowadays commonly used in statistical signal processing for real-world data analysis. The present work proposes the use of Auto Mutual Information (Mutual…
Systematic inaccuracy is inherent in any computational estimate of a non-linear average, due to the availability of only a finite number of data values, N. Free energy differences (DF) between two states or systems are critically important…
In statistical physics, if we successively divide an equilibrium system into two parts, we will face a situation that, within a certain length $\xi$, the physics of a subsystem is no longer the same as the original system. Then the…
Asymptotically entropy of chaotic systems increases linearly and the sensitivity to initial conditions is exponential with time: these two behaviors are related. Such relationship is the analogous of and under specific conditions has been…
We consider Hamiltonian quantum systems with energy bandwidth \Delta E and show that each measurement that determines the time up to an error \Delta t generates at least the entropy (\hbar/(\Delta t \Delta E))^2/2. Our result describes…
The problem of compression in standard information theory consists of assigning codes as short as possible to numbers. Here we consider the problem of optimal coding -- under an arbitrary coding scheme -- and show that it predicts Zipf's…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible while constrained to match empirically estimated feature expectations. However, in many real-world…
The frequencies at which individual words occur across languages follow power law distributions, a pattern of findings known as Zipf's law. A vast literature argues over whether this serves to optimize the efficiency of human communication,…
Phase transitions are conventionally defined by nonanalyticities of thermodynamic potentials in the thermodynamic limit. In this Letter, we show that the singularity is not the definition of criticality but its asymptotic outcome:…