Related papers: The Boltzmann/Shannon entropy as a measure of corr…
The foundations of the Boltzmann-Gibbs (BG) distributions for describing equilibrium statistical mechanics of systems are examined. Broadly, they fall into: (i) probabilistic paaroaches based on the principle of equal a priori probability…
Boltzmann-Gibbs statistical mechanics is based on the entropy $S_{BG}=-k \sum_{i=1}^W p_i \ln p_i$. It enables a successful thermal approach of ubiquitous systems, such as those involving short-range interactions, markovian processes, and,…
Information theoretic measures (entropies, entropy rates, mutual information) are nowadays commonly used in statistical signal processing for real-world data analysis. The present work proposes the use of Auto Mutual Information (Mutual…
The aim of this paper is two-fold: in probing the statistical mechanical properties of interacting quantum fields, and in providing a field theoretical justification for a stochastic source term in the Boltzmann equation. We start with the…
The Tsallis-Cirto non-extensive statistics with $\delta=2$ describes the processes of splitting and merging of black holes and their thermodynamics. Here we consider a toy model, which matches this generalized statistics and extends it by…
The concept of Entropy plays a key role in Information Theory, Statistics, and Machine Learning.This paper introduces a new entropy measure, called the t-entropy, which exploits the concavity of the inverse-tan function. We analytically…
Inference and learning are commonly cast in terms of optimisation, yet the fundamental constraints governing uncertainty reduction remain unclear. This work presents a first-principles framework inherent to Bayesian updating, termed…
The time variation of entropy, as an alternative to the variance, is proposed as a measure of the diffusion rate. It is shown that for linear and time-translationally invariant systems having a large-time limit for the density, at large…
An axiomatic foundation regarding the entropy for complex systems is established. Missing from decades of research was the requirement that entropy must measure the uncertainty at the informational scale of the maximizing distribution,…
We consider an isolated system in an arbitrary state and provide a general formulation using first principles for an additive and non-negative statistical quantity that is shown to reproduce the equilibrium thermodynamic entropy of the…
A multi-parametric version of the nonadditive entropy $S_{q}$ is introduced. This new entropic form, denoted by $S_{a,b,r}$, possesses many interesting statistical properties, and it reduces to the entropy $S_q$ for $b=0$, $a=r:=1-q$ (hence…
An analysis of quantum measurement is presented that relies on an information-theoretic description of quantum entanglement. In a consistent quantum information theory of entanglement, entropies (uncertainties) conditional on measurement…
Kawai, Parrondo, and Van den Broeck [Phys. Rev. Lett. 98, 080602 (2007)] have recently established a quantitative relationship between dissipated work and a microscopic, information-theoretic measure of irreversibility. We illustrate this…
It is not obvious how to extend Shannon's original information entropy to higher dimensions, and many different approaches have been tried. We replace the English text symbol sequence originally used to illustrate the theory by a discrete,…
Entanglement entropy, which is a measure of quantum correlations between separate parts of a many-body system, has emerged recently as a fundamental quantity in broad areas of theoretical physics, from cosmology and field theory to…
We define a general notion of entropy in elementary, algebraic terms. Based on that, weak forms of a scalar product and a distance measure are derived. We give basic properties of these quantities, generalize the Cauchy-Schwarz inequality,…
Although an intimate relation between entropy and diffusion has been advocated for many years and even seems to have been verified in theory and experiments, a quantitatively reliable study, and any derivation of an algebraic relation…
The cornerstones of Boltzmann-Gibbs and nonextensive statistical mechanics respectively are the entropies $S_{BG} \equiv -k \sum_{i=1}^W p_i \ln p_i $ and $S_{q}\equiv k (1-\sum_{i=1}^Wp_i^{q})/(q-1) (q\in{\mathbb R} ; S_1=S_{BG})$. Through…
We explore the relation between entanglement entropy of quantum many body systems and the distribution of corresponding, properly selected, observables. Such a relation is necessary to actually measure the entanglement entropy. We show that…
This article provides a completion to theories of information based on entropy, resolving a longstanding question in its axiomatization as proposed by Shannon and pursued by Jaynes. We show that Shannon's entropy function has a…