Related papers: Combinatorial Entropies and Statistics
Boltzmann defined the entropy of a macroscopic system in a macrostate $M$ as the $\log$ of the volume of phase space (number of microstates) corresponding to $M$. This agrees with the thermodynamic entropy of Clausius when $M$ specifies the…
The canonical ensemble describes an open system in equilibrium with a heat bath of fixed temperature. The probability distribution of such a system, the Boltzmann distribution, is derived from the uniform probability distribution of the…
We present novel bounds for estimating discrete probability distributions under the $\ell_\infty$ norm. These are nearly optimal in various precise senses, including a kind of instance-optimality. Our data-dependent convergence guarantees…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
Let $ X_1, \ldots, X_n $ be independent random variables taking values in the alphabet $ \{0, 1, \ldots, r\} $, and $ S_n = \sum_{i = 1}^n X_i $. The Shepp--Olkin theorem states that, in the binary case ($ r = 1 $), the Shannon entropy of $…
During the past dozen years there have been numerous articles on a relation between entropy and probability which is non-additive and has a parameter $q$ that depends on the nature of the thermodynamic system under consideration. For $q=1$…
We propose a general approach, named by us hyperstatistics, to treat complex systems, in which Boltzmann-Gibbs statistics breaks down in domains of the system. Hyperstatistics preserves the concavity of nonadditive $q$-entropy. We obtain…
The violation of the Pauli principle has been surmised in several models of the Fractional Exclusion Statistics and successfully applied to several quantum systems. In this paper, a classical alternative of the exclusion statistics is…
Attempts to replicate probabilistic reasoning in expert systems have typically overlooked a critical ingredient of that process. Probabilistic analysis typically requires extensive judgments regarding interdependencies among hypotheses and…
We define a diagonal entropy (d-entropy) for an arbitrary Hamiltonian system as $S_d=-\sum_n \rho_{nn}\ln \rho_{nn}$ with the sum taken over the basis of instantaneous energy states. In equilibrium this entropy coincides with the…
The foundations of the Boltzmann-Gibbs (BG) distributions for describing equilibrium statistical mechanics of systems are examined. Broadly, they fall into: (i) probabilistic paaroaches based on the principle of equal a priori probability…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…
In order to study as a whole a wide part of entropy measures, we introduce a two-parameter non-extensive entropic form with respect to the $h$-derivative, which generalizes the conventional Newton--Leibniz calculus. This new entropy,…
Building on work of Kontsevich, we introduce a definition of the entropy of a finite probability distribution in which the "probabilities" are integers modulo a prime p. The entropy, too, is an integer mod p. Entropy mod p is shown to be…
Entropy is a fundamental property of both classical and quantum systems, spanning myriad theoretical and practical applications in physics and computer science. We study the problem of obtaining estimates to within a multiplicative factor…
After a brief introduction to Probability Bracket Notation (PBN), indicator operator and conditional density operator (CDO), we investigate probability spaces associated with various quantum systems: system with one observable (discrete or…
In this letter we show that the Shore--Johnson axioms for Maximum Entropy Principle in statistical estimation theory account for a considerably wider class of entropic functional than previously thought. Apart from a formal side of the…
The entropic way of formulating Heisenberg's uncertainty principle not only plays a fundamental role in applications of quantum information theory but also is essential for manifesting genuine nonclassical features of quantum systems. In…
The issue of the thermodynamics of a system of distinguishable particles is discussed in this paper. In constructing the statistical mechanics of distinguishable particles from the definition of Boltzmann entropy, it is found that the…
By using the Jacobi metric of the configuration space, and assuming ergodicity, we calculate the Boltzmann entropy $S$ of a finite-dimensional system around a non-degenerate critical point of its potential energy $V$. We compare $S$ with…