Related papers: Combinatorial Entropy for Distinguishable Entities…
Shannon and Khinchin showed that assuming four information theoretic axioms the entropy must be of Boltzmann-Gibbs type, $S=-\sum_i p_i \log p_i$. Here we note that in physical systems one of these axioms may be violated. For non-ergodic…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
In this paper, we study the Bernstein polynomial model for estimating the multivariate distribution functions and densities with bounded support. As a mixture model of multivariate beta distributions, the maximum (approximate) likelihood…
The entropic way of formulating Heisenberg's uncertainty principle not only plays a fundamental role in applications of quantum information theory but also is essential for manifesting genuine nonclassical features of quantum systems. In…
We evaluate Shannon entropy for the position and momentum eigenstates of some conditionally exactly solvable potentials which are isospectral to harmonic oscillator and whose solutions are given in terms of exceptional orthogonal…
We consider the uncertainty between two pairs of local projective measurements performed on a multipartite system. We show that the optimal bound in any linear uncertainty relation, formulated in terms of the Shannon entropy, is additive.…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
A common statistical situation concerns inferring an unknown distribution Q(x) from a known distribution P(y), where X (dimension n), and Y (dimension m) have a known functional relationship. Most commonly, n<m, and the task is relatively…
Although compartmental dynamical systems are used in many different areas of science, model selection based on the maximum entropy principle (MaxEnt) is challenging because of the lack of methods for quantifying the entropy for this type of…
The issue of the thermodynamics of a system of distinguishable particles is discussed in this paper. In constructing the statistical mechanics of distinguishable particles from the definition of Boltzmann entropy, it is found that the…
The Boltzmann entropy $S^{(B)}$ is true in the case of equal probability of all microstates of a system. In the opposite case it should be averaged over all microstates that gives rise to the Boltzmann--Shannon entropy (BSE). Maximum…
Maximum entropy (MAXENT) method has a large number of applications in theoretical and applied machine learning, since it provides a convenient non-parametric tool for estimating unknown probabilities. The method is a major contribution of…
We study maximum-entropy inference for finite-dimensional quantum states under linear moment constraints. Given expectation values of finitely many observables, the feasible set of states is convex but typically non-unique. The…
There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a…
We consider a generic system composed of a fixed number of particles distributed over a finite number of energy levels. We make only general assumptions about system's properties and the entropy. System's constraints other than fixed number…
Entropic uncertainty relations, based on sums of entropies of probability distributions arising from different measurements on a given pure state, can be seen as a generalization of the Heisenberg uncertainty relation that is in many cases…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
We present a method to quantify entanglement in mixed states of highly symmetric systems. Symmetry constrains interactions between parts and predicts the degeneracies of the states. While symmetry alone produces entangled eigenstates, the…
Let $ X_1, \ldots, X_n $ be independent random variables taking values in the alphabet $ \{0, 1, \ldots, r\} $, and $ S_n = \sum_{i = 1}^n X_i $. The Shepp--Olkin theorem states that, in the binary case ($ r = 1 $), the Shannon entropy of $…
Typically, the entropy of an isolated system in equilibrium is calculated by counting the number of accessible microstates, or in more general cases by using the Gibbs formula. In irreversible processes entropy spontaneously increases and…