Related papers: Combinatorial Entropies and Statistics
Estimating the uncertainty of a model's prediction on a test point is a crucial part of ensuring reliability and calibration under distribution shifts. A minimum description length approach to this problem uses the predictive normalized…
It has recurrently been proposed that the Boltzmann textbook definition of entropy $S(E)=k\ln \Omega (E)$ in terms of the number of microstates $\Omega (E)$ with energy $E$ should be replaced by the expression $S_G(E)=k\ln \sum_{E^\prime…
It is supposed that the exponential multiplier in the method of the non-equilibrium statistical operator (Zubarev`s approach) can be considered as a distribution density of the past lifetime of the system, and can be replaced by an…
A nonmonotonic logic of thresholded generalizations is presented. Given propositions A and B from a language L and a positive integer k, the thresholded generalization A=>B{k} means that the conditional probability P(B|A) falls short of one…
The paper analyzes the entropy of a system composed by non-interacting and indistinguishable particles whose quantum state numbers are modelled as independent and identically distributed classical random variables. The crucial observation…
Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order…
Analysis of a probabilistic system often requires to learn the joint probability distribution of its random variables. The computation of the exact distribution is usually an exhaustive precise analysis on all executions of the system. To…
Two new information-theoretic methods are introduced for establishing Poisson approximation inequalities. First, using only elementary information-theoretic techniques it is shown that, when $S_n=\sum_{i=1}^nX_i$ is the sum of the (possibly…
Although an intimate relation between entropy and diffusion has been advocated for many years and even seems to have been verified in theory and experiments, a quantitatively reliable study, and any derivation of an algebraic relation…
Observational entropy provides a general notion of quantum entropy that appropriately interpolates between Boltzmann's and Gibbs' entropies, and has recently been argued to provide a useful measure of out-of-equilibrium thermodynamic…
One of the difficulties in calculating the capacity of certain Poisson channels is that H(lambda), the entropy of the Poisson distribution with mean lambda, is not available in a simple form. In this work we derive upper and lower bounds…
We introduce a binary relation on the finite discrete probability distributions which generalizes notions of majorization that have been studied in quantum information theory. Motivated by questions in thermodynamics, our relation describes…
Bohl dichotomy is a notion of hyperbolicity for linear nonautonomous difference equations that is weaker than the classical concept of exponential dichotomy. In the class of systems with bounded invertible coefficient matrices which have…
The notion of utility maximising entropy (u-entropy) of a probability density, which was introduced and studied by Slomczynski and Zastawniak (Ann. Prob 32 (2004) 2261-2285, arXiv:math.PR/0410115 v1), is extended in two directions. First,…
We consider entropy in Generalized Non-Signalling Theory (also known as box world) where the most common definition of entropy is the measurement entropy. In this setting, we completely characterize the set of allowed entropies for a…
We introduce a ``two-particle factorization'' condition which allows us to formulate the homogeneous Boltzmann equation for non-reversible collision kernels in terms of an entropy inequality. This formulation yields an H-Theorem. We provide…
An essential component of inference based on familiar frequentist notions, such as $p$-values, significance and confidence levels, is the relevant sampling distribution. This feature results in violations of a principle known as the strong…
For classical dynamical systems, the polynomial entropy serves as a refined invariant of the topological entropy. In the setting of categorical dynamical systems, that is, triangulated categories endowed with an endofunctor, we develop the…
The primordial confrontation underlying the existence of our universe can be conceived as the battle between entropy and complexity. The law of ever-increasing entropy (Boltzmann H-theorem) evokes an irreversible, one-directional evolution…
In this paper we give a method to associate a graph with an arbitrary density matrix referred to a standard orthonormal basis in the Hilbert space of a finite dimensional quantum system. We study the related issues like classification of…