Related papers: Existence and Uniqueness of Proper Scoring Rules
We develop new approaches in multi-class settings for constructing proper scoring rules and hinge-like losses and establishing corresponding regret bounds with respect to the zero-one or cost-weighted classification loss. Our construction…
We show that the essential properties of entropy (monotonicity, additivity and subadditivity) are consequences of entropy being a monoidal natural transformation from the under category functor $-/\mathsf{LProb}_{\rho}$ (where…
The new estimates of the conditional Shannon entropy are introduced in the framework of the model describing a discrete response variable depending on a vector of d factors having a density w.r.t. the Lebesgue measure in R^d. Namely, the…
Building on work of Kontsevich, we introduce a definition of the entropy of a finite probability distribution in which the "probabilities" are integers modulo a prime p. The entropy, too, is an integer mod p. Entropy mod p is shown to be…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
Reliable data-driven estimation of Shannon entropy from small data sets, where the number of examples is potentially smaller than the number of possible outcomes, is a critical matter in several applications. In this paper, we introduce a…
The concept of entropy, firstly introduced in information theory, rapidly became popular in many applied sciences via Shannon's formula to measure the degree of heterogeneity among observations. A rather recent research field aims at…
We consider a family of probability distributions depending on a real parameter and including the binomial, Poisson and negative binomial distributions. The corresponding index of coincidence satisfies a Heun differential equation and is a…
Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…
Recently, in [P. Jizba and J. Korbel, Physica A 444, 2016, 808-827], four generalized Shannon-Khinchin [GSK] axioms have been proposed and a generalized entropy which uniquely satisfies the GSK axioms has been derived. In this comment, we…
We establish the existence of an integer degree for the natural projection map from the space of parameterizations of asymptotically conical self-expanders to the space of parameterizations of the asymptotic cones when this map is proper.…
It was recently shown that estimating the Shannon entropy $H({\rm p})$ of a discrete $k$-symbol distribution ${\rm p}$ requires $\Theta(k/\log k)$ samples, a number that grows near-linearly in the support size. In many applications $H({\rm…
This paper studies the behavior of the entropy numbers of classes of functions with bounded integral norms from a given finite dimensional linear subspace. Upper bounds of these entropy numbers in the uniform norm are obtained and applied…
This article provides a completion to theories of information based on entropy, resolving a longstanding question in its axiomatization as proposed by Shannon and pursued by Jaynes. We show that Shannon's entropy function has a…
We study how the Shannon entropy of sequences produced by an information source converges to the source's entropy rate. We synthesize several phenomenological approaches to applying information theoretic measures of randomness and memory to…
Statistical mechanics relies on the complete though probabilistic description of a system in terms of all the microscopic variables. Its object is to derive therefrom static and dynamic properties involving some reduced set of variables.…
The first aims of this work are to endorse the advent of finitely additive set functions as equilibrium states and the possibility to replace the metric entropy by an upper semi-continuous map associated to a general variational principle.…
Real continuous submodular functions, as a generalization of the corresponding discrete notion to the continuous domain, gained considerable attention recently. The analog notion for entropy functions requires additional properties: a real…
We interpret the Hilbert entropy of a convex projective structure on a closed higher-genus surface as the Hausdorff dimension of the non-differentiability points of the limit set in the full flag space $\mathcal F(\mathbb R^3)$.…
This paper investigates the behavior of sets and functions at infinity by introducing new concepts, namely directional normal cones at infinity for unbounded sets, along with limiting and singular subdifferentials at infinity in the…