Related papers: Mathematical inequalities for some divergences
Since its original formulation, Jensen's inequality has played a fundamental role across mathematics, statistics, and machine learning, with its probabilistic version highlighting the nonnegativity of the so-called Jensen's gap, i.e., the…
This paper is devoted to the mathematical study of some divergences based on the mutual information well-suited to categorical random vectors. These divergences are generalizations of the "entropy distance" and "information distance". Their…
We define a general notion of entropy in elementary, algebraic terms. Based on that, weak forms of a scalar product and a distance measure are derived. We give basic properties of these quantities, generalize the Cauchy-Schwarz inequality,…
There are three classical divergence measures exist in the literature on information theory and statistics. These are namely, Jeffryes-Kullback-Leiber J-divergence. Sibson-Burbea-Rao Jensen-Shannon divegernce and Taneja Arithmetic-Geometric…
We find the value of constants related to constraints in characterization of some known statistical distributions and then we proceed to use the idea behind maximum entropy principle to derive generalized version of this distributions using…
The coupled entropy is proven to correct a flaw in the derivation of the Tsallis entropy and thereby solidify the theoretical foundations for analyzing the uncertainty of complex systems. The Tsallis entropy originated from considering…
We consider the generalized differential entropy of normalized sums of independent and identically distributed (IID) continuous random variables. We prove that the R\'{e}nyi entropy and Tsallis entropy of order $\alpha\ (\alpha>0)$ of the…
We consider a two-parameter family of R\'enyi relative entropies $D_{\alpha,z}(\rho||\sigma)$ that are quantum generalisations of the classical R\'enyi divergence $D_{\alpha}(p||q)$. This family includes many known relative entropies (or…
Shearer's inequality bounds the sum of joint entropies of random variables in terms of the total joint entropy. We give another lower bound for the same sum in terms of the individual entropies when the variables are functions of…
We define a class of divergences to measure differences between probability density functions in one-dimensional sample space. The construction is based on the convex function with the Jacobi operator of mapping function that pushforwards…
Change of measure inequalities translate divergences between probability measures into explicit bounds on event probabilities, and play an important role in deriving probabilistic guarantees in learning theory, information theory, and…
Estimating entropies from limited data series is known to be a non-trivial task. Naive estimations are plagued with both systematic (bias) and statistical errors. Here, we present a new 'balanced estimator' for entropy functionals Shannon,…
A class of distortions termed functional Bregman divergences is defined, which includes squared error and relative entropy. A functional Bregman divergence acts on functions or distributions, and generalizes the standard Bregman divergence…
In this paper, we propose a new discriminative model named \emph{nonextensive information theoretical machine (NITM)} based on nonextensive generalization of Shannon information theory. In NITM, weight parameters are treated as random…
There are many information and divergence measures exist in the literature on information theory and statistics. The most famous among them are Kullback-Leiber relative information and Jeffreys J-divergence. The measures like, Bhattacharya…
We present a path toward determining the statistical origin of the thermodynamic limit for systems with long-range interactions. We assume throughout that the systems under consideration have thermodynamic properties given by the Tsallis…
One way to define the concentration of measure phenomenon is via Talagrand inequalities, also called transportation-information inequalities. That is, a comparison of the Wasserstein distance from the given measure to any other absolutely…
We revisit the derivation of a formula for the $q$-generalised multinomial coefficient rooted in the $q$-deformed algebra, a foundational framework in the study of nonextensive statistics. Previous approximate expressions in the literature…
This paper proposes the meeting of fuzzy logic with paraconsistency in a very precise and foundational way. Specifically, in this paper we introduce expansions of the fuzzy logic MTL by means of primitive operators for consistency and…
Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann)…