相关论文: Nonextensive Generalizations of the Jensen-Shannon…
There are three classical divergence measures in the literature on information theory and statistics, namely, Jeffryes-Kullback-Leiber's J-divergence, Sibson-Burbea-Rao's Jensen-Shannon divegernce and Taneja's arithemtic-geometric mean…
It is well-known that if a real valued function acting on a convex set satisfies the $n$-variable Jensen inequality, for some natural number $n\geq 2$, then, for all $k\in\{1,\dots, n\}$, it fulfills the $k$-variable Jensen inequality as…
Any physical system can be viewed from the perspective that information is implicitly represented in its state. However, the quantification of this information when it comes to complex networks has remained largely elusive. In this work, we…
In this paper we develop a general method for improving Jensen-type inequalities for convex and, even more generally, for piecewise convex functions. Our main result relies on the linear interpolation of a convex function. As a consequence,…
The information convex allows us to look into certain information-theoretic constraints in two-dimensional topological orders. We provide a derivation of the topological contribution $\ln d_a$ to the von Neumann entropy, where $d_a$ is the…
The Tsallis entropy is shown to be an additive entropy of degree-q that information scientists have been using for almost forty years. Neither is it a unique solution to the nonadditive functional equation from which random entropies are…
It is presented a generalization of the von Neumann mutual information in the context of Tsallis' nonextensive statistics. As an example, entanglement between two (two-level) quantum subsystems is discussed. Important changes occur in the…
Statements of Shannon's Noiseless Coding Theorem by various authors, including the original, are reviewed and clarified. Traditional statements of the theorem are often unclear as to when it applies. A new notation is introduced and the…
The nonextensive statistics based on Tsallis entropy have been so far used for the systems composed of subsystems having same $q$. The applicability of this statistics to the systems with different $q$'s is still a matter of investigation.…
We find the value of constants related to constraints in characterization of some known statistical distributions and then we proceed to use the idea behind maximum entropy principle to derive generalized version of this distributions using…
In information theory -- as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition -- many flexibilizations of the omnipresent Kullback-Leibler information distance…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
It is well known that in Information Theory and Machine Learning the Kullback-Leibler divergence, which extends the concept of Shannon entropy, plays a fundamental role. Given an {\it a priori} probability kernel $\hat{\nu}$ and a…
This work studies the Geometric Jensen-Shannon divergence, based on the notion of geometric mean of probability measures, in the setting of Gaussian measures on an infinite-dimensional Hilbert space. On the set of all Gaussian measures…
The aim of this paper is to discuss new results concerning some kinds of parametric extended entropies and divergences. As a result of our studies for mathematical properties on entropy and divergence, we give new bounds for the Tsallis…
Deep neural networks (DNNs) exhibit an exceptional capacity for generalization in practical applications. This work aims to capture the effect and benefits of depth for supervised learning via information-theoretic generalization bounds. We…
The Shannon-Khinchin axioms for the ordinary information entropy are generalized in a natural way to the nonextensive systems based on the concept of nonextensive conditional entropy, and a complete proof of the uniqueness theorem for the…
Complementarity relations between various characterizations of a probability distribution are at the core of information theory. In particular, lower and upper bounds for the entropic function are of great importance. In applied topics, we…
The Johnson-Lindenstrauss (JL) lemma is a cornerstone of dimensionality reduction in Euclidean space, but its applicability to non-Euclidean data has remained limited. This paper extends the JL lemma beyond Euclidean geometry to handle…
Divergences are quantities that measure discrepancy between two probability distributions and play an important role in various fields such as statistics and machine learning. Divergences are non-negative and are equal to zero if and only…