相关论文: Nonextensive Generalizations of the Jensen-Shannon…

This article serves as a brief introduction to the Shannon information theory. Concepts of information, Shannon entropy and channel capacity are mainly covered. All these concepts are developed in a totally combinatorial flavor. Some issues…

Skew-symmetric densities recently received much attention in the literature, giving rise to increasingly general families of univariate and multivariate skewed densities. Most of those families, however, suffer from the inferential drawback…

The Jeans gravitational instability in nonextensive statistical mechanics is studied and a general form of the generalized Jeans criterion is obtained that is related to the q-function . In this approach, the nonextensive model of classical…

From the output produced by a memoryless deletion channel with a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…

Estimating the Shannon entropy of a discrete distribution from which we have only observed a small sample is challenging. Estimating other information-theoretic metrics, such as the Kullback-Leibler divergence between two sparsely sampled…

In this paper we use basic properties of strongly convex functions to obtain new inequalities including Jensen's type and Jensen-Mercer type inequalities. Applications for special means are pointed out as well. We also give a Jensen's…

We examine the estimation of the Kullback-Leibler (KL) divergence and the use of the goodness-of-fit test for multivariate continuous distributions. Our starting point is the maximum entropy principle for Shannon entropy: among all…

This is a study of the information evolution of complex systems by geometrical consideration. We look at chaotic systems evolving in fractal phase space. The entropy change in time due to the fractal geometry is assimilated to the…

We discuss different statistical distances in probability space, with emphasis on the Jensen-Shannon divergence, vis-a-vis {\it metrics} in Hilbert space and their relationship with Fisher's information measure. This study provides further…

Recently, information theoretic analysis has become a popular framework for understanding the generalization behavior of deep neural networks. It allows a direct analysis for stochastic gradient/Langevin descent (SGD/SGLD) learning…

There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the…

Jensen's operator inequality for convexifiable functions is obtained. This result contains classical Jensen's operator inequality as a particular case. As a consequence, a new refinement and a reverse of Young's inequality is given.

Logical information theory is the quantitative version of the logic of partitions just as logical probability theory is the quantitative version of the dual Boolean logic of subsets. The resulting notion of information is about…

This paper explores useful modifications of the recent development in contrastive learning via novel probabilistic modeling. We derive a particular form of contrastive loss named Joint Contrastive Learning (JCL). JCL implicitly involves the…

We compare two different techniques for proving non-Shannon-type information inequalities. The first one is the original Zhang-Yeung's method, commonly referred to as the copy/pasting lemma/trick. The copy lemma was used to derive the first…

Probability distributions defined on the half space are known to be quite different from those in the full space. Here, a nonextensive entropic treatment is presented for the half space in an analytic and self-consistent way. In this…

We have discussed the Tsallis entropy in finite $N$-unit nonextensive systems, by using the multivariate $q$-Gaussian probability distribution functions (PDFs) derived by the maximum entropy methods with the normal average and the…

The main goal of this paper is to investigate strong duality of non-convex semidefinite programming problems (SDPs). In the optimization community, it is well-known that a convex optimization problem satisfies strong duality if the Slater's…

This article provides a completion to theories of information based on entropy, resolving a longstanding question in its axiomatization as proposed by Shannon and pursued by Jaynes. We show that Shannon's entropy function has a…

We analyze the Schwarz inequality and its generalizations, as well as inequalities resulting from the Jensen inequality. They are used in quantum theory to derive the Heisenberg-Robertson (HR) and Schroedinger-Robertson (SR) uncertainty…