相关论文: Equivalence of Two Proof Techniques for Non-Shanno…

In this paper, we present a new multi-scale information content calculation method based on Shannon information (and Shannon entropy). The original method described by Claude E. Shannon and based on the logarithm of the probability of…

In this paper, we consider the simultaneous approximation of real points by rational points with the error of approximation given by the functions of `non-standard' heights. We prove analogues of Khintchine and Jarn\'ik-Besicovitch theorems…

Copies have been proposed as a viable alternative to endow machine learning models with properties and features that adapt them to changing needs. A fundamental step of the copying process is generating an unlabelled set of points to…

Convexity is a key concept in information theory, namely via the many implications of Jensen's inequality, such as the non-negativity of the Kullback-Leibler divergence (KLD). Jensen's inequality also underlies the concept of Jensen-Shannon…

While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, Shannon's entropy power inequality (EPI) seems to be an exception: available information theoretic proofs of the…

Using proof-theoretic methods in the style of proof mining, we give novel computationally effective limit theorems for the convergence of the Cesaro-means of certain sequences of random variables. These results are intimately related to…

Visual search, recommendation, and contrastive similarity learning power technologies that impact billions of users worldwide. Modern model architectures can be complex and difficult to interpret, and there are several competing techniques…

We refine Khintchine Transference Principle which relates the measure of simultaneous rational approximation of an $n$ real numbers with the measure of linear independence of these $n$ numbers. Khintchine's inequalities are known to be…

This monograph offers a toolbox of mathematical techniques, which have been effective and widely applicable in information-theoretic analysis. The first tool is a generalization of the method of types to Gaussian settings, and then to…

Computing similarity between two legal documents is an important and challenging task in the domain of Legal Information Retrieval. Finding similar legal documents has many applications in downstream tasks, including prior-case retrieval,…

Shannon entropy is the most crucial foundation of Information Theory, which has been proven to be effective in many fields such as communications. Renyi entropy and Chernoff information are other two popular measures of information with…

We show that many theorems which assert that two kinds of partitions of the same integer $n$ are equinumerous are actually special cases of a much stronger form of equality. We show that in fact there correspond partition statistics $X$ and…

It is known that many classical inequalities linked to convolutions can be obtained by looking at the monotonicity in time of convolutions of powers of solutions to the heat equation, provided that both the exponents and the coefficients of…

We report a proof of the quantum Sanov Theorem by elementary application of basic facts about representations of the symmetric group, together with a complete characterization of the optimal error exponent in a situation where the null…

This paper considers the problem of testing the equality of two unspecified distributions. The classical omnibus tests such as the Kolmogorov-Smirnov and Cram\`er-von Mises are known to suffer from low power against essentially all but…

We study equivalence, in the context of a variable diffusion problem, between (conforming) mixed methods and (primal) nonconforming methods defined on potentially general polytopal partitions. In this first paper of a series of two, we…

Claude Shannon coined entropy to quantify the uncertainty of a random distribution for communication coding theory. We observe that the uncertainty nature of entropy also limits its direct usage in mathematical modeling. Therefore we…

We exhibit infinitely many new, constrained inequalities for the von Neumann entropy, and show that they are independent of each other and the known inequalities obeyed by the von Neumann entropy (basically strong subadditivity). The new…

In recent years important progress has been achieved towards proving the validity of the replica predictions for the (asymptotic) mutual information (or "free energy") in Bayesian inference problems. The proof techniques that have emerged…

Based on an apparently new Lagrange-type identity, a Cauchy--Schwarz-type inequality is proved. The mentioned identity is obtained by using certain ``macro'' variables; it is hoped that such a method can be used to prove or produce other…