Related papers: Maximum Entropy on Compact Groups
Although some information-theoretic measures of uncertainty or granularity have been proposed in rough set theory, these measures are only dependent on the underlying partition and the cardinality of the universe, independent of the lower…
Given a point $A$ in the convex hull of a given adjoint orbit $\mathcal{O}(F)$ of a compact Lie group $G$, we give a polynomial time algorithm to compute the probability density supported on $\mathcal{O}(F)$ whose expectation is $A$ and…
The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example.…
We consider powers of the absolute value of the characteristic polynomial of Haar distributed random orthogonal or symplectic matrices, as well as powers of the exponential of its argument, as a random measure on the unit circle minus small…
Sufficient conditions are developed, under which the compound Poisson distribution has maximal entropy within a natural class of probability measures on the nonnegative integers. Recently, one of the authors [O. Johnson, {\em Stoch. Proc.…
A formulation of the density functional theory is constructed on the foundations of entropic inference. The theory is introduced as an application of maximum entropy for inhomogeneous fluids in thermal equilibrium. It is shown that entropic…
We show that the maximum expected inner product between a random vector and the standard normal vector over all couplings subject to a mutual information constraint or regularization is equivalent to a truncated integral involving the…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
The maximum entropy principle advocates to evaluate events' probabilities using a distribution that maximizes entropy among those that satisfy certain expectations' constraints. Such principle can be generalized for arbitrary decision…
The concept of Entropy plays a key role in Information Theory, Statistics, and Machine Learning.This paper introduces a new entropy measure, called the t-entropy, which exploits the concavity of the inverse-tan function. We analytically…
The fundamental inequality of Guivarc'h relates the entropy and the drift of random walks on groups. It is strict if and only if the random walk does not behave like the uniform measure on balls. We prove that, in any nonelementary…
We apply the Principle of Maximum Entropy to the study of a general class of deterministic fractal sets. The scaling laws peculiar to these objects are accounted for by means of a constraint concerning the average content of information in…
We develop entropy and variance results for the product of independent identically distributed random variables on Lie groups. Our results apply to the study of stationary measures in various contexts.
The principle of entropy increase is not only the basis of statistical mechanics, but also closely related to the irreversibility of time, the origin of life, chaos and turbulence. In this paper, we first discuss the dynamic system…
It is known that the fluctuations of suitable linear statistics of Haar distributed elements of the compact classical groups satisfy a central limit theorem. We show that if the corresponding test functions are sufficiently smooth, a rate…
The profile of a sample is the multiset of its symbol frequencies. We show that for samples of discrete distributions, profile entropy is a fundamental measure unifying the concepts of estimation, inference, and compression. Specifically,…
Organising the relevant literature and by letting statistical convergence play the main role in the theory of compactness, a variant of compactness called statistical compactness has been achieved. As in case of sequential compactness, one…
We uncover connections between maximum likelihood estimation in statistics and norm minimization over a group orbit in invariant theory. We focus on Gaussian transformation families, which include matrix normal models and Gaussian graphical…
Formalising the confrontation of opinions (models) to observations (data) is the task of Inferential Statistics. Information Theory provides us with a basic functional, the relative entropy (or Kullback-Leibler divergence), an asymmetrical…
The field of complex networks studies a wide variety of interacting systems by representing them as networks. To understand their properties and mutual relations, the randomisation of network connections is a commonly used tool. However,…