Related papers: Maximum Entropy on Compact Groups
Information theoretic measures (entropies, entropy rates, mutual information) are nowadays commonly used in statistical signal processing for real-world data analysis. The present work proposes the use of Auto Mutual Information (Mutual…
We consider random walks on a non-elementary hyperbolic group endowed with a word distance. To a probability measure on the group are associated two numerical quantities, the rate of escape and the entropy. On the set of admissible…
We present a general holistic theory for the organization of complex networks, both human-engineered and naturally-evolved. Introducing concepts of value of interactions and satisfaction as generic network performance measures, we show that…
It is always some constraint that yields any nontrivial structure from statistical averages. As epitomized by the Boltzmann distribution, the energy conservation is often the principal constraint acting on mechanical systems. Here, we…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
This paper addresses a fundamental problem in random variate generation: given access to a random source that emits a stream of independent fair bits, what is the most accurate and entropy-efficient algorithm for sampling from a discrete…
The classical Density Functional Theory (DFT) is introduced as an application of entropic inference for inhomogeneous fluids at thermal equilibrium. It is shown that entropic inference reproduces the variational principle of DFT when…
We generalize the well-known mixtures of Gaussians approach to density estimation and the accompanying Expectation--Maximization technique for finding the maximum likelihood parameters of the mixture to the case where each data point…
Entropy can signify different things: For instance, heat transfer in thermodynamics or a measure of information in data analysis. Many entropies have been introduced and it can be difficult to ascertain their different importance and…
Quantitative estimates are derived, on the whole space, for the relative entropy between the joint law of random interacting particles and the tensorized law at the limiting systeme. The developed method combines the relative entropy method…
Distributed consensus and other linear systems with system stochastic matrices $W_k$ emerge in various settings, like opinion formation in social networks, rendezvous of robots, and distributed inference in sensor networks. The matrices…
Mixture distributions are a workhorse model for multimodal data in information theory, signal processing, and machine learning. Yet even when each component density is simple, the differential entropy of the mixture is notoriously hard to…
Consider a measure-preserving transition kernel $T$ on an arbitrary probability space $(\mathbb X,\mathcal cA,\pi)$. In this level of generality, we prove that a one-step hyper-contractivity estimate of the form $\|T\|_{p\to q}\le 1$ with…
Existing training criteria in automatic speech recognition(ASR) permit the model to freely explore more than one time alignments between the feature and label sequences. In this paper, we use entropy to measure a model's uncertainty, i.e.…
The convergence rate is a crucial issue in opinion dynamics, which characterizes how quickly opinions reach a consensus and tells when the collective behavior can be formed. However, the key factors that determine the convergence rate of…
The entropy of network ensembles characterizes the amount of information encoded in the network structure, and can be used to quantify network complexity, and the relevance of given structural properties observed in real network datasets…
Recently, a new type of set, named as random permutation set (RPS), is proposed by considering all the permutations of elements in a certain set. For measuring the uncertainty of RPS, the entropy of RPS is presented. However, the maximum…
In reliability theory and survival analysis, observed data are often weakly dependent and subject to additive measurement errors. Such contamination arises when the underlying data are neither independent nor strongly mixed but instead…
A map is given showing that convolutions of independent random variables over a finite group and matrix multiplications of doubly stochastic matrices are homomorphic. As an application, a short proof is given to the theorem that the…
Hypergraphs are structures that can be decomposed or described; in other words they are recursively countable. Here, we get exact and asymptotic enumeration results on hypergraphs by means of exponential generating functions. The number of…