Related papers: A Matlab Program to Calculate the Maximum Entropy …
Maximum-Entropy Distributions offer an attractive family of probability densities suitable for moment closure problems. Yet finding the Lagrange multipliers which parametrize these distributions, turns out to be a computational bottleneck…
The problem of assigning probability distributions which objectively reflect the prior information available about experiments is one of the major stumbling blocks in the use of Bayesian methods of data analysis. In this paper the method of…
A common statistical situation concerns inferring an unknown distribution Q(x) from a known distribution P(y), where X (dimension n), and Y (dimension m) have a known functional relationship. Most commonly, n<m, and the task is relatively…
The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example.…
The method of Maximum (relative) Entropy (ME) is used to translate the information contained in the known form of the likelihood into a prior distribution for Bayesian inference. The argument is guided by intuition gained from the…
This paper shows how to evolve numerically the maximum entropy probability distributions for a given set of constraints, which is a variational calculus problem. An evolutionary algorithm can obtain approximations to some well-known…
The method of maximum entropy (ME) is extended to address the following problem: Once one accepts that the ME distribution is to be preferred over all others, the question is to what extent are distributions with lower entropy supposed to…
Maximum entropy modeling is a flexible and popular framework for formulating statistical models given partial knowledge. In this paper, rather than the traditional method of optimizing over the continuous density directly, we learn a smooth…
In this paper we study the problem of computing max-entropy distributions over a discrete set of objects subject to observed marginals. Interest in such distributions arises due to their applicability in areas such as statistical physics,…
We explore a method of statistical estimation called Maximum Entropy on the Mean (MEM) which is based on an information-driven criterion that quantifies the compliance of a given point with a reference prior probability measure. At the core…
We develop the method of Maximum Entropy (ME) as a technique to generate approximations to probability distributions. The central results consist in (a) justifying the use of relative entropy as the uniquely natural criterion to select a…
Reconstructing 4D or 6D phase space distributions from 1D or 2D measurements is a challenging inverse problem encountered in particle accelerators. Entropy maximization is an established method to incorporate prior information in the…
The montecarlo method, which is quite commonly used to solve maximum entropy problems in statistical physics, can actually be used to solve inverse problems in a much wider context. The probability distribution which maximizes entropy can…
The problem of determining the joint probability distributions for correlated random variables with pre-specified marginals is considered. When the joint distribution satisfying all the required conditions is not unique, the "most unbiased"…
Given a probability distribution ${\bf p} = (p_1, \dots, p_n)$ and an integer $1\leq m < n$, we say that ${\bf q} = (q_1, \dots, q_m)$ is a contiguous $m$-aggregation of ${\bf p}$ if there exist indices $0=i_0 < i_1 < \cdots < i_{m-1} < i_m…
Maximum entropy distributions with discrete support in $m$ dimensions arise in machine learning, statistics, information theory, and theoretical computer science. While structural and computational properties of max-entropy distributions…
Estimating the unconstrained mean and covariance matrix is a popular topic in statistics. However, estimation of the parameters of $N_p(\mu,\Sigma)$ under joint constraints such as $\Sigma\mu = \mu$ has not received much attention. It can…
The relaxed maximum entropy problem is concerned with finding a probability distribution on a finite set that minimizes the relative entropy to a given prior distribution, while satisfying relaxed max-norm constraints with respect to a…
Algebraic statistics is a recently evolving field, where one would treat statistical models as algebraic objects and thereby use tools from computational commutative algebra and algebraic geometry in the analysis and computation of…
Stochastic network models play a central role across a wide range of scientific disciplines, and questions of statistical inference arise naturally in this context. In this paper we investigate goodness-of-fit and two-sample testing…