Related papers: Maximum Entropy Interval Aggregations
Given two discrete random variables $X$ and $Y,$ with probability distributions ${\bf p}=(p_1, \ldots , p_n)$ and ${\bf q}=(q_1, \ldots , q_m)$, respectively, denote by ${\cal C}({\bf p}, {\bf q})$ the set of all couplings of ${\bf p}$ and…
A common statistical situation concerns inferring an unknown distribution Q(x) from a known distribution P(y), where X (dimension n), and Y (dimension m) have a known functional relationship. Most commonly, n<m, and the task is relatively…
Given two discrete random variables $X$ and $Y$, with probability distributions ${\bf p} =(p_1, \ldots , p_n)$ and ${\bf q}=(q_1, \ldots , q_m)$, respectively, denote by ${\cal C}({\bf p}, {\bf q})$ the set of all couplings of ${\bf p}$ and…
The problem of determining the joint probability distributions for correlated random variables with pre-specified marginals is considered. When the joint distribution satisfying all the required conditions is not unique, the "most unbiased"…
Given a collection of probability distributions $p_{1},\ldots,p_{m}$, the minimum entropy coupling is the coupling $X_{1},\ldots,X_{m}$ ($X_{i}\sim p_{i}$) with the smallest entropy $H(X_{1},\ldots,X_{m})$. While this problem is known to be…
The classical Maximum Entropy (ME) problem consists of determining a probability distribution function (pdf) from a finite set of expectations of known functions. The solution depends on $N+1$ Lagrange multipliers which are determined by…
Abstract: The number of points $x=(x_1 ,x_2 ,...x_n)$ that lie in an integer cube $C$ in $R^n$ and satisfy the constraints $\sum_j h_{ij}(x_j )=s_i ,1\le i\le d$ is approximated by an Edgeworth-corrected Gaussian formula based on the…
Maximum entropy distributions with discrete support in $m$ dimensions arise in machine learning, statistics, information theory, and theoretical computer science. While structural and computational properties of max-entropy distributions…
We describe a maximum entropy approach for computing volumes and counting integer points in polyhedra. To estimate the number of points from a particular set X in R^n in a polyhedron P in R^n, by solving a certain entropy maximization…
Maximum entropy modeling is a flexible and popular framework for formulating statistical models given partial knowledge. In this paper, rather than the traditional method of optimizing over the continuous density directly, we learn a smooth…
In this paper we study the problem of computing max-entropy distributions over a discrete set of objects subject to observed marginals. Interest in such distributions arises due to their applicability in areas such as statistical physics,…
Entropy maximization procedure has been a general practice in many diverse fields of science to obtain the concomitant probability distributions. The consistent use of the maximization procedure on the other hand requires the probability…
Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order…
Maximum entropy inference and learning of graphical models are pivotal tasks in learning theory and optimization. This work extends algorithms for these problems, including generalized iterative scaling (GIS) and gradient descent (GD), to…
The classical problem of maximizing the Shannon entropy of a sum of independent random variables supported on a finite alphabet is considered and settled in the ternary case. Namely, the following theorem is established: if…
We consider distributions of ordered random vectors with given one-dimensional marginal distributions. We give an elementary necessary and sufficient condition for the existence of such a distribution with finite entropy. In this case, we…
We study maximum-entropy inference for finite-dimensional quantum states under linear moment constraints. Given expectation values of finitely many observables, the feasible set of states is convex but typically non-unique. The…
The problem of assigning probability distributions which objectively reflect the prior information available about experiments is one of the major stumbling blocks in the use of Bayesian methods of data analysis. In this paper the method of…
The classical problem of moments is addressed by the maximum entropy approach for one-dimensional discrete distributions. The numerical technique of adaptive support approximation is proposed to reconstruct the distributions in the region…
Given a large real symmetric, positive semidefinite m-by-m matrix, the goal of this paper is to show how a numerical approximation of the entropy, given by the sum of the entropies of the individual eigenvalues, can be computed in an…