Related papers: The Optimal 'AND'
In minimization models for image recovery and data analysis problems, loss functions and linear operators are typically aggregated as an average of composite terms. Each term in the aggregate models a desired property of the ideal solution…
By far the most common way to estimate an expected loss in machine learning is to draw samples, compute the loss on each one, and take the empirical average. However, sampling is not necessarily optimal. Given an MLP at initialization, we…
Optimum designs for parameter estimation in generalized regression models are standardly based on the Fisher information matrix (cf. Atkinson et al (2014) for a recent exposition). The corresponding optimality criteria are related to the…
Real-world problems, often couched as machine learning applications, involve quantities of interest that have real-world meaning, independent of any statistical model. To avoid potential model misspecification bias or over-complicating the…
In this paper, we propose the primal-dual method of multipliers (PDMM) for distributed optimization over a graph. In particular, we optimize a sum of convex functions defined over a graph, where every edge in the graph carries a linear…
In this paper we derive an explicit formula for calculating the marginal likelihood of a given factorization of a categorical dataset. Since the marginal likelihood is proportional to the posterior probability of the factorization, these…
In this article, we first derive an explicit expression for the marginal best linear invariant predictor (BLIP) of an unobserved future order statistic based on a set of early observed ordered statistics. We then derive the joint BLIPs of…
We propose a probability distribution for multivariate binary random variables. The probability distribution is expressed as principal minors of the parameter matrix, which is a matrix analogous to the inverse covariance matrix in the…
The joint likelihood is a simple extension of the standard likelihood formalism that enables the estimation of common parameters across disjoint datasets. Joining the likelihood, rather than the data itself, means nuisance parameters can be…
It is known that non-commuting observables in quantum mechanics do not have joint probability. This statement refers to the precise (additive) probability model. I show that the joint distribution of any non-commuting pair of variables can…
Probabilistic graphical models (PGMs) provide a compact representation of knowledge that can be queried in a flexible way: after learning the parameters of a graphical model once, new probabilistic queries can be answered at test time…
Estimating the joint probability mass function (PMF) of a set of random variables lies at the heart of statistical learning and signal processing. Without structural assumptions, such as modeling the variables as a Markov chain, tree, or…
In this paper, we describe a method for estimating the joint probability density from data samples by assuming that the underlying distribution can be decomposed as a mixture of product densities with few mixture components. Prior works…
We study the minimax estimation of $\alpha$-divergences between discrete distributions for integer $\alpha\ge 1$, which include the Kullback--Leibler divergence and the $\chi^2$-divergences as special examples. Dropping the usual…
We introduce the proximal optimal transport divergence, a novel discrepancy measure that interpolates between information divergences and optimal transport distances via an infimal convolution formulation. This divergence provides a…
Given a random sample from a distribution with density function that depends on an unknown parameter $\theta$, we are interested in accurately estimating the true parametric density function at a future observation from the same…
A striking result of [Acharya et al. 2017] showed that to estimate symmetric properties of discrete distributions, plugging in the distribution that maximizes the likelihood of observed multiset of frequencies, also known as the profile…
This article introduces a framework for evaluating statistical decisions under both prior ambiguity and likelihood misspecification. We begin with an ambiguity set - a frequentist model that pairs a possibly misspecified likelihood with…
Mutual information $I(X;Y)$ is a useful definition in information theory to estimate how much information the random variable $Y$ holds about the random variable $X$. One way to define the mutual information is by comparing the joint…
A coupling of two distributions $P_{X}$ and $P_{Y}$ is a joint distribution $P_{XY}$ with marginal distributions equal to $P_{X}$ and $P_{Y}$. Given marginals $P_{X}$ and $P_{Y}$ and a real-valued function $f$ of the joint distribution…