Related papers: Relative Entropy Minimization over Hilbert Spaces …
In this essay, we discuss the notion of optimal transport on geodesic measure spaces and the associated (2-)Wasserstein distance. We then examine displacement convexity of the entropy functional on the space of probability measures. In…
The increasing recognition of the association between adverse human health conditions and many environmental substances as well as processes has led to the need to monitor them. An important problem that arises in environmental statistics…
We establish the geometric ergodicity of the preconditioned Hamiltonian Monte Carlo (HMC) algorithm defined on an infinite-dimensional Hilbert space, as developed in [Beskos et al., Stochastic Process. Appl., 2011]. This algorithm can be…
We leverage the connections between nonexpansive maps, monotone Lipschitz operators, and proximal mappings to obtain near-optimal (i.e., optimal up to poly-log factors in terms of iteration complexity) and parameter-free methods for solving…
The problem of finding suitable point embedding or geometric configurations given only Euclidean distance information of point pairs arises both as a core task and as a sub-problem in a variety of machine learning applications. In this…
In the context of control and estimation under information constraints, restoration entropy measures the minimal required data rate above which the state of a system can be estimated so that the estimation quality does not degrade over time…
Although some information-theoretic measures of uncertainty or granularity have been proposed in rough set theory, these measures are only dependent on the underlying partition and the cardinality of the universe, independent of the lower…
We consider the following frustrated optimization problem: given a prior probability distribution $q$, find the distribution $p$ minimizing the relative entropy with respect to $q$ such that $\textrm{mean}(p)$ is fixed and large. We show…
We consider the problem of estimating the population probability distribution given a finite set of multivariate samples, using the maximum entropy approach. In strict keeping with Jaynes' original definition, our precise formulation of the…
Large scale optimization problems are ubiquitous in machine learning and data analysis and there is a plethora of algorithms for solving such problems. Many of these algorithms employ sub-sampling, as a way to either speed up the…
We propose a general method for optimization with semi-infinite constraints that involve a linear combination of functions, focusing on the case of the exponential function. Each function is lower and upper bounded on sub-intervals by…
We study convergence rates for Gibbs measures, with density proportional to $e^{-f(x)/t}$, as $t \rightarrow 0$ where $f : \mathbb{R}^d \rightarrow \mathbb{R}$ admits a unique global minimum at $x^\star$. We focus on the case where the…
In this paper we study constrained subspace approximation problem. Given a set of $n$ points $\{a_1,\ldots,a_n\}$ in $\mathbb{R}^d$, the goal of the {\em subspace approximation} problem is to find a $k$ dimensional subspace that best…
We solve the problem of best approximation by partial isometries of given rank to an arbitrary rectangular matrix, when the distance is measured in any unitarily invariant norm. In the case where the norm is strictly convex, we parametrize…
We re-examine the notion of relative $(p,\eps)$-approximations, recently introduced in [CKMS06], and establish upper bounds on their size, in general range spaces of finite VC-dimension, using the sampling theory developed in [LLS01] and in…
We propose a mixedness quantifier based on entropy fluctuations. It provides information about the degree of mixedness either for finite dimensional and infinite dimensional Hilbert spaces. It may be used to determine the reduction of the…
Efficient global optimization is a widely used method for optimizing expensive black-box functions such as tuning hyperparameter, and designing new material, etc. Despite its popularity, less attention has been paid to analyzing the…
Non-convex methods for linear inverse problems with low-dimensional models have emerged as an alternative to convex techniques. We propose a theoretical framework where both finite dimensional and infinite dimensional linear inverse…
The maximum a-posteriori (MAP) perturbation framework has emerged as a useful approach for inference and learning in high dimensional complex models. By maximizing a randomly perturbed potential function, MAP perturbations generate unbiased…
This is the lecture notes on the interplay between optimal transport and Riemannian geometry. On a Riemannian manifold, the convexity of entropy along optimal transport in the space of probability measures characterizes lower bounds of the…