Related papers: Generalizing Bottleneck Problems
We review a number a recent advances in the study of two-dimensional statistical models with strong geometrical constraints. These include folding problems of regular and random lattices as well as the famous meander problem of enumerating…
We propose a branch-and-bound algorithm for minimizing a bilinear functional of the form \[ f(X,Y) = \mathrm{tr}((X\otimes Y)Q)+\mathrm{tr}(AX)+\mathrm{tr}(BY) , \] of pairs of Hermitian matrices $(X,Y)$ restricted by joint semidefinite…
It is shown that the correlation functions of the random variables $\det(\lambda - X)$, in which $X$ is a real symmetric $ N\times N$ random matrix, exhibit universal local statistics in the large $N$ limit. The derivation relies on an…
Recently, Samorodnitsky proved a strengthened version of Mrs. Gerber's Lemma, where the output entropy of a binary symmetric channel is bounded in terms of the average entropy of the input projected on a random subset of coordinates. Here,…
Jensen's inequality is ubiquitous in measure and probability theory, statistics, machine learning, information theory and many other areas of mathematics and data science. It states that, for any convex function $f\colon K \to \mathbb{R}$…
The information bottleneck (IB) method seeks a compressed representation of data that preserves information relevant to a target variable for prediction while discarding irrelevant information from the original data. In its classical…
The presence of symmetries imposes a stringent set of constraints on a system. This constrained structure allows intelligent agents interacting with such a system to drastically improve the efficiency of learning and generalization, through…
We study several constrained variational problem in the 2-Wasserstein metric for which the set of probability densities satisfying the constraint is not closed. For example, given a probability density $F_0$ on $\R^d$ and a time-step $h>0$,…
This paper studies simple bilevel problems, where a convex upper-level function is minimized over the optimal solutions of a convex lower-level problem. We first show the fundamental difficulty of simple bilevel problems, that the…
Understanding the source of the superior generalization ability of NNs remains one of the most important problems in ML research. There have been a series of theoretical works trying to derive non-vacuous bounds for NNs. Recently, the…
We show that there are functions $f$ in the H\"older class $C^{ { \alpha }}[0,1]$, $1< { \alpha }<2$ such that $f|_{A}$ is not convex, nor concave for any $A { \subset } [0,1]$ with $ { \bar { dim }_M } A> { \alpha }-1$. Our earlier result…
We investigate the problem of estimating a random variable $Y\in \mathcal{Y}$ under a privacy constraint dictated by another random variable $X\in \mathcal{X}$, where estimation efficiency and privacy are assessed in terms of two different…
We investigate the Helmholtz equation with suitable boundary conditions and uncertainties in the wavenumber. Thus the wavenumber is modeled as a random variable or a random field. We discretize the Helmholtz equation using finite…
Many binary classification problems minimize misclassification above (or below) a threshold. We show that instances of ranking problems, accuracy at the top or hypothesis testing may be written in this form. We propose a general framework…
We consider the problem of parameter estimation in a Bayesian setting and propose a general lower-bound that includes part of the family of $f$-Divergences. The results are then applied to specific settings of interest and compared to other…
The thesis concentrates on two problems in discrete geometry, whose solutions are obtained by analytic, probabilistic and combinatoric tools. The first chapter deals with the strong polarization problem. This states that for any sequence…
We prove the Courtade-Kumar conjecture, for certain classes of $n$-dimensional Boolean functions, $\forall n\geq 2$ and for all values of the error probability of the binary symmetric channel, $\forall 0 \leq p \leq \frac{1}{2}$. Let…
Let $A$ be a set and $V$ a real Hilbert space. Let $H$ be a real Hilbert space of functions $f:A\to V$ and assume $H$ is continuously embedded in the Banach space of bounded functions. For $i=1,\cdots,n$, let $(x_i,y_i)\in A\times V$…
The rational solutions of the Painlev\'e-II equation appear in several applications and are known to have many remarkable algebraic and analytic properties. They also have several different representations, useful in different ways for…
Invariant risk minimization (IRM) has recently emerged as a promising alternative for domain generalization. Nevertheless, the loss function is difficult to optimize for nonlinear classifiers and the original optimization objective could…