Related papers: Generalizing Bottleneck Problems

We compare two statistical models of three binary random variables. One is a mixture model and the other is a product of mixtures model called a restricted Boltzmann machine. Although the two models we study look different from their…

In this paper, we provide three applications for $f$-divergences: (i) we introduce Sanov's upper bound on the tail probability of the sum of independent random variables based on super-modular $f$-divergence and show that our generalized…

The problem of establishing out-of-sample bounds for the values of an unkonwn ground-truth function is considered. Kernels and their associated Hilbert spaces are the main formalism employed herein along with an observational model where…

We find a sharp combinatorial bound for the metric entropy of sets in R^n and general classes of functions. This solves two basic combinatorial conjectures on the empirical processes. 1. A class of functions satisfies the uniform Central…

We study the convolution function $$ C[f(x)] := \int_1^x f(y)f({x\over y}) {{\rm d} y\over y} $$ when $f(x)$ is a suitable number-theoretic error term. Asymptotics and upper bounds for $C[f(x)]$ are derived from mean square bounds for…

We will describe solvable lattice models whose partition functions depend on two sets of variables, $x_1,\cdots,x_n$ and $y_1, y_2, \cdots $ that have different connections with the representation theory of $\text{GL}(n,F)$ where $F$ is a…

We examine the relationship between the mutual information between the output model and the empirical sample and the generalization of the algorithm in the context of stochastic convex optimization. Despite increasing interest in…

Information bottleneck (IB) is a paradigm to extract information in one target random variable from another relevant random variable, which has aroused great interest due to its potential to explain deep neural networks in terms of…

We study an information theoretic privacy mechanism design problem for two scenarios where the private data is either observable or hidden. In each scenario, we first consider bounded mutual information as privacy leakage criterion, then we…

We provide lower error bounds for randomized algorithms that approximate integrals of functions depending on an unrestricted or even infinite number of variables. More precisely, we consider the infinite-dimensional integration problem on…

We consider upper exponential bounds for the probability of the event that an absolute deviation of sample mean from mathematical expectation p is bigger comparing with some ordered level epsilon. These bounds include 2 coefficients {alpha,…

In this paper, we propose a primal-dual algorithm with a novel momentum term using the partial gradients of the coupling function that can be viewed as a generalization of the method proposed by Chambolle and Pock in 2016 to solve saddle…

Bent functions, which are maximally nonlinear Boolean functions with even numbers of variables and whose Hamming distance to the set of all affine functions equals $2^{n-1}\pm 2^{\frac{n}{2}-1}$, were introduced by Rothaus in 1976 when he…

Shannon's Entropy Power Inequality can be viewed as characterizing the minimum differential entropy achievable by the sum of two independent random variables with fixed differential entropies. The entropy power inequality has played a key…

Convex optimization problems arise naturally in quantum information theory, often in terms of minimizing a convex function over a convex subset of the space of hermitian matrices. In most cases, finding exact solutions to these problems is…

Suppose $\alpha, \beta$ are Lipschitz strongly concave functions from $[0, 1]$ to $\mathbb{R}$ and $\gamma$ is a concave function from $[0, 1]$ to $\mathbb{R}$, such that $\alpha(0) = \gamma(0) = 0$, and $\alpha(1) = \beta(0) = 0$ and…

Given finite-dimensional random vectors $Y$, $X$, and $Z$ that form a Markov chain in that order (i.e., $Y \to X \to Z$), we derive upper bounds on the excess minimum risk using generalized information divergence measures. Here, $Y$ is a…

Finding upper and lower bounds to integrals with respect to copulas is a quite prominent problem in applied probability. In their 2014 paper, Hofer and Iaco showed how particular two dimensional copulas are related to optimal solutions of…

Let ($X,Y)$ be a random vector with distribution function $F(x,y),$ and $(X_{1},Y_{1}),(X_{2},Y_{2}),...,(X_{n},Y_{n})$ are independent copies of ($X,Y).$ Let $X_{i:n}$ be the $i$th order statistics constructed from the sample…

The addition of lower level integrality constraints to a bi-level linear program is known to result in significantly weaker analytical properties. Most notably, the upper level goal function in the optimistic setting lacks lower…