Related papers: A Note on Reverse Pinsker Inequalities
This paper proposes a unifying variational approach for proving and extending some fundamental information theoretic inequalities. Fundamental information theory results such as maximization of differential entropy, minimization of Fisher…
In this paper we establish lower bounds on information divergence from a distribution to certain important classes of distributions as Gaussian, exponential, Gamma, Poisson, geometric, and binomial. These lower bounds are tight and for…
Inference methods are often formulated as variational approximations: these approximations allow easy evaluation of statistics by marginalization or linear response, but these estimates can be inconsistent. We show that by introducing…
The Fisher information matrix (FIM) plays an important role in the analysis of parameter inference and system design problems. In a number of cases, however, the statistical data distribution and its associated information matrix are either…
An adaptive proximal method for a special class of variational inequalities and related problems is proposed. For example, the so-called mixed variational inequalities and composite saddle problems are considered. Some estimates of the…
Tight bounds for several symmetric divergence measures are introduced, given in terms of the total variation distance. Each of these bounds is attained by a pair of 2 or 3-element probability distributions. An application of these bounds…
Neural networks are popular state-of-the-art models for many different tasks.They are often trained via back-propagation to find a value of the weights that correctly predicts the observed data. Although back-propagation has shown good…
Pinsker's inequality sets a lower bound on the Umegaki divergence of two quantum states in terms of their trace distance. In this work, we formulate corresponding estimates for a variety of quantum and classical divergences including…
A novel approach is proposed to establish a sharp upper bound on the expected supremum of a separable martingale random field, serving as an alternative to classical universal chaining-based methods. The proposed approach begins by deriving…
We study a unified approach and algorithm for constructive discrepancy minimization based on a stochastic process. By varying the parameters of the process, one can recover various state-of-the-art results. We demonstrate the flexibility of…
In this article we consider the problem of choosing an optimal sampling scheme for the regression problem simultaneously with that of model selection. We consider a batch type approach and an on-line approach following algorithms recently…
Worst-case bounds on the expected shortfall risk given only limited information on the distribution of the random variables has been studied extensively in the literature. In this paper, we develop a new worst-case bound on the expected…
This paper considers the problem of guessing the realization of a finite alphabet source when some side information is provided. The only knowledge the guesser has about the source and the correlated side information is that the joint…
Known Bernstein-type upper bounds on the tail probabilities for sums of independent zero-mean sub-exponential random variables are improved in several ways at once. The new upper bounds have a certain optimality property.
In this work, we characterize the existence of solution for a certain variational inequality by means of a classical minimax theorem. In addition, we propose a numerical algorithm for the solution of an inverse problem associated with a…
We consider an adaptive finite element method with arbitrary but fixed polynomial degree $p \ge 1$, where adaptivity is driven by an edge-based residual error estimator. Based on the modified maximum criterion from [Diening et al, Found.…
Variational inequalities are a formalism that includes games, minimization, saddle point, and equilibrium problems as special cases. Methods for variational inequalities are therefore universal approaches for many applied tasks, including…
Convergence results are stated for the variational iteration method applied to solve an initial value problem for a system of ordinary differential equations.
This work establishes computable bounds between f-divergences for probability measures within a generalized quasi-$\varepsilon_{(M,m)}$-neighborhood framework. We make the following key contributions. (1) a unified characterization of local…
This paper is devoted to the variational inequality problems. We consider two classes of problems, the first is classical constrained variational inequality and the second is the same problem with functional (inequality type) constraints.…