Related papers: A Bayesian Proof and Interpretation of Talagrand's…

Despite the ubiquitous use of stochastic optimization algorithms in machine learning, the precise impact of these algorithms and their dynamics on generalization performance in realistic non-convex settings is still poorly understood. While…

We import the algebro-geometric notion of a complete collineation into the study of maximum likelihood estimation in directed Gaussian graphical models. A complete collineation produces a perturbation of sample data, which we call a…

A fundamental tool in network information theory is the covering lemma, which lower bounds the probability that there exists a pair of random variables, among a give number of independently generated candidates, falling within a given set.…

The exact maximum likelihood estimate (MLE) provides a test statistic for the unit root test that is more powerful \citep[p. 577]{Fuller96} than the usual least squares approach. In this paper a new derivation is given for the asymptotic…

In Bayesian inference, a widespread technique to compute integrals against a high-dimensional posterior is to use a Gaussian proxy to the posterior known as the Laplace approximation. We address the question of accuracy of the approximation…

We study a parametric estimation problem related to moment condition models. As an alternative to the generalized empirical likelihood (GEL) and the generalized method of moments (GMM), a Bayesian approach to the problem can be adopted,…

Recently, the precise performance of the Generalized LASSO algorithm for recovering structured signals from compressed noisy measurements, obtained via i.i.d. Gaussian matrices, has been characterized. The analysis is based on a framework…

An alternative to extrinsic information transfer (EXIT) charts called mean squared error (MSE) charts that use a measure related to the MSE instead of mutual information is proposed. Using the relationship between mutual information and…

The Maximum Entropy Method (MEM) is a popular data analysis technique based on Bayesian inference, which has found various applications in the research literature. While the MEM itself is well-grounded in statistics, I argue that its…

In parameter estimation, assumptions about the model are typically considered which allow us to build optimal estimation methods under many statistical senses. However, it is usually the case where such models are inaccurately known or not…

We show that certain statements related to the Fourier-Walsh expansion of functions with respect to a biased measure on the discrete cube can be deduced from the respective results for the uniform measure by a simple reduction. In…

We propose an adversarial evaluation framework for sensitive feature inference based on minimum mean-squared error (MMSE) estimation with a finite sample size and linear predictive models. Our approach establishes theoretical lower bounds…

We provide numerical indications of the $q$-generalised central limit theorem that has been conjectured (Tsallis 2004) in nonextensive statistical mechanics. We focus on $N$ binary random variables correlated in a {\it scale-invariant} way.…

Whitening is a classical technique in unsupervised learning that can facilitate estimation tasks by standardizing data. An important application is the estimation of latent variable models via the decomposition of tensors built from…

Traditionally, Multi-task Learning (MTL) models optimize the average of task-related objective functions, which is an intuitive approach and which we will be referring to as Average MTL. However, a more general framework, referred to as…

This note is concerned with an extension, at second order, of an inequality on the discrete cube $C_n=\{-1,1\}$ (equipped with the uniform measure) due to Talagrand (\cite{TalL1L2}). As an application, the main result of this note is a…

In this paper, we will discuss how to generalize nonparametric density estimators to MLE parametric estimators. Basing on the Parzen window theory and using the advantages of probability amplitude of quantum theory, we model a nonlinear…

In constrained stochastic optimization, one naturally expects that imposing a stricter feasible set does not increase the statistical risk of an estimator defined by projection onto that set. In this paper, we show that this intuition can…

Let $A$ be an isotropic, sub-gaussian $m \times n$ matrix. We prove that the process $Z_x := \|Ax\|_2 - \sqrt m \|x\|_2$ has sub-gaussian increments. Using this, we show that for any bounded set $T \subseteq \mathbb{R}^n$, the deviation of…

Let $(\mathfrak{M},\rho,\mu)$ be a metric measure space satisfying a doubling condition, $p_0\in (1,\infty)$, and $T(t):L^{p_0}(\mathfrak{M},\mu)\rightarrow L^{p_0}(\mathfrak{M},\mu)$, $t\geq 0$, a strongly continuous semi-group. We provide…