Related papers: Saddle Point Approximation and Central Limit Theor…
We survey developments, over the last thirty years, in the theory of Shape Preserving Approximation (SPA) by algebraic polynomials on a finite interval. In this article, "shape" refers to (finitely many changes of) monotonicity, convexity,…
Many machine learning and optimization algorithms are built upon the framework of stochastic approximation (SA), for which the selection of step-size (or learning rate) $\{\alpha_n\}$ is crucial for success. An essential condition for…
We derive a Gaussian approximation result for the maximum of a sum of high-dimensional random vectors. Specifically, we establish conditions under which the distribution of the maximum is approximated by that of the maximum of a sum of the…
We consider stochastic optimization problems in multi-agent settings, where a network of agents aims to learn parameters which are optimal in terms of a global objective, while giving preference to locally observed streaming information. To…
This paper concerns a high-dimensional stochastic programming problem of minimizing a function of expected cost with a matrix argument. To this problem, one of the most widely applied solution paradigms is the sample average approximation…
Large-scale saddle-point problems arise in such machine learning tasks as GANs and linear models with affine constraints. In this paper, we study distributed saddle-point problems (SPP) with strongly-convex-strongly-concave smooth…
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such…
In many areas of applied statistics and machine learning, generating an arbitrary number of independent and identically distributed (i.i.d.) samples from a given distribution is a key task. When the distribution is known only through…
Probability density estimation is a core problem of statistics and signal processing. Moment methods are an important means of density estimation, but they are generally strongly dependent on the choice of feasible functions, which severely…
Central limit theorems (CLTs) for high-dimensional random vectors with dimension possibly growing with the sample size have received a lot of attention in the recent times. Chernozhukov et al. (2017) proved a Berry--Esseen type result for…
Sample average approximation (SAA) is a technique for obtaining approximate solutions to stochastic programs that uses the average from a random sample to approximate the expected value that is being optimized. Since the outcome from…
The Shannon sampling theorem for bandlimited wide sense stationary random processes was established in 1957, which and its extensions to various random processes have been widely studied since then. However, truncation of the Shannon series…
In this paper, we study the problem of escaping from saddle points in smooth nonconvex optimization problems subject to a convex set $\mathcal{C}$. We propose a generic framework that yields convergence to a second-order stationary point of…
In an earlier work we had considered a Gaussian ensemble of random matrices in the presence of a given external matrix source. The measure is no longer unitary invariant and the usual techniques based on orthogonal polynomials, or on the…
Assuming X is a random vector and A a non-invertible matrix, one sometimes need to perform inference while only having access to samples of Y = AX. The corresponding likelihood is typically intractable. One may still be able to perform…
\cite{HillMotegi2017} present a new general asymptotic theory for the maximum of a random array $\{\mathcal{X}_{n}(i)$ $:$ $1$ $\leq $ $i$ $\leq $ $\mathcal{L}\}_{n\geq 1}$, where each $\mathcal{X}_{n}(i)$ is assumed to converge in…
A central challenge to many fields of science and engineering involves minimizing non-convex error functions over continuous, high dimensional spaces. Gradient descent or quasi-Newton methods are almost ubiquitously used to perform such…
In this paper, we refine the Berry-Esseen bounds for the multivariate normal approximation of Polyak-Ruppert averaged iterates arising from the linear stochastic approximation (LSA) algorithm with decreasing step size. We consider the…
We study the maximum constraint satisfaction problem, Max-CSP, in the streaming setting. Given $n$ variables, the constraints arrive sequentially in an arbitrary order, with each constraint involving only a small subset of the variables.…
Nearest neighbor cells in $R^d,d\in\mathbb{N}$, are used to define coefficients of divergence ($\phi$-divergences) between continuous multivariate samples. For large sample sizes, such distances are shown to be asymptotically normal with a…