Related papers: On Sharp Stochastic Zeroth Order Hessian Estimator…

We consider an on-line least squares regression problem with optimal solution $\theta^*$ and Hessian matrix H, and study a time-average stochastic gradient descent estimator of $\theta^*$. For $k\ge2$, we provide an unbiased estimator of…

Let ${\mathcal P}$ be a family of probability measures on a measurable space $(S,{\mathcal A}).$ Given a Banach space $E,$ a functional $f:E\mapsto {\mathbb R}$ and a mapping $\theta: {\mathcal P}\mapsto E,$ our goal is to estimate…

We develop a general framework for generating estimators of a given quantity which are unbiased to a given order in the difference between the true value of the underlying quantity and the fiducial position in theory space around which we…

Let M be a smooth compact oriented manifold without boundary, imbedded in a euclidean space E and let f be a smooth map of M into a Riemannian manifold N. An unknown state x in M is observed via X=x+su where s>0 is a small parameter and u…

Let \Theta be a smooth compact oriented manifold without boundary, embedded in a euclidean space and let \gamma be a smooth map \Theta into a riemannian manifold \Lambda. An unknown state \theta \in \Theta is observed via X=\theta+\epsilon…

We present a finite-time analysis of two smoothed functional stochastic approximation algorithms for simulation-based optimization. The first is a two time-scale gradient-based method, while the second is a three time-scale Newton-based…

This paper presents the first optimal-rate $p$-th order methods with $p\geq 1$ for finding first and second-order stationary points of non-convex smooth objective functions over Riemannian manifolds. In contrast to the geodesically convex…

We study the properties of stochastic approximation applied to a tame nondifferentiable function subject to constraints defined by a Riemannian manifold. The objective landscape of tame functions, arising in o-minimal topology extended to a…

We propose a stochastic variance-reduced cubic regularized Newton algorithm to optimize the finite-sum problem over a Riemannian submanifold of the Euclidean space. The proposed algorithm requires a full gradient and Hessian update at the…

We study stochastic zeroth-order optimization with decision-dependent distributions, where the sampling law depends on the current decision and only noisy function values are available. For the non-smooth non-convex setting, we establish an…

We study the problem of estimating a manifold from random samples. In particular, we consider piecewise constant and piecewise linear estimators induced by k-means and k-flats, and analyze their performance. We extend previous results for…

In this paper, we establish the second order estimates of solutions to the first initial-boundary value problem for general Hessian type fully nonlinear parabolic equations on Riemannian manifolds. The techniques used in this article can…

We study online inference and asymptotic covariance estimation for the stochastic gradient descent (SGD) algorithm. While classical methods (such as plug-in and batch-means estimators) are available, they either require inaccessible…

In this work, we present a novel approach for solving stochastic shape optimization problems. Our method is the extension of the classical stochastic gradient method to infinite-dimensional shape manifolds. We prove convergence of the…

We consider Hessian quotient equations in Riemannian setting related to a problem posed by Delano\"e and Urbas. We prove unobstructed second order a priori estimate for the real Hessian quotient equation via the maximum principle argument…

The motivation of this paper is to study a second order elliptic operator which appears naturally in Riemannian geometry, for instance in the study of hypersurfaces with constant $r$-mean curvature. We prove a generalized Bochner-type…

Given a non-oscillating gradient trajectory G of a real analytic function f, we show that the limit v of the secants at the limit point O of G along the trajectory G is an eigen-vector of the limit of the direction of the Hessian matrix…

We present a new accelerated stochastic second-order method that is robust to both gradient and Hessian inexactness, which occurs typically in machine learning. We establish theoretical lower bounds and prove that our algorithm achieves…

Riemannian optimization is concerned with problems, where the independent variable lies on a smooth manifold. There is a number of problems from numerical linear algebra that fall into this category, where the manifold is usually specified…

We are concerned with stochastic processes on surfaces in three-dimensional contact sub-Riemannian manifolds. Employing the Riemannian approximations to the sub-Riemannian manifold which make use of the Reeb vector field, we obtain a second…