Related papers: A Comparison of First-Difference and Forward Ortho…
Forward gradient descent (FGD) has been proposed as a biologically more plausible alternative of gradient descent as it can be computed without backward pass. Considering the linear model with $d$ parameters, previous work has found that…
Goodness--of--fit tests for the distribution of the composed error term in a Stochastic Frontier Model (SFM) are suggested. The focus is on the case of a normal/gamma SFM and the heavy--tailed stable/gamma SFM. In the first case the moment…
The momentum acceleration technique is widely adopted in many optimization algorithms. However, there is no theoretical answer on how the momentum affects the generalization performance of the optimization algorithms. This paper studies…
We propose an Gaussian Mixture Model (GMM) learning algorithm, based on our previous work of GMM expansion idea. The new algorithm brings more robustness and simplicity than classic Expectation Maximization (EM) algorithm. It also improves…
We prove a moderate deviation principle for the continuous time interpolation of discrete time recursive stochastic processes. The methods of proof are somewhat different from the corresponding large deviation result, and in particular the…
We present a forward sufficient dimension reduction method for categorical or ordinal responses by extending the outer product of gradients and minimum average variance estimator to multinomial generalized linear model. Previous work in…
This paper studies the application of the generalized method of moments (GMM) to multi-reference alignment (MRA): the problem of estimating a signal from its circularly-translated and noisy copies. We begin by proving that the GMM estimator…
We introduce a new class of algorithms, Stochastic Generalized Method of Moments (SGMM), for estimation and inference on (overidentified) moment restriction models. Our SGMM is a novel stochastic approximation alternative to the popular…
We consider method-of-quantiles estimators of unknown parameters, namely the analogue of method-of-moments estimators obtained by matching empirical and theoretical quantiles at some probability level lambda in (0,1). The aim is to present…
We perform numerical analysis of a nonlinear gradient flow, which can be regarded as a parabolic minimal surface problem or a regularised total variation flow, using the gradient discretisation method (GDM). GDM is a unified convergence…
General first order methods (GFOMs), including various gradient descent and AMP algorithms, constitute a broad class of iterative algorithms in modern statistical learning problems. Some GFOMs also serve as constructive proof devices,…
Interior point methods solve small to medium sized problems to high accuracy in a reasonable amount of time. However, for larger problems as well as stochastic problems, one needs to use first-order methods such as stochastic gradient…
This paper analyses the forecasting performance of a new class of factor models with martingale difference errors (FMMDE) recently introduced by Lee and Shao (2018). The FMMDE makes it possible to retrieve a transformation of the original…
The computation of Gaussian orthant probabilities has been extensively studied for low-dimensional vectors. Here, we focus on the high-dimensional case and we present a two-step procedure relying on both deterministic and stochastic…
The Generalized Method of Moments (GMM) is a partition of unity based technique for solving electromagnetic and acoustic boundary integral equations. Past work on the GMM for electromagnetics was confined to geometries modeled by piecewise…
Finite difference schemes, using Backward Differentiation Formula (BDF), are studied for the approximation of one-dimensional diffusion equations with an obstacle term, of the form $$\min(v_t - a(t,x) v_{xx} + b(t,x) v_x + r(t,x) v, v-…
In this note, we consider the performance of the classic method of moments for parameter estimation of symmetric variance-gamma (generalized Laplace) distributions. We do this through both theoretical analysis (multivariate delta method)…
We consider estimation in moment condition models and show that under any bound on identification strength, asymptotically admissible (i.e. undominated) estimators in a wide class of estimation problems must be uniformly continuous in the…
Despite Adam demonstrating faster empirical convergence than SGD in many applications, much of the existing theory yields guarantees essentially comparable to those of SGD, leaving the empirical performance gap insufficiently explained. In…
This paper applies the gradient discretisation method (GDM) for fourth order elliptic variational inequalities. The GDM provides a new formulation of error estimates and a complete convergence analysis of several numerical methods. We show…