Related papers: Improved data analysis on two-point correlation fu…
To minimize the average of a set of log-convex functions, the stochastic Newton method iteratively updates its estimate using subsampled versions of the full objective's gradient and Hessian. We contextualize this optimization problem as…
Using quasi-Newton methods in stochastic optimization is not a trivial task given the difficulty of extracting curvature information from the noisy gradients. Moreover, pre-conditioning noisy gradient observations tend to amplify the noise.…
A new method is presented for the analysis of small angle neutron scattering data from quasi-2D systems such as flux lattices, Skyrmion lattices, and aligned liquid crystals. A significant increase in signal to noise ratio, and a natural…
We show that, for finite-sum minimization problems, incorporating partial second-order information of the objective function can dramatically improve the robustness to mini-batch size of variance-reduced stochastic gradient methods, making…
We present recent progress in calculating the semileptonic form factors $h_{A_1}(w)$ for the $\bar{B} \to D^{\ast} \ell \bar{\nu}$ decays. We use the Oktay-Kronfeld (OK) action for the charm and bottom valence quarks and the HISQ action for…
Quantum optics experiments, involving the measurement of low-probability photon events, are known to be extremely time-consuming. We present a new methodology for accelerating such experiments using physically-motivated ansatzes together…
A new, fast second-order method is proposed that achieves the optimal $\mathcal{O}\left(|\log(\epsilon)|\epsilon^{-3/2}\right)$ complexity to obtain first-order $\epsilon$-stationary points. Crucially, this is deduced without assuming the…
A new variant of Newton's method for empirical risk minimization is studied, where at each iteration of the optimization algorithm, the gradient and Hessian of the objective function are replaced by robust estimators taken from existing…
We propose and analyze several inexact regularized Newton-type methods for finding a global saddle point of convex-concave unconstrained min-max optimization problems. Compared to first-order methods, our understanding of second-order…
In this paper, we propose an inexact proximal Newton-type method for nonconvex composite problems. We establish the global convergence rate of the order $\mathcal{O}(k^{-1/2})$ in terms of the minimal norm of the KKT residual mapping and…
Minimizing loss functions is central to machine-learning training. Although first-order methods dominate practical applications, higher-order techniques such as Newton's method can deliver greater accuracy and faster convergence, yet are…
The analysis of second-order optimization methods based either on sub-sampling, randomization or sketching has two serious shortcomings compared to the conventional Newton method. The first shortcoming is that the analysis of the iterates…
We present two new remarkably simple stochastic second-order methods for minimizing the average of a very large number of sufficiently smooth and strongly convex functions. The first is a stochastic variant of Newton's method (SN), and the…
We consider minimization of a smooth nonconvex objective function using an iterative algorithm based on Newton's method and the linear conjugate gradient algorithm, with explicit detection and use of negative curvature directions for the…
Learning a good distance metric in feature space potentially improves the performance of the KNN classifier and is useful in many real-world applications. Many metric learning algorithms are however based on the point estimation of a…
The Poisson compound decision problem is a long-standing problem in statistics, where empirical Bayes methodologies are commonly used to estimate Poisson's means in static or batch domains. In this paper, we study the Poisson compound…
Parameter reconstructions are indispensable in metrology. Here, the objective is to to explain $K$ experimental measurements by fitting to them a parameterized model of the measurement process. The model parameters are regularly determined…
In this paper, we describe an algorithm and associated software package (sfit_minimize) for maximizing the likelihood function of a set of parameters by minimizing $\chi^2$. The key element of this method is that the algorithm estimates the…
In this paper, we consider stochastic second-order methods for minimizing a finite summation of nonconvex functions. One important key is to find an ingenious but cheap scheme to incorporate local curvature information. Since the true…
We present a new accelerated stochastic second-order method that is robust to both gradient and Hessian inexactness, which occurs typically in machine learning. We establish theoretical lower bounds and prove that our algorithm achieves…