Related papers: Least Squares Shadowing method for sensitivity ana…
Developing efficient methods for solving parametric partial differential equations is crucial for addressing inverse problems. This work introduces a Least-Squares-based Neural Network (LS-Net) method for solving linear parametric PDEs. It…
We study the dynamics of a continuous-time model of the Stochastic Gradient Descent (SGD) for the least-square problem. Indeed, pursuing the work of Li et al. (2019), we analyze Stochastic Differential Equations (SDEs) that model SGD either…
Instead of minimizing the sum of all $n$ squared residuals as the classical least squares (LS) does, Rousseeuw (1984) proposed to minimize the sum of $h$ ($n/2 \leq h < n$) smallest squared residuals, the resulting estimator is called least…
We consider stochastic approximation for the least squares regression problem in the non-strongly convex setting. We present the first practical algorithm that achieves the optimal prediction error rates in terms of dependence on the noise…
A distributed discrete-time algorithm is proposed for multi-agent networks to achieve a common least squares solution of a group of linear equations, in which each agent only knows some of the equations and is only able to receive…
Sensitivity analysis methods are important tools for research and design with simulations. Many important simulations exhibit chaotic dynamics, including scale-resolving turbulent fluid flow simulations. Unfortunately, conventional…
We consider an on-line least squares regression problem with optimal solution $\theta^*$ and Hessian matrix H, and study a time-average stochastic gradient descent estimator of $\theta^*$. For $k\ge2$, we provide an unbiased estimator of…
We present a frequency-domain method for computing the sensitivities of time-averaged quantities of chaotic systems with respect to input parameters. Such sensitivities cannot be computed by conventional adjoint analysis tools, because the…
In this paper, we study the distributed adaptive estimation problem of continuous-time stochastic dynamic systems over sensor networks where each agent can only communicate with its local neighbors. A distributed least squares (LS)…
We present a new finite-time analysis of the estimation error of the Ordinary Least Squares (OLS) estimator for stable linear time-invariant systems. We characterize the number of observed samples (the length of the observed trajectory)…
We consider the least-squares finite element method (lsfem) for systems of nonlinear ordinary differential equations and establish an optimal error estimate for this method when piecewise linear elements are used. The main assumptions are…
We present a non-conforming least squares method for approximating solutions of second order elliptic problems with discontinuous coefficients. The method is based on a general Saddle Point Least Squares (SPLS) method introduced in previous…
Least squares estimation, a regression technique based on minimisation of residuals, has been invaluable in bringing the best fit solutions to parameters in science and engineering. However, in dynamic environments such as in Geomatics…
Computational methods for sensitivity analysis are invaluable tools for scientists and engineers investigating a wide range of physical phenomena. However, many of these methods fail when applied to chaotic systems, such as the…
Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an $l_0$-constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm…
A discrete-time linear dynamical system (LDS) is given by an update matrix $M \in \mathbb{R}^{d\times d}$, and has the trajectories $\langle s, Ms, M^2s, \ldots \rangle$ for $s \in \mathbb{R}^d$. Reachability-type decision problems of…
We consider the problem of optimizing the sum of a smooth, nonconvex function for which derivatives are unavailable, and a convex, nonsmooth function with easy-to-evaluate proximal operator. Of particular focus is the case where the smooth…
In this paper, we study the \emph{sparse integer least squares problem} (SILS), an NP-hard variant of least squares with sparse $\{0, \pm 1\}$-vectors. We propose an $\ell_1$-based SDP relaxation, and a randomized algorithm for SILS, which…
For multiple treatments D=0,1,...,J, covariates X and outcome Y, the ordinary least squares estimator (OLS) of Y on (D1,...,DJ,X) is widely applied to a constant-effect linear model, where Dj is the dummy variable for D=j. However, the…
Many computer vision problems (e.g., camera calibration, image alignment, structure from motion) are solved with nonlinear optimization methods. It is generally accepted that second order descent methods are the most robust, fast, and…