Related papers: Analysis of Scale-Variant Robust Kernel Optimizati…
In this paper, the problem of target localization in the presence of outlying sensors is tackled. This problem is important in practice because in many real-world applications the sensors might report irrelevant data unintentionally or…
A distributed adaptive algorithm is proposed to solve a node-specific parameter estimation problem where nodes are interested in estimating parameters of local interest and parameters of global interest to the whole network. To address the…
Incremental learning aims to adapt to new sets of categories over time with minimal computational overhead. Prior work often addresses this task by training efficient task-specific adaptors that modify frozen layer weights or features to…
We present a novel adaptive random subspace learning algorithm (RSSL) for prediction purpose. This new framework is flexible where it can be adapted with any learning technique. In this paper, we tested the algorithm for regression and…
This paper studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a novel off-policy…
We address the numerical solution of minimal norm residuals of {\it nonlinear} equations in finite dimensions. We take inspiration from the problem of finding a sparse vector solution by using greedy algorithms based on iterative residual…
A two-stage batch estimation algorithm for solving a class of nonlinear, static parameter estimation problems that appear in aerospace engineering applications is proposed. It is shown how these problems can be recast into a form suitable…
We propose an adaptive refinement algorithm to solve total variation regularized measure optimization problems. The method iteratively constructs dyadic partitions of the unit cube based on i) the resolution of discretized dual problems and…
This paper presents adaptive observers for online state and parameter estimation of a class of nonlinear systems motivated by biophysical models of neuronal circuits. We first present a linear-in-the-parameters design that solves a…
The classical iteratively reweighted least-squares (IRLS) algorithm aims to recover an unknown signal from linear measurements by performing a sequence of weighted least squares problems, where the weights are recursively updated at each…
Many common loss functions such as mean-squared-error, cross-entropy, and reconstruction loss are unnecessarily rigid. Under a probabilistic interpretation, these common losses correspond to distributions with fixed shapes and scales. We…
Empirical scaling laws prescribe how to allocate parameters, data, and compute, while maximal-update parameterization ($\mu$P) enables learning-rate transfer across widths by equalizing early-time update magnitudes. However, in modern…
The local least squares estimator for a regression curve cannot provide optimal results when non-Gaussian noise is present. Both theoretical and empirical evidence suggests that residuals often exhibit distributional properties different…
The problem of least squares regression of a $d$-dimensional unknown parameter is considered. A stochastic gradient descent based algorithm with weighted iterate-averaging that uses a single pass over the data is studied and its convergence…
Many scientific and engineering applications require fitting regression models that are nonlinear in the parameters. Advances in computer hardware and software in recent decades have made it easier to fit such models. Relative to fitting…
This work presents a distributed algorithm for nonlinear adaptive learning. In particular, a set of nodes obtain measurements, sequentially one per time step, which are related via a nonlinear function; their goal is to collectively…
Principal component analysis (PCA) is often used to reduce the dimension of data by selecting a few orthonormal vectors that explain most of the variance structure of the data. L1 PCA uses the L1 norm to measure error, whereas the…
A novel regression method is introduced and studied. The procedure weights squared residuals based on their magnitude. Unlike the classic least squares which treats every squared residual equally important, the new procedure exponentially…
We develop a new least squares method for solving the second-order elliptic equations in non-divergence form. Two least-squares-type functionals are proposed for solving the equations in two steps. We first obtain a numerical approximation…
We present a novel algorithm which can overcome the drawbacks of the conventional linear scaling method with minimal computational overhead. This is achieved by introducing additional constraints, thus eliminating the redundancy of the…