Related papers: System Identification on the Families of Auto-Regr…

One way to understand time-series data is to identify the underlying dynamical system which generates it. This task can be done by selecting an appropriate model and a set of parameters which best fits the dynamics while providing the…

The so-called constrained least mean-square algorithm is one of the most commonly used linear-equality-constrained adaptive filtering algorithms. Its main advantages are adaptability and relative simplicity. In order to gain analytical…

We consider a general stochastic input-output dynamical system with output evolving in time as the solution to a functional coefficients, It\^{o}'s stochastic differential equation, excited by an input process. This general class of…

This work develops robust diffusion recursive least squares algorithms to mitigate the performance degradation often experienced in networks of agents in the presence of impulsive noise. The first algorithm minimizes an exponentially…

Linear Least Squares is a very well known technique for parameter estimation, which is used even when sub-optimal, because of its very low computational requirements and the fact that exact knowledge of the noise statistics is not required.…

In this work we investigate the practicality of stochastic gradient descent and recently introduced variants with variance-reduction techniques in imaging inverse problems. Such algorithms have been shown in the machine learning literature…

This paper presents a novel feature of the kernel-based system identification method. We prove that the regularized kernel-based approach for the estimation of a finite impulse response is equivalent to a robust least-squares problem with a…

This paper considers the order estimation problem of stochastic autoregressive exogenous input (ARX) systems by using quantized data. Based on the least squares algorithm and inspired by the control systems information criterion (CIC), a…

We pose the deterministic, nonparametric, approximation problem for scalar nonnegative input/output systems via finite impulse response convolutions, based on repeated observations of input/output signal pairs. The problem is converted into…

In [Steinerberger, Q. Appl. Math., 79:3, 419-429, 2021] and [Shao, SIAM J. Matrix Anal. Appl. 44(1), 212-239, 2023], two new types of Kaczmarz algorithms, which share some similarities, for consistent linear systems were proposed. These two…

We consider stochastic approximation for the least squares regression problem in the non-strongly convex setting. We present the first practical algorithm that achieves the optimal prediction error rates in terms of dependence on the noise…

Gaussian processes are a powerful framework for quantifying uncertainty and for sequential decision-making but are limited by the requirement of solving linear systems. In general, this has a cubic cost in dataset size and is sensitive to…

Although stochastic approximation learning methods have been widely used in the machine learning literature for over 50 years, formal theoretical analyses of specific machine learning algorithms are less common because stochastic…

Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past…

Algorithmic decision making process now affects many aspects of our lives. Standard tools for machine learning, such as classification and regression, are subject to the bias in data, and thus direct application of such off-the-shelf tools…

In this paper, we present the convergence analysis of proportionate-type least mean square (Pt-LMS) algorithm that identifies the sparse system effectively and more suitable for real time VLSI applications. Both first and second order…

We propose a new method for blind system identification. Resorting to a Gaussian regression framework, we model the impulse response of the unknown linear system as a realization of a Gaussian process. The structure of the covariance matrix…

Robust regression techniques rely on least-squares optimization, which works well for Gaussian noise but fails in the presence of asymmetric structured noise. We propose a hybrid neural-symbolic architecture where a transformer encoder…

This dissertation shows that careful injection of noise into sample data can substantially speed up Expectation-Maximization algorithms. Expectation-Maximization algorithms are a class of iterative algorithms for extracting maximum…

This work deals with the numerical approximation of backward stochastic differential equations (BSDEs). We propose a new algorithm which is based on the regression-later approach and the least squares Monte Carlo method. We give some…