Related papers: Error bounds of constant gain least-mean-squares a…
The so-called constrained least mean-square algorithm is one of the most commonly used linear-equality-constrained adaptive filtering algorithms. Its main advantages are adaptability and relative simplicity. In order to gain analytical…
Convergence of Extremum Seeking (ES) algorithms has been established in the limit of small gains. Using averaging theory and contraction analysis, we propose a framework for computing explicit bounds on the departure of the ES scheme from…
We study the effect of fading in the communication channels between nodes on the performance of the incremental least mean square (ILMS) algorithm. We derive steady-state performance metrics, including the mean-square deviation (MSD),…
A moment bound for the normalized conditional-sum-of-squares (CSS) estimate of a general autoregressive fractionally integrated moving average (ARFIMA) model with an arbitrary unknown memory parameter is derived in this paper. To achieve…
Consider the problem of estimating the mean of a Gaussian random vector when the mean vector is assumed to be in a given convex set. The most natural solution is to take the Euclidean projection of the data vector on to this convex set; in…
We study the effect of fading in the communication channels between sensor nodes on the performance of the incremental least mean square (ILMS) algorithm, and derive steady state performance metrics, including the mean-square deviation…
Most studies of adaptive algorithm behavior consider performance measures based on mean values such as the mean-square error. The derived models are useful for understanding the algorithm behavior under different environments and can be…
The stochastic gradient descent (SGD) optimization algorithm plays a central role in a series of machine learning applications. The scientific literature provides a vast amount of upper error bounds for the SGD method. Much less attention…
In large-scale data processing scenarios, data often arrive in sequential streams generated by complex systems that exhibit drifting distributions and time-varying system parameters. This nonstationarity challenges theoretical analysis, as…
The least-mean-squares (LMS) algorithm is the most popular algorithm in adaptive filtering. Several variable step-size strategies have been suggested to improve the performance of the LMS algorithm. These strategies enhance the performance…
As one of the recently proposed algorithms for sparse system identification, $l_0$ norm constraint Least Mean Square ($l_0$-LMS) algorithm modifies the cost function of the traditional method with a penalty of tap-weight sparsity. The…
We consider least squares estimation in a general nonparametric regression model. The rate of convergence of the least squares estimator (LSE) for the unknown regression function is well studied when the errors are sub-Gaussian. We find…
This paper concerns error bounds for recursive equations subject to Markovian disturbances. Motivating examples abound within the fields of Markov chain Monte Carlo (MCMC) and Reinforcement Learning (RL), and many of these algorithms can be…
The least mean-square (LMS) filter is one of the most common adaptive linear estimation algorithms. In many practical scenarios, and particularly in digital communications systems, the signal of interest (SOI) and the input signal are…
Error bounds, which refer to inequalities that bound the distance of vectors in a test set to a given set by a residual function, have proven to be extremely useful in analyzing the convergence rates of a host of iterative methods for…
Online averaged stochastic gradient algorithms are more and more studied since (i) they can deal quickly with large sample taking values in high dimensional spaces, (ii) they enable to treat data sequentially, (iii) they are known to be…
This article presents the formulation and steady-state analysis of the distributed estimation algorithms based on the diffusion cooperation scheme in the presence of errors due to the unreliable data transfer among nodes. In particular, we…
In this paper, we consider a least-squares (LS)-based distributed algorithm build on a sensor network to estimate an unknown parameter vector of a dynamical system, where each sensor in the network has partial information only but is…
Using a support vector machine requires to set two types of hyperparameters: the soft margin parameter C and the parameters of the kernel. To perform this model selection task, the method of choice is cross-validation. Its leave-one-out…
The optimization foundations of deep linear networks have recently received significant attention. However, due to their inherent non-convexity and hierarchical structure, analyzing the loss functions of deep linear networks remains a…