Related papers: Properties of the Least Squares Temporal Differenc…

Stochastic gradient descent (SGD) is a popular algorithm for minimizing objective functions that arise in machine learning. For constant step-sized SGD, the iterates form a Markov chain on a general state space. Focusing on a class of…

We study episodic reinforcement learning in non-stationary linear (a.k.a. low-rank) Markov Decision Processes (MDPs), i.e, both the reward and transition kernel are linear with respect to a given feature map and are allowed to evolve either…

Multi Task Learning (MTL) efficiently leverages useful information contained in multiple related tasks to help improve the generalization performance of all tasks. This article conducts a large dimensional analysis of a simple but, as we…

We investigate the problem of optimal control synthesis for Markov Decision Processes (MDPs), addressing both qualitative and quantitative objectives. Specifically, we require the system to satisfy a qualitative task specified by a Linear…

Temporal difference learning (TD) is a foundational concept in reinforcement learning (RL), aimed at efficiently assessing a policy's value function. TD($\lambda$), a potent variant, incorporates a memory trace to distribute the prediction…

We study computational and statistical aspects of learning Latent Markov Decision Processes (LMDPs). In this model, the learner interacts with an MDP drawn at the beginning of each epoch from an unknown mixture of MDPs. To sidestep known…

Least squares approximation is a technique to find an approximate solution to a system of linear equations that has no exact solution. In a typical setting, one lets $n$ be the number of constraints and $d$ be the number of variables, with…

This paper aims to present a local discontinuous Galerkin (LDG) method for solving backward stochastic partial differential equations (BSPDEs) with Neumann boundary conditions. We establish the $L^2$-stability and optimal error estimates of…

Gradient temporal difference (Gradient TD) algorithms are a popular class of stochastic approximation (SA) algorithms used for policy evaluation in reinforcement learning. Here, we consider Gradient TD algorithms with an additional heavy…

In this paper, we study the distributed adaptive estimation problem of continuous-time stochastic dynamic systems over sensor networks where each agent can only communicate with its local neighbors. A distributed least squares (LS)…

A space-time-parameters structure of parametric parabolic PDEs motivates the application of tensor methods to define reduced order models (ROMs). Within a tensor-based ROM framework, the matrix SVD - a traditional dimension reduction…

TD($\lambda$) with function approximation has proved empirically successful for some complex reinforcement learning problems. For linear approximation, TD($\lambda$) has been shown to minimise the squared error between the approximate value…

The reduced-rank method exploits the distortion-variance tradeoff to yield superior solutions for classic problems in statistical signal processing such as parameter estimation and filtering. The central idea is to reduce the variance of…

We study reinforcement learning with linear function approximation where the transition probability and reward functions are linear with respect to a feature mapping $\boldsymbol{\phi}(s,a)$. Specifically, we consider the episodic…

This work presents a new variation of the commonly used Least Mean Squares Algorithm (LMS) for the identification of sparse signals with an a-priori known sparsity using a hard threshold operator in every iteration. It examines some useful…

We propose a new discrete-time online parameter estimation algorithm that combines two different aspects, one that adds momentum, and another that includes a time-varying learning rate. It is well known that recursive least squares based…

Least-squares reverse time migration (LSRTM) is one of the classic seismic imaging methods to reconstruct model perturbations within a known reference medium. It can be computed in either data or image domain using different methods by…

We propose a novel algorithm for greedy forward feature selection for regularized least-squares (RLS) regression and classification, also known as the least-squares support vector machine or ridge regression. The algorithm, which we call…

For a parameterized hyperbolic system $\frac{du}{dt}=f(u,s)$ the derivative of the ergodic average $\langle J \rangle = \lim_{T \to \infty}\frac{1}{T}\int_0^T J(u(t),s)$ to the parameter $s$ can be computed via the Least Squares Shadowing…

Markov parameters play a key role in system identification. There exists many algorithms where these parameters are estimated using least-squares in a first, pre-processing, step, including subspace identification and multi-step…