Related papers: A Q-learning algorithm for discrete-time linear-qu…
Contraction properties of the Riccati operator are studied within the context of non-stationary linear-quadratic optimal control. A lifting approach is used to obtain a bound on the rate of strict contraction, with respect to the Riemannian…
In most real cases transition probabilities between operational modes of Markov jump linear systems cannot be computed exactly and are time-varying. We take into account this aspect by considering Markov jump linear systems where the…
In this paper, we present a novel method for computing the optimal feedback gain of the infinite-horizon Linear Quadratic Regulator (LQR) problem via an ordinary differential equation. We introduce a novel continuous-time Bellman error,…
This paper is concerned with a constrained stochastic linear-quadratic optimal control problem, in which the terminal state is fixed and the initial state is constrained to lie in a stochastic linear manifold. The controllability of…
Flow $Q$-learning has recently been introduced to integrate learning from expert demonstrations into an actor-critic structure. Central to this innovation is the ``the one-step policy'' network, which is optimized through a $Q$-function…
In the past couple of decades, non-quadratic convex penalties have reshaped signal processing and machine learning; in robust control, however, general convex costs break the Riccati and storage function structure that make the design…
In this paper we consider a class of conjugate discrete-time Riccati equations, arising originally from the linear quadratic regulation problem for discrete-time antilinear systems. Under mild and reasonable assumptions, the existence of…
We consider a class of stochastic control problems with a delayed control, both in drift and diffusion, of the type dX t = $\alpha$ t--d (bdt + $\sigma$dW t). We provide a new characterization of the solution in terms of a set of Riccati…
We revisit in this paper the discrete-time linear quadratic regulator (LQR) problem from the perspective of receding-horizon policy gradient (RHPG), a newly developed model-free learning framework for control applications. We provide a…
This paper proposes a reinforcement learning (RL) algorithm for infinite horizon $\rm {H_{2}/H_{\infty}}$ problem in a class of stochastic discrete-time systems, rather than using a set of coupled generalized algebraic Riccati equations…
The present work addresses a finite-horizon linear-quadratic optimal control problem for uncertain systems driven by piecewise constant controls. The precise values of the system parameters are unknown, but assumed to belong to a finite set…
The problem of robust distributed control arises in several large-scale systems, such as transportation networks and power grid systems. In many practical scenarios controllers might not have enough information to make globally optimal…
This paper addresses the mean-square optimal control problem for \a class of discrete-time linear systems with a quasi-colored control-dependent multiplicative noise via output feedback. The noise under study is novel and shown to have…
This paper is concerned with a stochastic linear-quadratic optimal control problem in a finite time horizon, where the coefficients of the control system are allowed to be random, and the weighting matrices in the cost functional are…
Q-learning is widely used algorithm in reinforcement learning community. Under the lookup table setting, its convergence is well established. However, its behavior is known to be unstable with the linear function approximation case. This…
In this paper, we formulate a general time-inconsistent stochastic linear--quadratic (LQ) control problem. The time-inconsistency arises from the presence of a quadratic term of the expected state as well as a state-dependent term in the…
The paper introduces the first formulation of convex Q-learning for Markov decision processes with function approximation. The algorithms and theory rest on a relaxation of a dual of Manne's celebrated linear programming characterization of…
Many real-world sequential decision-making problems can be formulated as optimal control with high-dimensional observations and unknown dynamics. A promising approach is to embed the high-dimensional observations into a lower-dimensional…
For an infinite-horizon control problem, the optimal control can be represented by the stable manifold of the characteristic Hamiltonian system of Hamilton-Jacobi-Bellman (HJB) equation in a semiglobal domain. In this paper, we first…
This paper considers a risk-constrained infinite-horizon optimal control problem and proposes to solve it in an iterative manner. Each iteration of the algorithm generates a trajectory from the starting point to the target equilibrium state…