Related papers: An Output Feedback Q-learning Algorithm for Optima…
This paper proposes a Koopman-based framework for modeling, prediction, and control of unknown nonlinear time-varying systems. We present a novel Koopman-based learning method for predicting the state of unknown nonlinear time-varying…
The Bellman equation and its continuous form, the Hamilton-Jacobi-Bellman equation, are ubiquitous in reinforcement learning and control theory. However, these equations become intractable for high-dimensional or nonlinear systems. This…
The $Q$-learning algorithm is a simple and widely-used stochastic approximation scheme for reinforcement learning, but the basic protocol can exhibit instability in conjunction with function approximation. Such instability can be observed…
Koopman-based learning methods can potentially be practical and powerful tools for dynamical robotic systems. However, common methods to construct Koopman representations seek to learn lifted linear models that cannot capture nonlinear…
This paper addresses a learning problem for nonlinear dynamical systems with incorporating any specified dissipativity property. The nonlinear systems are described by the Koopman operator, which is a linear operator defined on the…
We study reinforcement learning methods with linear function approximation under non-Markov state and cost processes. We first consider the policy evaluation method and show that the algorithm converges under suitable ergodicity conditions…
Koopman analysis provides a general framework from which to analyze a nonlinear dynamical system in terms of a linear operator acting on an infinite-dimensional observable space. This theoretical framework provides a rigorous underpinning…
This paper introduces new model parameterizations for learning discrete-time dynamical systems from data via the Koopman operator and studies their properties. Whereas most existing works on Koopman learning do not take into account the…
Koopman liftings have been successfully used to learn high dimensional linear approximations for autonomous systems for prediction purposes, or for control systems for leveraging linear control techniques to control nonlinear dynamics. In…
The Koopman operator has emerged as a powerful tool for the analysis of nonlinear dynamical systems as it provides coordinate transformations to globally linearize the dynamics. While recent deep learning approaches have been useful in…
Time-dependent structural reliability analysis of nonlinear dynamical systems is non-trivial; subsequently, scope of most of the structural reliability analysis methods is limited to time-independent reliability analysis only. In this work,…
The Koopman framework is a popular approach to transform a finite dimensional nonlinear system into an infinite dimensional, but linear model through a lifting process, using so-called observable functions. While there is an extensive…
In this paper, we present a new data-driven method for learning stable models of nonlinear systems. Our model lifts the original state space to a higher-dimensional linear manifold using Koopman embeddings. Interestingly, we prove that…
Koopman operator theory and Willems' fundamental lemma both can provide (approximated) data-driven linear representation for nonlinear systems. However, choosing lifting functions for the Koopman operator is challenging, and the quality of…
We study the problem of online tracking in unknown nonlinear dynamical systems, where only short-horizon predictions of future target states are available. This setting arises in practical scenarios where full future information and exact…
Q-learning is widely used algorithm in reinforcement learning community. Under the lookup table setting, its convergence is well established. However, its behavior is known to be unstable with the linear function approximation case. This…
Sparked by the Willems' fundamental lemma, a class of data-driven control methods has been developed for LTI systems. At the same time, the Koopman operator theory attempts to cast a nonlinear control problem into a standard linear one…
This paper proposes a method to identify a Koopman model of a feedback-controlled system given a known controller. The Koopman operator allows a nonlinear system to be rewritten as an infinite-dimensional linear system by viewing it in…
An outstanding challenge in nonlinear systems theory is identification or learning of a given nonlinear system's Koopman operator directly from data or models. Advances in extended dynamic mode decomposition approaches and machine learning…
In this study, we consider the application of max-plus-linear approximators for Q-function in offline reinforcement learning of discounted Markov decision processes. In particular, we incorporate these approximators to propose novel fitted…