Related papers: Koopman-Assisted Reinforcement Learning
The discovery of linear embedding is the key to the synthesis of linear control techniques for nonlinear systems. In recent years, while Koopman operator theory has become a prominent approach for learning these linear embeddings through…
We develop a Koopman operator framework for studying the {computational properties} of dynamical systems. Specifically, we show that the resolvent of the Koopman operator provides a natural abstraction of halting, yielding a ``Koopman…
Continuous-time reinforcement learning offers an appealing formalism for describing control problems in which the passage of time is not naturally divided into discrete increments. Here we consider the problem of predicting the distribution…
We present an approach to construct approximate Koopman-type decompositions for dynamical systems depending on static or time-varying parameters. Our method simultaneously constructs an invariant subspace and a parametric family of…
Controlling nonlinear dynamical systems remains a central challenge in a wide range of applications, particularly when accurate first-principle models are unavailable. Data-driven approaches offer a promising alternative by designing…
As autonomous systems become more ubiquitous in daily life, ensuring high performance with guaranteed safety is crucial. However, safety and performance could be competing objectives, which makes their co-optimization difficult.…
Soft robotic manipulators offer operational advantage due to their compliant and deformable structures. However, their inherently nonlinear dynamics presents substantial challenges. Traditional analytical methods often depend on simplifying…
The Koopman operator provides a linear perspective on non-linear dynamics by focusing on the evolution of observables in an invariant subspace. Observables of interest are typically linearly reconstructed from the Koopman eigenfunctions.…
The Koopman operator framework can be used to identify a data-driven model of a nonlinear system. Unfortunately, when the data is corrupted by noise, the identified model can be biased. Additionally, depending on the choice of lifting…
The estimation for the region of attraction (ROA) of an asymptotically stable equilibrium point is crucial in the analysis of nonlinear systems. There has been a recent surge of interest in estimating the solution to Zubov's equation, whose…
Bootstrapping and rollout are two fundamental principles for value function estimation in reinforcement learning (RL). We introduce a novel class of Bellman operators, called subgraph Bellman operators, that interpolate between…
The Koopman operator provides a powerful framework for modeling dynamical systems and has attracted growing interest from the machine learning community. However, its infinite-dimensional nature makes identifying suitable finite-dimensional…
This paper presents a class of linear predictors for nonlinear controlled dynamical systems. The basic idea is to lift the nonlinear dynamics into a higher dimensional space where its evolution is approximately linear. In an uncontrolled…
A learning based method for obtaining feedback laws for nonlinear optimal control problems is proposed. The learning problem is posed such that the open loop value function is its optimal solution. This infinite dimensional, function space,…
Classically, the optimal control problem in the presence of an adversary is formulated as a two-player zero-sum differential game or an $H_\infty$ control problem. The solution to these problems can be obtained by solving the…
Koopman operators and transfer operators represent nonlinear dynamics in state space through its induced action on linear spaces of observables and measures, respectively. This framework enables the use of linear operator theory for…
This work presents a data-driven Koopman operator-based modeling method using a model averaging technique. While the Koopman operator has been used for data-driven modeling and control of nonlinear dynamics, it is challenging to accurately…
Representation learning is critical to the empirical and theoretical success of reinforcement learning. However, many existing methods are induced from model-learning aspects, misaligning them with the RL task in hand. This work introduces…
Many optimal control problems are formulated as two point boundary value problems (TPBVPs) with conditions of optimality derived from the Hamilton-Jacobi-Bellman (HJB) equations. In most cases, it is challenging to solve HJBs due to the…
Inverse reinforcement learning aims to infer the reward function that explains expert behavior observed through trajectories of state--action pairs. A long-standing difficulty in classical IRL is the non-uniqueness of the recovered reward:…