相关论文: On Wasserstein Reinforcement Learning and the Fokk…

Wasserstein gradient flows provide a powerful means of understanding and solving many diffusion equations. Specifically, Fokker-Planck equations, which model the diffusion of probability measures, can be understood as gradient descent over…

We study the Fokker-Planck equation as the hydrodynamic limit of a stochastic particle system on one hand and as a Wasserstein gradient flow on the other. We write the rate functional, that characterizes the large deviations from the…

We describe an application of Wasserstein distance to Reinforcement Learning. The Wasserstein distance in question is between the distribution of mappings of trajectories of a policy into some metric space, and some other fixed distribution…

We study policy gradient methods for continuous-action, entropy-regularized reinforcement learning through the lens of Wasserstein geometry. Starting from a Wasserstein proximal update, we derive Wasserstein Proximal Policy Gradient (WPPG)…

Policy optimization is a core component of reinforcement learning (RL), and most existing RL methods directly optimize parameters of a policy based on maximizing the expected total reward, or its surrogate. Though often achieving…

We introduce a new approach for comparing reinforcement learning policies, using Wasserstein distances (WDs) in a newly defined latent behavioral space. We show that by utilizing the dual formulation of the WD, we can learn score functions…

We consider a class of time-homogeneous diffusion processes on $\mathbb{R}^{n}$ with common invariant measure but varying volatility matrices. In Euclidean space, we show via stochastic control of the diffusion coefficient that the…

Off-policy evaluation and learning are concerned with assessing a given policy and learning an optimal policy from offline data without direct interaction with the environment. Often, the environment in which the data are collected differs…

Wasserstein Policy Optimization (WPO) is a recently proposed reinforcement learning algorithm that leverages Wasserstein gradient flows to optimize stochastic policies in continuous action spaces. Despite its empirical success, the…

We consider a Fokker-Planck equation which is coupled to an externally given time-dependent constraint on its first moment. This constraint introduces a Lagrange-multiplier which renders the equation nonlocal and nonlinear. In this paper we…

We revisit the variational characterization of diffusion as entropic gradient flux and provide for it a probabilistic interpretation based on stochastic calculus. It was shown by Jordan, Kinderlehrer, and Otto that, for diffusions of…

We introduce Wasserstein Policy Optimization (WPO), an actor-critic algorithm for reinforcement learning in continuous action spaces. WPO can be derived as an approximation to Wasserstein gradient flow over the space of all policies…

We study the large deviation rate functional for the empirical distribution of independent Brownian particles with drift. In one dimension, it has been shown by Adams, Dirr, Peletier and Zimmer that this functional is asymptotically…

To ensure stability of learning, state-of-the-art generalized policy iteration algorithms augment the policy improvement step with a trust region constraint bounding the information loss. The size of the trust region is commonly determined…

In this paper we identify the Fokker-Planck equation for (reflected) Sticky Brownian Motion as a Wasserstein gradient flow in the space of probability measures. The driving functional is the relative entropy with respect to a non-standard…

Robust Reinforcement Learning aims to find the optimal policy with some extent of robustness to environmental dynamics. Existing learning algorithms usually enable the robustness through disturbing the current state or simulating…

Gradient flows of the Kullback--Leibler (KL) divergence, such as the Fokker--Planck equation and Stein Variational Gradient Descent, evolve a distribution toward a target density known only up to a normalizing constant. We introduce new…

In this work, we investigate a variational formulation for a time-fractional Fokker-Planck equation which arises in the study of complex physical systems involving anomalously slow diffusion. The model involves a fractional-order Caputo…

Trust-region methods based on Kullback-Leibler divergence are pervasively used to stabilize policy optimization in reinforcement learning. In this paper, we exploit more flexible metrics and examine two natural extensions of policy…

The steady state of the Fokker-Planck equation corresponding to a density dependent one-step process is approximated by a suitable normal distribution. Starting from the master equations of the process, written in terms of the time…