Related papers: Stochastic Resetting Accelerates Policy Convergenc…
Direct policy optimization in reinforcement learning is usually solved with policy-gradient algorithms, which optimize policy parameters via stochastic gradient ascent. This paper provides a new theoretical interpretation and justification…
The theory of stochastic resetting asserts that restarting a stochastic process can expedite its completion. In this paper, we study the escape process of a Brownian particle in an open Hamiltonian system that suffers noise-enhanced…
State-of-the-art model-based reinforcement learning methods train policies on imagined rollouts. These rollouts are trajectories generated by a learned dynamics model and are scored by a learned reward model, but without querying the true…
Offline reinforcement learning (RL) is challenged by the distributional shift between learning policies and datasets. To address this problem, existing works mainly focus on designing sophisticated algorithms to explicitly or implicitly…
Sparse reward problems are one of the biggest challenges in Reinforcement Learning. Goal-directed tasks are one such sparse reward problems where a reward signal is received only when the goal is reached. One promising way to train an agent…
We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slower recursions have non-additive controlled Markov noise…
Processes controlled by stochastic synthesis and degradation (SSD) are widespread in biology but their reaction kinetics are not well understood. Using methods borrowed from the theory of resetting processes, we determine the first-passage…
The first hitting times of a stochastic process, i.e., the first time a process reaches a particular level, are of significant interest across various scientific disciplines, including biology, chemistry, and economics. We modify the…
Experience replay is one of the most commonly used approaches to improve the sample efficiency of reinforcement learning algorithms. In this work, we propose an approach to select and replay sequences of transitions in order to accelerate…
This paper presents an approach for data-driven policy refinement in reinforcement learning, specifically designed for safety-critical applications. Our methodology leverages the strengths of data-driven optimization and reinforcement…
We present and characterize a method to accelerate the relaxation of a Brownian object between two distinct equilibrium states. Instead of relying on a deterministic time-dependent control parameter, we use stochastic resetting to guide and…
Policy gradient methods have shown success in learning control policies for high-dimensional dynamical systems. Their biggest downside is the amount of exploration they require before yielding high-performing policies. In a lifelong…
Stochastic resetting and noise-enhanced stability are two phenomena which can affect the lifetime and relaxation of nonequilibrium states. They can be considered as measures of controlling the efficiency of the completion process when a…
Policy gradient (PG) methods are successful approaches to deal with continuous reinforcement learning (RL) problems. They learn stochastic parametric (hyper)policies by either exploring in the space of actions or in the space of parameters.…
Contemporary reinforcement learning with verifiable reward methods post-train language models on multi-step reasoning by assigning a single outcome reward uniformly across all tokens in a trajectory. Such uniform assignment ignores which…
We study simple diffusion where a particle stochastically resets to its initial position at a constant rate r. A finite resetting rate leads to a nonequilibrium stationary state with non-Gaussian fluctuations for the particle position. We…
This paper investigates a type of instability that is linked to the greedy policy improvement in approximated reinforcement learning. We show empirically that non-deterministic policy improvement can stabilize methods like LSPI by…
Guided policy search algorithms have been proven to work with incredible accuracy for not only controlling a complicated dynamical system, but also learning optimal policies from various unseen instances. One assumes true nature of the…
In this paper we are introducing a new reinforcement learning method for control problems in environments with delayed feedback. Specifically, our method employs stochastic planning, versus previous methods that used deterministic planning.…
Leveraging machine learning methods to solve constraint satisfaction problems has shown promising, but they are mostly limited to a static situation where the problem description is completely known and fixed from the beginning. In this…