Related papers: Algorithms for reachability problems on stochastic…

Recently there is a large amount of work devoted to the study of Markov chain stochastic gradient methods (MC-SGMs) which mainly focus on their convergence analysis for solving minimization problems. In this paper, we provide a…

This paper deals with N-person nonzero-sum discrete-time Markov games under a probability criterion, in which the transition probabilities and reward functions are allowed to vary with time. Differing from the existing works on the expected…

A Markov decision process can be parameterized by a transition kernel and a reward function. Both play essential roles in the study of reinforcement learning as evidenced by their presence in the Bellman equations. In our inquiry of various…

We introduce a novel class of algorithms to efficiently approximate the unknown return distributions in policy evaluation problems from distributional reinforcement learning (DRL). The proposed distributional dynamic programming algorithms…

This paper tackles the challenge of parameter calibration in stochastic models, particularly in scenarios where the likelihood function is unavailable in an analytical form. We introduce a gradient-based simulated parameter estimation…

Reward comparisons are vital for evaluating differences in agent behaviors induced by a set of reward functions. Most conventional techniques utilize the input reward functions to learn optimized policies, which are then used to compare…

We adapt ideas and concepts developed in optimal transport (and its martingale variant) to give a geometric description of optimal stopping times of Brownian motion subject to the constraint that the distribution of the stopping time is a…

Stochastic optimization is a vital field in the realm of mathematical optimization, finding applications in diverse areas ranging from operations research to machine learning. In this paper, we introduce a novel first-order optimization…

Reward Models (RMs) are crucial for online alignment of language models (LMs) with human preferences. However, RM-based preference-tuning is vulnerable to reward hacking, whereby LM policies learn undesirable behaviors from flawed RMs. By…

We propose an algorithm for deterministic continuous Markov Decision Processes with sparse rewards that computes the optimal policy exactly with no dependency on the size of the state space. The algorithm has time complexity of $O( |R|^3…

Safety in stochastic control systems, which are subject to random noise with a known probability distribution, aims to compute policies that satisfy predefined operational constraints with high confidence throughout the uncertain evolution…

We study the regret guarantee for risk-sensitive reinforcement learning (RSRL) via distributional reinforcement learning (DRL) methods. In particular, we consider finite episodic Markov decision processes whose objective is the entropic…

A classic solution technique for Markov decision processes (MDP) and stochastic games (SG) is value iteration (VI). Due to its good practical performance, this approximative approach is typically preferred over exact techniques, even though…

We study a family of distributed stochastic optimization algorithms where gradients are sampled by a token traversing a network of agents in random-walk fashion. Typically, these random-walks are chosen to be Markov chains that…

In this paper we study reachability verification problems of stochastic discrete-time dynamical systems over the infinite time horizon. The reachability verification of interest in this paper is to certify specified lower and upper bounds…

We present a practically appealing extension of the probabilistic model checker PRISM rendering it to handle fixed-delay continuous-time Markov chains (fdCTMCs) with rewards, the equivalent formalism to the deterministic and stochastic…

First passage distributions of semi-Markov processes are of interest in fields such as reliability, survival analysis, and many others. The problem of finding or computing first passage distributions is, in general, quite challenging. We…

Motivated by the dynamics of resonant neurons we consider a differentiable, non-Markovian random process $x(t)$ and particularly the time after which it will reach a certain level $x_b$. The probability density of this first passage time is…

Hybrid systems, and Piecewise Deterministic Markov Processes in particular, are widely used to model and numerically study systems exhibiting multiple time scales in biochemical reaction kinetics and related areas. In this paper an almost…

The challenging problem of conducting fully Bayesian inference for the reaction rate constants governing stochastic kinetic models (SKMs) is considered. Given the challenges underlying this problem, the Markov jump process representation is…