Related papers: A Simple Algorithm for Solving Ramsey Optimal Poli…
We develop a family of reformulations of an arbitrary consistent linear system into a stochastic problem. The reformulations are governed by two user-defined parameters: a positive definite matrix defining a norm, and an arbitrary discrete…
The solution to a stochastic optimal control problem can be determined by computing the value function from a discretization of the associated Hamilton-Jacobi-Bellman equation. Alternatively, the problem can be reformulated in terms of a…
This paper considers the problem of designing optimal algorithms for reinforcement learning in two-player zero-sum games. We focus on self-play algorithms which learn the optimal policy by playing against itself without any direct…
We propose policy gradient algorithms for solving a risk-sensitive reinforcement learning (RL) problem in on-policy as well as off-policy settings. We consider episodic Markov decision processes, and model the risk using the broad class of…
Finding optimal policies which maximize long term rewards of Markov Decision Processes requires the use of dynamic programming and backward induction to solve the Bellman optimality equation. However, many real-world problems require…
We develop an approach for solving time-consistent risk-sensitive stochastic optimization problems using model-free reinforcement learning (RL). Specifically, we assume agents assess the risk of a sequence of random variables using dynamic…
We derive simplified formulas for analyzing the stability of stochastic parametrically forced linear systems. This extends the results in [T. Blass and L.A. Romero, SIAM J. Control Optim. 51(2):1099--1127, 2013] where, assuming the…
Recent work has shown how information theory extends conventional full-rationality game theory to allow bounded rational agents. The associated mathematical framework can be used to solve constrained optimization problems. This is done by…
We consider solving a convex, possibly stochastic optimization problem over a randomly time-varying multi-agent network. Each agent has access to some local objective function, and it only has unbiased estimates of the gradients of the…
Reinforcement learning considers the problem of finding policies that maximize an expected cumulative reward in a Markov decision process with unknown transition probabilities. In this paper we consider the problem of finding optimal…
There is a recent surge of interest in nonconvex reformulations via low-rank factorization for stochastic convex semidefinite optimization problem in the purpose of efficiency and scalability. Compared with the original convex formulations,…
We consider the following two-player game: using observational data, the leader chooses a prediction function for a response variable $Y$ from given covariates. The follower then reacts with an intervention on some covariates in the…
We propose two policy gradient algorithms for solving the problem of control in an off-policy reinforcement learning (RL) context. Both algorithms incorporate a smoothed functional (SF) based gradient estimation scheme. The first algorithm…
Existing methods for learning Stackelberg equilibria typically assume that the followers' (variational, generalized) Nash equilibrium is unique. However, in the presence of multiple equilibria, without a selection convention, the problem…
We develop a fourth order simulation algorithm for solving the stochastic Langevin equation. The method consists of identifying solvable operators in the Fokker-Planck equation, factorizing the evolution operator for small time steps to…
A new stochastic control problem of a dam-reservoir system installed in a river is analyzed both mathematically and numerically. Water balance dynamics of the reservoir are piece-wise deterministic and are driven by a stochastic…
We study the problem of optimal portfolio selection under stochastic volatility within a continuous time reinforcement learning framework with portfolio constraints. Exploration is modeled through entropy-regularized relaxed controls, where…
Non-smooth dynamics driven by stochastic disturbance arise in a wide variety of engineering problems. Impulsive interventions are often employed to control stochastic systems; however, the modeling and analysis subject to execution delay…
This paper studies the dynamic programming principle for general convex stochastic optimization problems introduced by Rockafellar and Wets in [30]. We extend the applicability of the theory by relaxing compactness and boundedness…
We present new results on the classical algorithm of variable elimination, which underlies many algorithms including for probabilistic inference. The results relate to exploiting functional dependencies, allowing one to perform inference…