Related papers: An exact solution in Markov decision process with …
A finite horizon linear quadratic(LQ) optimal control problem is studied for a class of discrete-time linear fractional systems (LFSs) affected by multiplicative, independent random perturbations. Based on the dynamic programming technique,…
We study a McKean-Vlasov optimal control problem with common noise, in order to establish the corresponding limit theory, as well as the equivalence between different formulations, including the strong, weak and relaxed formulation. In…
Robots performing manipulation tasks must operate under uncertainty about both their pose and the dynamics of the system. In order to remain robust to modeling error and shifts in payload dynamics, agents must simultaneously perform…
The linear quadratic regulator (LQR) problem has reemerged as an important theoretical benchmark for reinforcement learning-based control of complex dynamical systems with continuous state and action spaces. In contrast with nearly all…
We develop several new algorithms for learning Markov Decision Processes in an infinite-horizon average-reward setting with linear function approximation. Using the optimism principle and assuming that the MDP has a linear structure, we…
This paper studies mean field games for multi-agent systems with control-dependent multiplicative noises. For the general systems with nonuniform agents, we obtain a set of decentralized strategies by solving an auxiliary limiting optimal…
Gaussian Process (GP) models have also become extremely useful for optimization under uncertainty algorithms, especially where the objective functions are costly to compute. Yet, the more classical methods usually adopt strategies that, in…
The linear Markov Decision Process (MDP) framework offers a principled foundation for reinforcement learning (RL) with strong theoretical guarantees and sample efficiency. However, its restrictive assumption-that both transition dynamics…
Under suitable assumptions of regularity and non-degeneracy on the covariance of the driving additive noise, any Markov solution to the stochastic Navier-Stokes equations has an associated generator of the diffusion and is the unique…
In this paper we study the asymptotic behavior of a stochastic approximation scheme on two timescales with set-valued drift functions and in the presence of non-additive iterate-dependent Markov noise. It is shown that the recursion on each…
We propose a macroscopic market making model \`a la Avellaneda-Stoikov, using continuous processes for orders instead of discrete point processes. The model intends to bridge the gap between market making and optimal execution problems,…
We study the setting of \emph{performative reinforcement learning} where the deployed policy affects both the reward, and the transition of the underlying Markov decision process. Prior work~\parencite{MTR23} has addressed this problem…
McKean-Vlasov (MKV) systems provide a unifying framework for recent state-of-the-art particlebased methods for global optimization. While individual particles follow stochastic trajectories, the probability law evolves deterministically in…
We study the sample complexity of the plug-in approach for learning $\varepsilon$-optimal policies in average-reward Markov decision processes (MDPs) with a generative model. The plug-in approach constructs a model estimate then computes an…
The purpose of this paper is to review and highlight some connections between the problem of nonlinear smoothing and optimal control of the Liouville equation. The latter has been an active area of recent research interest owing to work in…
For a general class of diffusion processes with multiplicative noise, describing a variety of physical as well as financial phenomena, mostly typical of complex systems, we obtain the analytical solution for the moments at all times. We…
Iterative linear solvers have gained recent popularity due to their computational efficiency and low memory footprint for large-scale linear systems. The relaxation method, or Motzkin's method, can be viewed as an iterative method that…
In this note, we introduce a general version of the well-known elliptical potential lemma that is a widely used technique in the analysis of algorithms in sequential learning and decision-making problems. We consider a stochastic linear…
We propose a method of approximating multivariate Gaussian probabilities using dynamic programming. We show that solving the optimization problem associated with a class of discrete-time finite horizon Markov decision processes with…
In this paper we present a rigorous asymptotic analysis for stochastic systems with two fast relaxation times. The mathematical model analyzed in this paper consists of a Langevin equation for the particle motion with time-dependent force…