Related papers: An exact solution in Markov decision process with …

We consider the problem of controlling a fully specified Markov decision process (MDP), also known as the planning problem, when the state space is very large and calculating the optimal policy is intractable. Instead, we pursue the more…

This note summarizes the optimization formulations used in the study of Markov decision processes. We consider both the discounted and undiscounted processes under the standard and the entropy-regularized settings. For each setting, we…

We study stochastic team problems with static information structure where we assume controllers have linear information and quadratic cost but allow the noise to be from a non-Gaussian class. When the noise is Gaussian, it is well known…

We study stochastic approximation procedures for approximately solving a $d$-dimensional linear fixed point equation based on observing a trajectory of length $n$ from an ergodic Markov chain. We first exhibit a non-asymptotic bound of the…

This paper proposes an algorithm Alice having no access to the physics law of the environment, which is actually linear with stochastic noise, and learns to make decisions directly online without a training phase or a stable policy as…

We present a stochastic model predictive control framework for nonlinear systems subject to unbounded process noise with closed-loop guarantees. First, we provide a conceptual shrinking-horizon framework that utilizes general probabilistic…

One of the most fundamental problems in Markov decision processes is analysis and control synthesis for safety and reachability specifications. We consider the stochastic reach-avoid problem, in which the objective is to synthesize a…

Decision processes with incomplete state feedback have been traditionally modeled as Partially Observable Markov Decision Processes. In this paper, we present an alternative formulation based on probabilistic regular languages. The proposed…

We propose a formulation of the stochastic cutting stock problem as a discounted infinite-horizon Markov decision process. At each decision epoch, given current inventory of items, an agent chooses in which patterns to cut objects in stock…

We consider an infinite horizon optimal control problem for a pure jump Markov process $X$, taking values in a complete and separable metric space $I$, with noise-free partial observation. The observation process is defined as $Y_t =…

There are two major routes to address the ubiquitous family of inverse problems appearing in signal and image processing, such as denoising or deblurring. A first route relies on Bayesian modeling, where prior probabilities are used to…

We consider the stabilization of an unstable discrete-time linear system that is observed over a channel corrupted by continuous multiplicative noise. Our main result shows that if the system growth is large enough, then the system cannot…

We study finite horizon linear quadratic control with additive noise in a perturbancewise framework that unifies the classical model, a constraint embedded affine policy class, and a distributionally robust formulation with a Wasserstein…

We consider estimating a matrix from noisy observations coming from an arbitrary additive bi-rotational invariant perturbation. We propose an estimator which is optimal among the class of rectangular rotational invariant estimators and can…

We give algorithms for designing near-optimal sparse controllers using policy gradient with applications to control of systems corrupted by multiplicative noise, which is increasingly important in emerging complex dynamical networks.…

Specifying reward functions for complex tasks like object manipulation or driving is challenging to do by hand. Reward learning seeks to address this by learning a reward model using human feedback on selected query policies. This shifts…

Nonequilibrium systems driven by additive or multiplicative dichotomous Markov noise appear in a wide variety of physical and mathematical models. We review here some prototypical examples, with an emphasis on {\em analytically-solvable}…

This paper investigates the limit behavior of Markov Decision Processes (MDPs) made of independent particles evolving in a common environment, when the number of particles goes to infinity. In the finite horizon case or with a discounted…

We investigate a fully discrete finite element approximation for the stochastic Kuramoto-Sivashinsky equation, combining the standard finite element methods in spatial discretization with the implicit Euler-Maruyama scheme in time. Rigorous…

Many learning algorithms can be represented as Markov processes, and understanding their generalization error is a central topic in learning theory. For specific continuous-time noisy algorithms, a prominent analysis technique relies on…