Related papers: A pseudo-Markov property for controlled diffusion …
The thermodynamic formalism, which was first developed for dynamical systems and then applied to discrete Markov processes, turns out to be well suited for continuous time Markov processes as well, provided the definitions are interpreted…
We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by `controlled' Markov noise. In particular, both the faster and slower recursions have non-additive controlled Markov noise…
We present an elementary state augmentation method for a class of static risk measure applied to the total cost for both Markov decision processes and stochastic optimal control, such that dynamic programming equations can be derived on the…
Given a Brownian motion $W$ and a stationary Poisson point process $p$ with values in ${\mathbb R}^d$, we prove a Dynamic Programming Principle (DPP) in a strong formulation for a stochastic control problem involving controlled SDEs of the…
Conditional independence and Markov properties are powerful tools allowing expression of multidimensional probability distributions by means of low-dimensional ones. As multidimensional possibilistic models have been studied for several…
Dual control denotes a class of control problems where the parameters governing the system are imperfectly known. The challenge is to find the optimal balance between probing, i.e. exciting the system to understand it more, and caution,…
Complex physical dynamics can often be modeled as a Markov jump process between mesoscopic configurations. When jumps between mesoscopic states are mediated by thermodynamic reservoirs, the time-irreversibility of the jump process is a…
Diffusion models have achieved huge empirical success in data generation tasks. Recently, some efforts have been made to adapt the framework of diffusion models to discrete state space, providing a more natural approach for modeling…
In this article, we study the ergodic risk-sensitive control problem for controlled regime-switching diffusions. Under a blanket stability hypothesis, we solve the associated nonlinear eigenvalue problem for weakly coupled systems and…
We are interested in understanding stability (almost sure boundedness) of stochastic approximation algorithms (SAs) driven by a `controlled Markov' process. Analyzing this class of algorithms is important, since many reinforcement learning…
We prove two duality descriptions of the value function for a generic stochastic optimal problem. These descriptions also hold when the diffusion is controlled, a case left open by the literature so far.
This paper considers the problem of designing time-dependent, real-time control policies for controllable nonlinear diffusion processes, with the goal of obtaining maximally-informative observations about parameters of interest. More…
This paper is addressed to studying the exact controllability for stochastic transport equations by two controls: one is a boundary control imposed on the drift term and the other is an internal control imposed on the diffusion term. By…
In this paper, we study the dynamics of a linear control system with given state feedback control law in the presence of fast periodic sampling at temporal frequency $1/\delta$ ($0 < \delta \ll 1$), together with small white noise…
We consider a large family of discrete and continuous time controlled Markov processes and study an ergodic risk-sensitive minimization problem. Under a blanket stability assumption, we provide a complete analysis to this problem. In…
This paper deals with the question of how to most effectively conduct experiments in Partially Observed Markov Decision Processes so as to provide data that is most informative about a parameter of interest. Methods from Markov decision…
In this paper we are concerned with the approximate controllability of a multidimensional semilinear reaction-diffusion equation governed by a multiplicative control, which is locally distributed in the reaction term. For a given initial…
We revisit the work of Mitter and Newton on an information-theoretic interpretation of Bayes' formula through the Gibbs variational principle. This formulation allowed them to pose nonlinear estimation for diffusion processes as a problem…
In contexts where data samples represent a physically stable state, it is often assumed that the data points represent the local minima of an energy landscape. In control theory, it is well-known that energy can serve as an effective…
Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain…