Related papers: Ergodic Risk-Sensitive Control of Markov Processes…
A simple model of the new notion of "Markov up" processes is proposed; its positive recurrence and ergodic properties are shown under the appropriate conditions.
In this work, we show the consistency of an approach for solving robust optimization problems using sequences of sub-problems generated by ergodic measure preserving transformations. The main result of this paper is that the minimizers and…
The paper proposes a new stochastic intervention control model conducted in various commodity and stock markets. The essence of the phenomenon of intervention is described in accordance with current economic theory. A review of papers on…
We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the…
This paper focuses on the design of time-invariant memoryless control policies for fully observed controlled Markov chains, with a finite state space. Safety constraints are imposed through a pre-selected set of forbidden states. A state is…
We study a general class of PageRank optimization problems which consist in finding an optimal outlink strategy for a web site subject to design constraints. We consider both a continuous problem, in which one can choose the intensity of a…
We consider a lattice of weakly interacting quantum Markov processes. Without interaction, the dynamics at each site is relaxing exponentially to a unique stationary state. With interaction, we show that there remains a unique stationary…
Stochastic optimal control problems have a long tradition in applied probability, with the questions addressed being of high relevance in a multitude of fields. Even though theoretical solutions are well understood in many scenarios, their…
This paper considers an optimal impulse control problem of dynamical systems generated by a flow. The performance criteria are total costs over the infinite time horizon. Apart from the main performance to be minimized, there are multiple…
We consider a hidden Markov model with multiple observation processes, one of which is chosen at each point in time by a policy---a deterministic function of the information state---and attempt to determine which policy minimises the…
We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slower recursions have non-additive controlled Markov noise…
It is well known that stability is the most fundamental nature with regard to a control system, in view of this, the stabilization becomes an inevitable control problem. This article mainly discusses the optimal control and stabilization…
We study multiclass many-server queues for which the arrival, service and abandonment rates are all modulated by a common finite-state Markov process. We assume that the system operates in the "averaged" Halfin-Whitt regime, which means…
We study the estimation of the value function for continuous-time Markov diffusion processes using a single, discretely observed ergodic trajectory. Our work provides non-asymptotic statistical guarantees for the least-squares…
We propose and analyze a stabilizing iteration scheme for the algorithmic implementation of model predictive control for linear discrete-time systems. Polytopic input and state constraints are considered and handled by means of so-called…
We consider finite model approximations of discrete-time partially observed Markov decision processes (POMDPs) under the discounted cost criterion. After converting the original partially observed stochastic control problem to a fully…
In this paper, we consider the problem of controlling a partially observed Markov decision process (POMDP) in order to actively estimate its state trajectory over a fixed horizon with minimal uncertainty. We pose a novel active smoothing…
We consider a family of measure preserving transformations, which act on a common probability space and are chosen at random by a stationary ergodic Markov chain. This setting defines an instance of a random dynamical system (RDS), which…
We study one-sided and $\alpha$-correct sequential hypothesis testing for data generated by an ergodic Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the…
Controlled discrete time Markov processes are studied first with long run general discounting functional. It is shown that optimal strategies for average reward per unit time problem are also optimal for average generally discounting…