相关论文: Singularly perturbed Markov chains: Limit results …
We present two related anytime algorithms for control of nonlinear systems when the processing resources available are time-varying. The basic idea is to calculate tentative control input sequences for as many time steps into the future as…
A general formalism is developed to construct a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are therefore internal to the system and not externally specified. For…
This chapter surveys progress on three related topics in perturbations of Markov chains: the motivating question of when and how "perturbed" MCMC chains are developed, the theoretical problem of how perturbation theory can be used to…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
We consider an additive functional driven by a time-inhomogeneous Markov chain with a finite state space. Our study focuses on the joint distribution of the two-sided exit time and the state of the driving Markov chain at the time of exit,…
We consider Markovian models on graphs with local dynamics. We show that, under suitable conditions, such Markov chains exhibit both rapid convergence to equilibrium and strong concentration of measure in the stationary distribution. We…
In the context of Markov decision processes running in continuous time, one of the most intriguing challenges is the efficient approximation of finite horizon reachability objectives. A multitude of sophisticated model checking algorithms…
Perturbation theory for Markov chains addresses the question how small differences in the transitions of Markov chains are reflected in differences between their distributions. We prove powerful and flexible bounds on the distance of the…
Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…
Viewing a two time scale stochastic approximation scheme as a noisy discretization of a singularly perturbed differential equation, we obtain a concentration bound for its iterates that captures its behavior with quantifiable high…
We present a new class of interacting Markov chain Monte Carlo algorithms for solving numerically discrete-time measure-valued equations. The associated stochastic processes belong to the class of self-interacting Markov chains. In contrast…
Comparison results are given for time-inhomogeneous Markov processes with respect to function classes induced stochastic orderings. The main result states comparison of two processes, provided that the comparability of their infinitesimal…
We study the limit fluctuations of the rescaled occupation time process of a branching particle system in $\mathbb{R}^d$, where the particles are subject to symmetric $\alpha$-stable migration ($0<\alpha\leq2$), critical binary branching,…
The stability of stochastic Model Predictive Control (MPC) subject to additive disturbances is often demonstrated in the literature by constructing Lyapunov-like inequalities that guarantee closed-loop performance bounds and boundedness of…
We use the two-time scale subordination in order to describe dynamical processes in continuous media with a long-term memory. Our consideration touches two physical examples in detail. First we study a temporal evolution of the species…
In this article, we prove that a small random perturbation of dynamical system with multiple stable equilibria converges to a Markov chain whose states are neighborhoods of the deepest stable equilibria, under a suitable time-rescaling,…
The Markowitz problem consists of finding in a financial market a self-financing trading strategy whose final wealth has maximal mean and minimal variance. We study this in continuous time in a general semimartingale model and under cone…
The expansion of global production networks has raised many important questions about the interdependence among countries and how future changes in the world economy are likely to affect the countries' positioning in global value chains. We…
In the paper we study continuous time controlled Markov processes using discrete time controlled Markov processes. We consider long run functionals: average reward per unit time or long run risk sensitive functional. We also investigate…
Motivated by queueing systems with heterogeneous parallel servers, we consider a class of structured multi-dimensional Markov processes whose state space can be partitioned into two parts: a finite set of boundary states and a structured…