相关论文: On some transformations between positive self--sim…

Many integrable stochastic particle systems in one space dimension (such as TASEP - Totally Asymmetric Simple Exclusion Process - and its $q$-deformation, the $q$-TASEP) remain integrable if we equip each particle with its own speed…

We consider the problem of finding a control policy for a Markov Decision Process (MDP) to maximize the probability of reaching some states while avoiding some other states. This problem is motivated by applications in robotics, where such…

Our purpose is to study a particular class of optimal stopping problems for Markov processes. We justify the value function convexity and we deduce that there exists a boundary function such that the smallest optimal stopping time is the…

We construct a family of self-similar Markov martingales with given marginal distributions. This construction uses the self-similarity and Markov property of a reference process to produce a family of Markov processes that possess the same…

We study hitting probabilities for $\mathbb{Z}^d$-extensions of Gibbs-Markov maps. The goal is to estimate, given a finite $\Sigma \subset \mathbb{Z}^d$ and $p$, $q \in \Sigma$, the probability $P_{pq}$ that the process starting from $p$…

In the recent article D\"oring et al. [4] the authors conditioned a stable process with two-sided jumps to avoid an interval. As usual the strategy was to find an invariant function for the process killed on entering the interval and to…

The particle Gibbs (PG) sampler is a Markov Chain Monte Carlo (MCMC) algorithm, which uses an interacting particle system to perform the Gibbs steps. Each Gibbs step consists of simulating a particle system conditioned on one particle path.…

A critical branching process $\left\{ Z_{k},k=0,1,2,...\right\} $ in a random environment is considered. A conditional functional limit theorem for the properly scaled process $\left\{ \log Z_{pu},0\leq u<\infty \right\} $ is established…

This paper considers the speed of convergence (mixing) of a finite Markov kernel $P$ with respect to the Kullback-Leibler divergence (entropy). Given a Markov kernel one defines either a discrete-time Markov chain (with the $n$-step…

We introduce a framework to approximate a Markov Decision Process that stands on two pillars: state aggregation -- as the algorithmic infrastructure; and central-limit-theorem-type approximations -- as the mathematical underpinning of…

Let S be a denumerable state space and let P be a transition probability matrix on S. If a denumerable set M of nonnegative matrices is such that the sum of the matrices is equal to P, then we call M a partition of P. Let K denote the set…

Markov decision processes (MDPs) with rewards are a widespread and well-studied model for systems that make both probabilistic and nondeterministic choices. A fundamental result about MDPs is that their minimal and maximal expected rewards…

We consider partially observable Markov decision processes (POMDPs) with a set of target states and positive integer costs associated with every transition. The traditional optimization objective (stochastic shortest path) asks to minimize…

We describe an exact approach for calculating transition probabilities and waiting times in finite-state discrete-time Markov processes. All the states and the rules for transitions between them must be known in advance. We can then…

We consider the small mass asymptotic (Smoluchowski-Kramers approximation) for the Langevin equation with a variable friction coefficient. The friction coefficient is assumed to be vanishing within certain region. We introduce a…

We consider stochastic approximations which arise from such applications as data communications and image processing. We demonstrate why constraints are needed in a stochastic approximation and how a constrained approximation can be…

An infinite system of point particles placed in $\mathds{R}^d$ is studied. Its constituents perform random jumps with mutual repulsion described by a translation-invariant jump kernel and interaction potential, respectively. The pure states…

We study the Markovianity of a composite system and its subsystems. We show how the dissipative nature of a subsystem's dynamics can be modified without having to change properties of the composite system environment. By preparing different…

We connect self-interacting processes, that is, stochastic processes where transitions depend on the time spent by a trajectory in each configuration, to Doob conditioning. In this way we demonstrate that Markov processes with constrained…

Let $X$ be a Markov process taking values in $\mathbf{E}$ with continuous paths and transition function $(P_{s,t})$. Given a measure $\mu$ on $(\mathbf{E}, \mathscr{E})$, a Markov bridge starting at $(s,\varepsilon_x)$ and ending at…