Related papers: Pitman's discrete $2M-X$ theorem for arbitrary ini…

We consider Motzkin paths of length $L$, not fixed at zero at both end points, with constant weights on the edges and general weights on the end points. We investigate, as the length $L$ tends to infinity, the limit behaviors of (a)…

Building upon the theory of graph limits and the Aldous-Hoover representation and inspired by Panchenko's work on asymptotic Gibbs measures (Annals of Probability 2013), we construct continuous embeddings of discrete probability…

We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…

Let P := {X_i,i >= 1} be a stationary Poisson point process in R^d, {C_i,i >= 1} be a sequence of i.i.d. random sets in R^d, and {Y_i^t; t \geq 0, i >= 1} be i.i.d. {0,1}-valued continuous time stationary Markov chains. We define the…

We study the computational complexity of approximating general constrained Markov decision processes. Our primary contribution is the design of a polynomial time $(0,\epsilon)$-additive bicriteria approximation algorithm for finding optimal…

For each $n\geq 1$, let $ {X_{in}, \quad i \geq 1} $ be independent copies of a nonnegative continuous stochastic process $X_{n}=(X_n(t))_{t\in T}$ indexed by a compact metric space $T$. We are interested in the process of partial maxima…

Although introduced in the case of Poisson random measures, the lent particle method applies as well in other situations. We study here the case of marked point processes. In this case the Malliavin calculus (here in the sense of Dirichlet…

Adaptive Markov Chain Monte Carlo (AMCMC) is a class of MCMC algorithms where the proposal distribution changes at every iteration of the chain. In this case it is important to verify that such a Markov Chain indeed has a stationary…

We consider a class of piecewise-deterministic Markov processes where the state evolves according to a linear dynamical system. This continuous time evolution is interspersed by discrete events that occur at random times and change (reset)…

The dynamics of a point particle in a periodic array of spherical scatterers converges, in the limit of small scatterer size, to a random flight process, whose paths are piecewise linear curves generated by a Markov process with memory two.…

Consider a Markov decision process (MDP) that admits a set of state-action features, which can linearly express the process's probabilistic transition model. We propose a parametric Q-learning algorithm that finds an approximate-optimal…

Suppose that $(X_t)_{t \ge 0}$ is a one-dimensional Brownian motion with negative drift $-\mu$. It is possible to make sense of conditioning this process to be in the state $0$ at an independent exponential random time and if we kill the…

In this paper, we study a continuous-time discounted jump Markov decision process with both controlled actions and observations. The observation is only available for a discrete set of time instances. At each time of observation, one has to…

We derive the KPZ equation as a continuum limit of height functions in asymmetric simple exclusion processes with drift that depends on the local particle configuration. To our knowledge, it is a first such result for a class of particle…

This paper discusses algorithms for solving Markov decision processes (MDPs) that have monotone optimal policies. We propose a two-stage alternating convex optimization scheme that can accelerate the search for an optimal policy by…

We study the time-bounded reachability problem for continuous-time Markov decision processes (CTMDPs) and games (CTMGs). Existing techniques for this problem use discretisation techniques to break time into discrete intervals, and optimal…

We prove that certain asymptotic moments exist for some random distance expanding dynamical systems and Markov chains in random dynamical environment, and compute them in terms of the derivatives at the $0$ of an appropriate pressure…

This paper presents with justifications a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. This technique produces an auxiliary…

Many processes, such as discrete event systems in engineering or population dynamics in biology, evolve in discrete space and continuous time. We consider the problem of optimal decision making in such discrete state and action space…

We study normal approximations for a class of discrete-time occupancy processes, namely, Markov chains with transition kernels of product Bernoulli form. This class encompasses numerous models which appear in the complex networks…