Related papers: Occupation Times for Jump Processes
A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be…
We propose a method for approximating solutions to optimization problems involving the global stability properties of parameter-dependent continuous-time autonomous dynamical systems. The method relies on an approximation of the…
We present an abstract framework for establishing smoothing properties within a specific class of inhomogeneous discrete-time Markov processes. These properties, in turn, serve as a basis for demonstrating the existence of density functions…
We study computational and statistical aspects of learning Latent Markov Decision Processes (LMDPs). In this model, the learner interacts with an MDP drawn at the beginning of each epoch from an unknown mixture of MDPs. To sidestep known…
We study the fluctuations of systems modeled by Markov jump processes with periodic generators. We focus on observables defined through time-periodic functions of the system's states or transitions. Using large deviation theory, canonical…
A jump process for the positions of interacting quantum particles on a lattice, with time-dependent transition rates governed by the state vector, was first considered by J.S. Bell. We review this process and its continuum variants…
This paper studies function approximation for finite horizon discrete time Markov decision processes under certain convexity assumptions. Uniform convergence of these approximations on compact sets is proved under several sampling schemes…
We consider a Markov jump process on a general state space to which we apply a time-dependent weak perturbation over a finite time interval. By martingale-based stochastic calculus, under a suitable exponential moment bound for the…
This paper presents a set of results relating to the occupation time $\alpha(t)$ of a process $X(\cdot)$. The first set of results concerns exact characterizations of $\alpha(t)$ for $t\geq0$, e.g., in terms of its transform up to an…
Dunkl processes are multidimensional Markov processes defined through the use of Dunkl operators. These processes have discontinuities, and they can be separated into their continuous (radial) part, and their discontinuous (jump) part.…
We develop algorithms with low regret for learning episodic Markov decision processes based on kernel approximation techniques. The algorithms are based on both the Upper Confidence Bound (UCB) as well as Posterior or Thompson Sampling…
We study the large time behavior of the survival probability $\mathbb{P}_x\left(\tau_D>t\right)$ for symmetric jump processes in unbounded domains with a positive bottom of the spectrum. We prove asymptotic upper and lower bounds with…
We introduce a framework to approximate a Markov Decision Process that stands on two pillars: state aggregation -- as the algorithmic infrastructure; and central-limit-theorem-type approximations -- as the mathematical underpinning of…
We prove that the restriction of the vertex-reinforced jump process to a subset of the vertex set is a mixture of vertex-reinforced jump processes. A similar statement holds for the non-linear hyperbolic supersymmetric sigma model. This is…
We present sufficient conditions for the transience and the existence of local times of a Feller process, and the ultracontractivity of the associated Feller semigroup; these conditions are sharp for L\'{e}vy processes. The proof uses a…
Consider a system of interacting particles indexed by the nodes of a graph whose vertices are equipped with marks representing parameters of the model such as the environment or initial data. Each particle takes values in a countable state…
Sufficient conditions for a symmetric jump-diffusion process to be conservative and recurrent are given in terms of the volume of the state space and the jump kernel of the process. A number of examples are presented to illustrate the…
We consider a model of interacting neurons where the membrane potentials of the neurons are described by a multidimensional piecewise deterministic Markov process (PDMP) with values in ${\mathbb R}^N, $ where $ N$ is the number of neurons…
We develop criteria for recurrence and transience of one-dimensional Markov processes which have jumps and oscillate between $+\infty$ and $-\infty$. The conditions are based on a Markov chain which only consists of jumps (overshoots) of…
We study the almost sure convergence of the occupation measure of evolution models where mutation rates decrease over time. We show that if the mutation parameter vanishes at a controlled rate, then the empirical occupation measure…