Related papers: Time-changed Markov processes and coupled non-loca…
We consider a Markov jump process on a general state space to which we apply a time-dependent weak perturbation over a finite time interval. By martingale-based stochastic calculus, under a suitable exponential moment bound for the…
The purpose of this paper is to consider the exit-time problem for a finite-range Markov jump process, i.e, the distance the particle can jump is bounded independent of its location. Such jump diffusions are expedient models for anomalous…
We consider a continuous-time financial market with an asset whose price is modeled by a linear stochastic differential equation with drift and volatility switching driven by a uniformly ergodic jump Markov process with a countable state…
The `local time on curves' formula of Peskir provides a stochastic change of variables formula for a function whose derivatives may be discontinuous over a time-dependent curve, a setting which occurs often in applications in optimal…
The objective of this work is to study continuous-time Markov decision processes on a general Borel state space with both impulsive and continuous controls for the infinite-time horizon discounted cost. The continuous-time controlled…
We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…
Modern methods of simulating molecular systems are based on the mathematical theory of Markov operators with a focus on autonomous equilibrated systems. However, non-autonomous physical systems or non-autonomous simulation processes are…
In this short paper, we consider discrete-time Markov chains on lattices as approximations to continuous-time diffusion processes. The approximations can be interpreted as finite difference schemes for the generator of the process. We…
This paper deals with the long run average continuous control problem of piecewise deterministic Markov processes (PDMP's) taking values in a general Borel space and with compact action space depending on the state variable. The control…
The problem of appropriately matching items subject to compatibility constraints arises in a number of important applications. While most of the literature on matching theory focuses on a static setting with a fixed number of items, several…
In this paper, we study a continuous-time discounted jump Markov decision process with both controlled actions and observations. The observation is only available for a discrete set of time instances. At each time of observation, one has to…
Markovian diffusion processes yield a system of conservation laws which couple various conditional expectation values (local moments). Solutions of that closed system of deterministic partial differential equations stand for a regular…
We present two related anytime algorithms for control of nonlinear systems when the processing resources available are time-varying. The basic idea is to calculate tentative control input sequences for as many time steps into the future as…
Markov jump processes are continuous-time stochastic processes with a wide range of applications in both natural and social sciences. Despite their widespread use, inference in these models is highly non-trivial and typically proceeds via…
We consider continuous and discontinuous Galerkin time stepping methods of arbitrary order as applied to nonlinear initial value problems in real Hilbert spaces. Our only assumption is that the nonlinearities are continuous; in particular,…
Anomalous diffusions arise as scaling limits of continuous-time random walks (CTRWs) whose innovation times are distributed according to a power law. The impact of a non-exponential waiting time does not vanish with time and leads to…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
We study non-stationary stochastic processes arising from sequential dynamical systems built on maps with a neutral fixed points and prove the existence of Extreme Value Laws for such processes. We use an approach developed in \cite{FFV16},…
This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…