Related papers: Time Averages of Markov Processes and Applications…
We study algorithms to analyze a particular class of Markov population processes that is often used in epidemiology. More specifically, Markov binomial chains are the model that arises from stochastic time-discretizations of classical…
For a Markov process the detailed balance condition is equivalent to the time-reversibility of the process. For stochastic differential equations (SDE's) time discretization numerical schemes usually destroy the property of…
This is an expository review paper illustrating the ``martingale method'' for proving many-server heavy-traffic stochastic-process limits for queueing models, supporting diffusion-process approximations. Careful treatment is given to an…
We describe two algorithms for the eigenvalue, eigenvector problem which, on input a Gaussian matrix with complex entries, finish with probability 1 and in average polynomial time.
We exhibit conditions under which the flow of marginal distributions of a discontinuous semimartingale $\xi$ can be matched by a Markov process, whose infinitesimal generator is expressed in terms of the local characteristics of $\xi$. Our…
The problem of stochastic deadline scheduling is considered. A constrained Markov decision process model is introduced in which jobs arrive randomly at a service center with stochastic job sizes, rewards, and completion deadlines. The…
Computing smoothing distributions, the distributions of one or more states conditional on past, present, and future observations is a recurring problem when operating on general hidden Markov models. The aim of this paper is to provide a…
We study the multiple definitions of the entropy production for discrete-time Markov processes in single systems and composite systems. These definitions have been studied in single systems, but less so in composite systems. With a clear…
Semi-Markov processes are a generalization of Markov processes since the exponential distribution of time intervals is replaced with an arbitrary distribution. This paper provides an integro-differential form of the Kolmogorov's backward…
This paper establishes explicit solutions for fractional diffusion problems on bounded domains. It also gives stochastic solutions, in terms of Markov processes time-changed by an inverse stable subordinator whose index equals the order of…
In this paper we generalize known workload decomposition results for L\'{e}vy queues with secondary jump inputs and queues with server vacations or service interruptions. Special cases are polling systems with either compound Poisson or…
In this paper, we employ Markov process theory to prove asymptotic results for a class of stochastic processes which arise as solutions of a stochastic evolution inclusion and are given by the representation formula \begin{align*}…
We solve the problem of mean-variance hedging for general semimartingale models via stochastic control methods. After proving that the value process of the associated stochastic control problem has a quadratic structure, we characterize its…
We consider a Markov jump process on a general state space to which we apply a time-dependent weak perturbation over a finite time interval. By martingale-based stochastic calculus, under a suitable exponential moment bound for the…
An analogue of the classical Mecke formula for Poisson point processes is proved for the class of space-time STIT tessellation processes. From this key identity the Markov property of a class of associated random processes is derived. This…
We present a new class of interacting Markov chain Monte Carlo algorithms for solving numerically discrete-time measure-valued equations. The associated stochastic processes belong to the class of self-interacting Markov chains. In contrast…
In this paper, we study the averaging principle and central limit theorem for multi-scale stochastic differential equations with state-dependent switching. To accomplish this, we first study the Poisson equation associated with a Markov…
We consider stochastic discrete event dynamic systems that have time evolution represented with two-dimensional state vectors through a vector equation that is linear in terms of an idempotent semiring. The state transitions are governed by…
This paper is concerned with classes of models of stochastic reaction dynamics with time-scales separation. We demonstrate that the existence of the time-scale separation naturally leads to the application of the averaging principle and…
We give an extension of the $G$ method, with results, the extension and results being partly suggested by the finite Markov chains and specially by the finite-time consensus problem for the DeGroot model and that for the DeGroot model on…