Related papers: Nonuniform-Grid Markov Chain Approximation of Cont…

We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…

We propose a Bayesian nonparametric mixture model for the reconstruction and prediction from observed time series data, of discretized stochastic dynamical systems, based on Markov Chain Monte Carlo methods (MCMC). Our results can be used…

Finite state machines (FSMs) are a theoretically and practically important model of computation. We propose a general, thermodynamically consistent model of FSMs and characterise the resource requirements of these machines. We model FSMs as…

Semi-Markov processes generalize Markov processes by adding temporal memory effects as expressed by a semi-Markov kernel. We recall the path weight for a semi-Markov trajectory and the fact that thermodynamic consistency in equilibrium…

In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior…

We study long time behavior of a discrete time weakly interacting particle system, and the corresponding nonlinear Markov process in $\mathbb{R}^d$, described in terms of a general stochastic evolution equation. In a setting where the state…

In this work, we generalize the concept of bisimulation metric in order to metrize the behaviour of continuous-time processes. Similarly to what is done for discrete-time systems, we follow two approaches and show that they coincide: as a…

We present a novel algorithm to solve a non-linear system of equations, whose solution can be interpreted as a tight lower bound on the vector of expected hitting times of a Markov chain whose transition probabilities are only partially…

This work introduces a notion of approximate probabilistic trace equivalence for labelled Markov chains, and relates this new concept to the known notion of approximate probabilistic bisimulation. In particular this work shows that the…

This paper deals with state estimation of stochastic models with linear state dynamics, continuous or discrete in time. The emphasis is laid on a numerical solution to the state prediction by the time-update step of the grid-point-based…

The dynamic emulation of non-linear deterministic computer codes where the output is a time series, possibly multivariate, is examined. Such computer models simulate the evolution of some real-world phenomenon over time, for example models…

In order to approximate a continuous time stochastic process by discrete time Markov chains one has several options to embed the Markov chains into continuous time processes. On the one hand there is the Markov embedding, which uses…

We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…

In this paper, we deal with a class of time-homogeneous continuous-time Markov processes with transition probabilities bearing a nonparametric uncertainty. The uncertainty is modeled by considering perturbations of the transition…

We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…

We consider the analysis of sets of categorical sequences consisting of piecewise homogeneous Markov segments. The sequences are assumed to be governed by a common underlying process with segments occurring in the same order for each…

We study the so-called two-time-scale stochastic approximation, a simulation-based approach for finding the roots of two coupled nonlinear operators. Our focus is to characterize its finite-time performance in a Markov setting, which often…

We propose a method for approximating the large deviation rate function of time-integrated observables of diffusion processes, used in statistical physics to characterize the fluctuations of nonequilibrium systems. The method is based on…

In this paper, we present a kernel-based, multi-task Gaussian Process (GP) model for approximating the underlying function of an individual's mobility state using a time-inhomogeneous Markov Process with two states: moves and pauses. Our…

For continuous-time Markov jump processes on irreducible networks with time-independent rate constants, we employ a transition-based formalism to express the long-time precision of a single integrated current over an observable channel in…