Related papers: Timed Comparisons of Semi-Markov Processes

We introduce LAMP: the Linear Additive Markov Process. Transitions in LAMP may be influenced by states visited in the distant history of the process, but unlike higher-order Markov processes, LAMP retains an efficient parametrization. LAMP…

Markov decision processes model systems subject to nondeterministic and probabilistic uncertainty. A plethora of verification techniques addresses variations of reachability properties, such as: Is there a scheduler resolving the…

Markov models are widely used to describe processes of stochastic dynamics. Here, we show that Markov models are a natural consequence of the dynamical principle of Maximum Caliber. First, we show that when there are different possible…

This paper tackles the issue of establishing a lower-bound on the asymptotic ratio of survival probabilities between two different initial conditions, asymptotically in time for a given Markov process with extinction. Such a comparison is a…

Recently, a class of stochastic processes known as piecewise deterministic Markov processes has been used to define continuous-time Markov chain Monte Carlo algorithms with a number of attractive properties, including compatibility with…

We study algorithms to analyze a particular class of Markov population processes that is often used in epidemiology. More specifically, Markov binomial chains are the model that arises from stochastic time-discretizations of classical…

The inference of thermodynamic quantities from the description of an only partially accessible physical system is a central challenge in stochastic thermodynamics. A common approach is coarse-graining, which maps the dynamics of such a…

Let S be a denumerable state space and let P be a transition probability matrix on S. If a denumerable set M of nonnegative matrices is such that the sum of the matrices is equal to P, then we call M a partition of P. Let K denote the set…

A comparison theorem for state-dependent regime-switching diffusion processes is established, which enables us to control pathwisely the evolution of the state-dependent switching component simply by Markov chains. Moreover, a sharp…

Piecewise Deterministic Markov Processes (PDMPs) are studied in a general framework. First, different constructions are proven to be equivalent. Second, we introduce a coupling between two PDMPs following the same differential flow which…

We study the long time behaviour of a Markov process evolving in $\mathbb{N}$ and conditioned not to hit 0. Assuming that the process comes back quickly from infinity, we prove that the process admits a unique quasi-stationary distribution…

In spiking neural networks, the information is conveyed by the spike times, that depend on the intrinsic dynamics of each neuron, the input they receive and on the connections between neurons. In this article we study the Markovian nature…

Continuous-time Markov processes over finite state-spaces are widely used to model dynamical processes in many fields of natural and social science. Here, we introduce an maximum likelihood estimator for constructing such models from data…

This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine…

The notion of a successful coupling of Markov processes, based on the idea that both components of the coupled system ``intersect'' in finite time with probability one, is extended to cover situations when the coupling is unnecessarily…

We consider a general method for the approximation of the distribution of a process conditioned to not hit a given set. Existing methods are based on particle system that are failable, in the sense that, in many situations , they are not…

This paper studies temporal planning in probabilistic environments, modeled as labeled Markov decision processes (MDPs), with user preferences over multiple temporal goals. Existing works reflect such preferences as a prioritized list of…

We develop a semi-parametric state-space model for time-series data with latent regime transitions. Classical Markov-switching models use fixed parametric transition functions, such as logistic or probit links, which restrict flexibility…

Filtering---estimating the state of a partially observable Markov process from a sequence of observations---is one of the most widely studied problems in control theory, AI, and computational statistics. Exact computation of the posterior…

Markov decision processes (MDP) are a well-established model for sequential decision-making in the presence of probabilities. In robust MDP (RMDP), every action is associated with an uncertainty set of probability distributions, modelling…