Related papers: From empirical data to continuous Markov processes…
Stochastic convergence of discrete time Markov processes has been analysed based on a dual Lyapunov approach. Using some existing results on ergodic theory of Markov processes, it has been shown that existence of a properly subinvariant…
We consider the filtering of continuous-time finite-state hidden Markov models, where the rate and observation matrices depend on unknown time-dependent parameters, for which no prior or stochastic model is available. We quantify and…
Learning to sample from complex unnormalized distributions is a fundamental challenge in computational physics and machine learning. While score-based and variational methods have achieved success in continuous domains, extending them to…
The notion of a successful coupling of Markov processes, based on the idea that both components of the coupled system ``intersect'' in finite time with probability one, is extended to cover situations when the coupling is unnecessarily…
This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of $\omega$-regular properties can be reduced to the…
Markov jump processes are continuous-time stochastic processes with a wide range of applications in both natural and social sciences. Despite their widespread use, inference in these models is highly non-trivial and typically proceeds via…
This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent…
In the context of Markov decision processes running in continuous time, one of the most intriguing challenges is the efficient approximation of finite horizon reachability objectives. A multitude of sophisticated model checking algorithms…
There is a lack of methodological results for continuous time change detection due to the challenges of noninformative prior specification and efficient posterior inference in this setting. Most methodologies to date assume data are…
In this paper, we present a kernel-based, multi-task Gaussian Process (GP) model for approximating the underlying function of an individual's mobility state using a time-inhomogeneous Markov Process with two states: moves and pauses. Our…
This paper provides full classification of dynamics for continuous time Markov chains (CTMCs) on the non-negative integers with polynomial transition rate functions. Such stochastic processes are abundant in applications, in particular in…
Empirical processes for stationary, causal sequences are considered. We establish empirical central limit theorems for classes of indicators of left half lines, absolutely continuous functions and piecewise differentiable functions. Sample…
A measure-preserving dynamical system can be approximated by a Markov shift with a bistochastic matrix. This leads to using empirical stochastic matrices to measure and estimate properties of stirring protocols. Specifically, the second…
Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…
The paper deals with a certain class of random evolutions. We develop a construction that yields an invariant measure for a continuous-time Markov process with random transitions. The approach is based on a particular way of constructing…
Dynamical processes can be transformed into graphs through a family of mappings called visibility algorithms, enabling the possibility of (i) making empirical data analysis and signal processing and (ii) characterising classes of dynamical…
The goal of this paper is to develop a general method to establish conditional ergodicity of infinite-dimensional Markov chains. Given a Markov chain in a product space, we aim to understand the ergodic properties of its conditional…
State-space models are commonly used to describe different forms of ecological data. We consider the case of count data with observation errors. For such data the system process is typically multi-dimensional consisting of coupled Markov…
We consider a large family of discrete and continuous time controlled Markov processes and study an ergodic risk-sensitive minimization problem. Under a blanket stability assumption, we provide a complete analysis to this problem. In…
Recently, a class of stochastic processes known as piecewise deterministic Markov processes has been used to define continuous-time Markov chain Monte Carlo algorithms with a number of attractive properties, including compatibility with…