Related papers: Exchangeable, Markov multi-state survival process
We establish sufficient conditions for exponential convergence to a unique quasi-stationary distribution in the total variation norm. These conditions also ensure the existence and exponential ergodicity of the Q-process, the process…
Asymptotic properties of Markov Processes, such as steady state probabilities or hazard rate for absorbing states can be efficiently calculated by means of linear algebra even for large-scale problems. This paper discusses the methods for…
Switching dynamical systems are an expressive model class for the analysis of time-series data. As in many fields within the natural and engineering sciences, the systems under study typically evolve continuously in time, it is natural to…
Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
Population dynamics are often subject to random independent changes in the environment. For the two strategy stochastic replicator dynamic, we assume that stochastic changes in the environment replace the payoffs and variance. This is…
Arguing about the equilibrium distribution of continuous-time Markov chains can be vital for showing properties about the underlying systems. For example in biological systems, bistability of a chemical reaction network can hint at its…
Multi-state survival analysis considers several potential events of interest along a disease pathway. Such analyses are crucial to model complex patient trajectories and are increasingly being used in epidemiological and health economic…
In this paper, we demonstrate through the use of matrix calculus a transparent analysis of fractional inhomogeneous Markov models for life insurance where transition matrices commute. The resulting formulae are intuitive matrix…
Dynamics on networks is considered from the perspective of Markov stochastic processes. We partially describe the state of the system through network motifs and infer any missing data using the available information. This versatile approach…
Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…
In this paper, we study consistent and partially exchangeable sequences of Markov chains on a finite state space. We provide a characterisation of the admissible transition rates via a decomposition into individual and coordinated motion of…
Non-Markovian dynamics are ubiquitous across physics, biology, and engineering. Yet our understanding of non-Markovian processes significantly lags that of simpler Markovian processes, due largely to a lack of tractable models. In this…
We propose a solution to a time-varying variant of Markov Decision Processes which can be used to address decision-theoretic planning problems for autonomous systems operating in unstructured outdoor environments. We explore the time…
Every exchangeable Feller process taking values in a suitably nice combinatorial state space can be constructed by a system of iterated random Lipschitz functions. In discrete time, the construction proceeds by iterated application of…
Epidemic dynamics in a stochastic network of interacting epidemic centers is considered. The epidemic and migration processes are modelled by Markov's chains. Explicit formulas for probability distribution of the migration process are…
Markov Chains with variable length are useful stochastic models for data compression that avoid the curse of dimensionality faced by that full Markov Chains. In this paper we introduce a Variable Length Markov Chain whose transition…
We consider a continuous time Markov process on $\mathbb{N}_0$ which can be interpreted as generalized alternating birth-death process in a non-autonomous random environment. Depending on the status of the environment the process either…
Experiments, in particular on biological systems, typically probe lower-dimensional observables which are projections of high-dimensional dynamics. In order to infer consistent models capturing the relevant dynamics of the system, it is…
We introduce an ensemble Markov chain Monte Carlo approach to sampling from a probability density with known likelihood. This method upgrades an underlying Markov chain by allowing an ensemble of such chains to interact via a process in…