Related papers: Conditional Markov Chains Revisited Part I: Constr…

In this paper we continue the study of conditional Markov chains (CMCs) with finite state spaces, that we initiated in Bielecki, Jakubowski and Niew\k{e}g\l owski (2015). Here, we turn our attention to the study of Markov consistency and…

In this paper we study various properties of finite stochastic systems or hidden Markov chains as they are alternatively called. We discuss their construction following different approaches and we also derive recursive filtering formulas…

Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models…

This paper contributes an in-depth study of properties of continuous time Markov chains (CTMCs) on non-negative integer lattices $\N_0^d$, with particular interest in one-dimensional CTMCs with polynomial transitions rates. Such stochastic…

Continuous Time Markov Chain (CMTC) is widely used to describe and analyze systems in several knowledge areas. Steady state availability is one important analysis that can be made through Markov chain formalism that allows researchers…

In this review-type paper written at the occasion of the Oberwolfach workshop {\em One-sided vs. Two-sided stochastic processes} (february 22-29, 2020), we discuss and compare Markov properties and generalisations thereof in more…

Interactive Markov chains (IMC) are compositional behavioural models extending labelled transition systems and continuous-time Markov chains. We provide a framework and algorithms for compositional verification and optimization of IMC with…

Predictive constructions are a powerful way of characterizing the probability law of stochastic processes with certain forms of invariance, such as exchangeability or Markov exchangeability. When de Finetti-like representation theorems are…

Labeled continuous-time Markov chains (CTMCs) describe processes subject to random timing and partial observability. In applications such as runtime monitoring, we must incorporate past observations. The timing of these observations matters…

Quantum Markov chains (QMCs) are positive maps on a trace-class space describing open quantum dynamics on graphs. Such objects have a statistical resemblance with classical random walks, while at the same time it allows for internal…

For a stochastically monotone Markov chain taking values in a Polish space, we present a number of conditions for existence and for uniqueness of its stationary regime, as well as for closeness of its transient trajectories. In particular,…

We propose a Markov chain Monte Carlo (MCMC) scheme to perform state inference in non-linear non-Gaussian state-space models. Current state-of-the-art methods to address this problem rely on particle MCMC techniques and its variants, such…

Nonlinear Markov chains with finite state space have been introduced in Kolokoltsov (2010). The characteristic property of these processes is that the transition probabilities do not only depend on the state, but also on the distribution of…

Variable Length Memory Chains (VLMC), which are generalizations of finite order Markov chains, turn out to be an essential tool to modelize random sequences in many domains, as well as an interesting object in contemporary probability…

In the present paper, we give some examples of stochastic differential equations which have delicateness in the Markov and strong Markov properties, the uniqueness locally in time and globally in time, and initial conditions. Moreover, we…

Analyses of serially-sampled data often begin with the assumption that the observations represent discrete samples from a latent continuous-time stochastic process. The continuous-time Markov chain (CTMC) is one such generative model whose…

In this paper, we consider general Markov chains (MC), specified by the transition probability (kernel) $ P (x, E) $, finitely additive in the second argument. Such MC are studied within the framework of the functional operator treatment.…

In this paper we develop the elements of the theory of algorithmic randomness in continuous-time Markov chains (CTMCs). Our main contribution is a rigorous, useful notion of what it means for an individual trajectory of a CTMC to be random.…

We introduce the Conditional Mutual Information (CMI) for the estimation of the Markov chain order. For a Markov chain of $K$ symbols, we define CMI of order $m$, $I_c(m)$, as the mutual information of two variables in the chain being $m$…

Verification of infinite-state Markov chains is still a challenge despite several fruitful numerical or statistical approaches. For decisive Markov chains, there is a simple numerical algorithm that frames the reachability probability as…