Related papers: Markov two-components processes
We consider a class of piecewise-deterministic Markov processes where the state evolves according to a linear dynamical system. This continuous time evolution is interspersed by discrete events that occur at random times and change (reset)…
The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…
Stochastic models for performance analysis, optimization and control of queues hinge on a multitude of alternatives for input point processes. In case of bursty traffic, one very popular model is the \textit{Markov Modulated Poisson…
The stability of stochastic Model Predictive Control (MPC) subject to additive disturbances is often demonstrated in the literature by constructing Lyapunov-like inequalities that guarantee closed-loop performance bounds and boundedness of…
A possibly time-dependent transition intensity matrix or generator $(Q(t))$ characterizes the law of a Markov jump process (MP). For a time homogeneous MP, the transition probability matrix (TPM) can be expressed as a matrix exponential of…
We introduce multiple hidden Markov models (MHMMs) where an observed multivariate categorical time series depends on an unobservable multivariate Mar- kov chain. MHMMs provide an elegant framework for specifying various independence…
Causal reversibility blends reversibility and causality for concurrent systems. It indicates that an action can be undone provided that all of its consequences have been undone already, thus making it possible to bring the system back to a…
We study Markov processes conditioned so that their local time must grow slower than a prescribed function. Building upon recent work on Brownian motion with constrained local time in [5] and [33], we study transience and recurrence for a…
In this paper we propose a stochastic model predictive control (MPC) algorithm for linear discrete-time systems affected by possibly unbounded additive disturbances and subject to probabilistic constraints. Constraints are treated in…
The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant "additivity" properties: (i) existence of a…
We consider Markov decision processes (MDP) as generators of sequences of probability distributions over states. A probability distribution is p-synchronizing if the probability mass is at least p in a single state, or in a given set of…
The literature on Bayesian methods for the analysis of discrete-time semi-Markov processes is sparse. In this paper, we introduce the semi-Markov beta-Stacy process, a stochastic process useful for the Bayesian non-parametric analysis of…
Interactive Markov chains (IMC) are compositional behavioural models extending labelled transition systems and continuous-time Markov chains. We provide a framework and algorithms for compositional verification and optimization of IMC with…
In this paper, we consider a general class of two-time-scale Markov chains whose transition rate matrix depends on a parameter $\lambda>0$. We assume that some transition rates of the Markov chain will tend to infinity as…
In this paper, we present a kernel-based, multi-task Gaussian Process (GP) model for approximating the underlying function of an individual's mobility state using a time-inhomogeneous Markov Process with two states: moves and pauses. Our…
A discrete time branching process where the offspring distribution is generation-dependent, and the number of reproductive individuals is controlled by a random mechanism is considered. This model is a Markov chain but, in general, the…
The problem of appropriately matching items subject to compatibility constraints arises in a number of important applications. While most of the literature on matching theory focuses on a static setting with a fixed number of items, several…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
There is a lack of methodological results for continuous time change detection due to the challenges of noninformative prior specification and efficient posterior inference in this setting. Most methodologies to date assume data are…
We introduce a probabilistic framework for two-sample comparison based on a nonparametric process taking the form of a Markov model that transitions between a "divide" and a "merge" state on a multi-resolution partition tree of the sample…