Related papers: Markov two-components processes
We study a class of Markovian systems of $N$ elements taking values in $[0,1]$ that evolve in discrete time $t$ via randomized replacement rules based on the ranks of the elements. These rank-driven processes are inspired by variants of the…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
The modeling of natural phenomena via a Markov process --- a process for which the future is independent of the past, given the present--- is ubiquitous in many fields of science. Within this context, it is of foremost importance to develop…
In this article, we define the new concept of local coupling property for Markov processes and study its relationship with distributional properties of the transition probability. In the special case of L\'evy processes we show that this…
In this article we consider an aggregate loss model with dependent losses. The losses occurrence process is governed by a two-state Markovian arrival process (MAP2), a Markov renewal process process that allows for (1) correlated…
We consider symmetric Markov chains on $\Bbb Z^d$ where we do {\bf not} assume that the conductance between two points must be zero if the points are far apart. Under a uniform second moment condition on the conductances, we obtain upper…
This paper introduces the concept of random context representations for the transition probabilities of a finite-alphabet stochastic process. Processes with these representations generalize context tree processes (a.k.a. variable length…
Designing control policies for large, distributed systems is challenging, especially in the context of critical, temporal logic based specifications (e.g., safety) that must be met with high probability. Compositional methods for such…
Gibbs sampling is a Markov Chain Monte Carlo (MCMC) method often used in Bayesian learning. MCMC methods can be difficult to deploy on parallel and distributed systems due to their inherently sequential nature. We study asynchronous Gibbs…
We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…
We formulate a probabilistic Markov property in discrete time under a dynamic risk framework with minimal assumptions. This is useful for recursive solutions to risk-sensitive versions of dynamic optimisation problems such as optimal…
The fundamental concepts underlying in Markov networks are the conditional independence and the set of rules called Markov properties that translates conditional independence constraints into graphs. In this article we introduce the concept…
Marked Temporal Point Process (MTPP) has been well studied to model the event distribution in marked event streams, which can be used to predict the mark and arrival time of the next event. However, existing studies overlook that the…
Controllers for autonomous systems that operate in safety-critical settings must account for stochastic disturbances. Such disturbances are often modelled as process noise, and common assumptions are that the underlying distributions are…
In this paper, we present a novel framework to synthesize robust strategies for discrete-time nonlinear systems with random disturbances that are unknown, against temporal logic specifications. The proposed framework is data-driven and…
We propose a sequential Markov chain Monte Carlo (SMCMC) algorithm to sample from a sequence of probability distributions, corresponding to posterior distributions at different times in on-line applications. SMCMC proceeds as in usual MCMC…
It is well-known that compositions of Markov processes with inverse subordinators are governed by integro-differential equations of generalized fractional type. This kind of processes are of wide interest in statistical physics as they are…
Conditional independence and Markov properties are powerful tools allowing expression of multidimensional probability distributions by means of low-dimensional ones. As multidimensional possibilistic models have been studied for several…
Distributed coordination algorithms (DCA) carry out information processing processes among a group of networked agents without centralized information fusion. Though it is well known that DCA characterized by an SIA (stochastic,…
A piecewise-deterministic Markov process, specified by random jumps and switching semi-flows, as well as the associated Markov chain given by its post-jump locations, are investigated in this paper. The existence of an exponentially…