Related papers: Markov two-components processes
In this paper, we introduce a time-continuous production model that enables random machine failures, where the failure probability depends historically on the production itself. This bidirectional relationship between historical failure…
We develop an approach to time-consistent risk evaluation of continuous-time processes in Markov systems. Our analysis is based on dual representation of coherent risk measures, differentiability concepts for multivalued mappings, and a…
A random phase property establishing a link between quasi-one-dimensional random Schroedinger operators and full random matrix theory is advocated. Briefly summarized it states that the random transfer matrices placed into a normal system…
We introduce the concept of a Markov influence system (MIS) and analyze its dynamics. An MIS models a random walk in a graph whose edges and transition probabilities change endogenously as a function of the current distribution. This…
We study an anytime control algorithm for situations where the processing resources available for control are time-varying in an a priori unknown fashion. Thus, at times, processing resources are insufficient to calculate control inputs. To…
We present a short introduction into the framework of piecewise deterministic Markov processes. We illustrate the abstract mathematical setting with a series of examples related to dispersal of biological systems, cell cycle models, gene…
Piecewise Deterministic Markov Processes (PDMPs) are studied in a general framework. First, different constructions are proven to be equivalent. Second, we introduce a coupling between two PDMPs following the same differential flow which…
A novel class of non-reversible Markov chain Monte Carlo schemes relying on continuous-time piecewise-deterministic Markov Processes has recently emerged. In these algorithms, the state of the Markov process evolves according to a…
We present a stochastic model predictive control (MPC) method for linear discrete-time systems subject to possibly unbounded and correlated additive stochastic disturbance sequences. Chance constraints are treated in analogy to robust MPC…
Hyperproperties are properties that describe the correctness of a system as a relation between multiple executions. Hyperproperties generalize trace properties and include information-flow security requirements, like noninterference, as…
We consider the synthesis of control policies for probabilistic systems, modeled by Markov decision processes, operating in partially known environments with temporal logic specifications. The environment is modeled by a set of Markov…
Markov jump processes (or continuous-time Markov chains) are a simple and important class of continuous-time dynamical systems. In this paper, we tackle the problem of simulating from the posterior distribution over paths in these models,…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…
Marked temporal point processes (MTPPs) model sequences of events occurring at irregular time intervals, with wide-ranging applications in fields such as healthcare, finance and social networks. We propose the state-space point process…
We present a learning model predictive control (MPC) scheme for chance-constrained Markov jump systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are…
We consider a class of semi-Markov processes (SMP) such that the embedded discrete time Markov chain may be non-homogeneous. The corresponding augmented processes are represented as semi-martingales using stochastic integral equation…
We study algorithms to analyze a particular class of Markov population processes that is often used in epidemiology. More specifically, Markov binomial chains are the model that arises from stochastic time-discretizations of classical…
The paper deals with a certain class of random evolutions. We develop a construction that yields an invariant measure for a continuous-time Markov process with random transitions. The approach is based on a particular way of constructing…
Markov decision process (MDP) is a decision making framework where a decision maker is interested in maximizing the expected discounted value of a stream of rewards received at future stages at various states which are visited according to…
This paper studies the stability of sampled and networked control systems with sampling and communication times governed by probabilistic clocks. The clock models have few restrictions, and can be used to model numerous phenomena such as…