Related papers: Markov processes of cubic stochastic matrices: {\i…
The chemical master equation (CME) is frequently used in systems biology to quantify the effects of stochastic fluctuations that arise due to biomolecular species with low copy numbers. The CME is a system of ordinary differential equations…
Matrix product states provide a natural entanglement basis to represent a quantum register and operate quantum gates on it. This scheme can be materialized to simulate a quantum adiabatic algorithm solving hard instances of a NP-Complete…
We introduce a new class of continuous matrix product (CMP) states and establish the stochastic master equations (quantum filters) for an arbitrary quantum system probed by a bosonic input field in this class of states. We show that this…
The goal of a traditional Markov decision process (MDP) is to maximize expected cumulative reward over a defined horizon (possibly infinite). In many applications, however, a decision maker may be interested in optimizing a specific…
We introduce the notion of Quasi-Stationary State (QSS) in the context of quantum Markov semigroups that generalizes the one of quasi-stationary distribution in the case of classical Markov chains. We provide an operational interpretation…
Markov population models (MPMs) are a widely used modelling formalism in the area of computational biology and related areas. The semantics of a MPM is an infinite-state continuous-time Markov chain. In this paper, we use the established…
The field of classical stochastic processes forms a major branch of mathematics. They are, of course, also very well studied in biology, chemistry, ecology, geology, finance, physics, and many more fields of natural and social sciences.…
Weconsider Markov decision processes arising from a Markov model of an underlying natural phenomenon. Such phenomena are usually periodic (e.g. annual) in time, and so the Markov processes modelling them must be time-inhomogeneous, with…
We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…
This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
We introduce a family of Markov processes on set partitions with a bounded number of blocks, called Lipschitz partition processes. We construct these processes explicitly by a Poisson point process on the space of Lipschitz continuous maps…
This paper introduces several new classes of mathematical structures that have close connections with physics and with the theory of dynamical systems. The most general of these structures, called indivisible stochastic processes,…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
We study K-processes, which are Markov processes in a denumerable state space, all of whose elements are stable, with the exception of a single state, starting from which the process enters finite sets of stable states with uniform…
The notion of stability can be generalised to point processes by defining the scaling operation in a randomised way: scaling a configuration by $t$ corresponds to letting such a configuration evolve according to a Markov branching particle…
Motivated by queueing systems with heterogeneous parallel servers, we consider a class of structured multi-dimensional Markov processes whose state space can be partitioned into two parts: a finite set of boundary states and a structured…
Hybrid systems, and Piecewise Deterministic Markov Processes in particular, are widely used to model and numerically study systems exhibiting multiple time scales in biochemical reaction kinetics and related areas. In this paper an almost…
Stochastic matrices and positive maps in matrix algebras proved to be very important tools for analysing classical and quantum systems. In particular they represent a natural set of transformations for classical and quantum states,…
Quantum theory predicts probabilities as well as relative phases between different alternatives of the system. A unified description of both probabilities and phases comes through a generalisation of the notion of a density matrix for…