Related papers: Pathwise duals of monotone and additive Markov pro…
Dating from the work of Neuts in the 1980s, the field of matrix-analytic methods has been developed to analyse discrete or continuous-time Markov chains with a two-dimensional state space in which the increment of a level variable is…
Dynamical processes can be transformed into graphs through a family of mappings called visibility algorithms, enabling the possibility of (i) making empirical data analysis and signal processing and (ii) characterising classes of dynamical…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
A two-type continuous-state branching process in varying environments is constructed as the pathwise unique solution of a system of stochastic equations driven by time-space noises, where the pathwise uniqueness is derived from a comparison…
We consider average-cost Markov decision processes (MDPs) with Borel state spaces, countable, discrete action spaces, and strictly unbounded one-stage costs. For the minimum pair approach, we introduce a new majorization condition on the…
We study nonlinear time-inhomogeneous Markov processes in the sense of McKean's seminal work [32]. These are given as families of laws $\mathbb{P}_{s,\zeta}$, $s\geq 0$, on path space, where $\zeta$ runs through a set of admissible initial…
In this paper, we consider general Markov chains (MC), specified by the transition probability (kernel) $ P (x, E) $, finitely additive in the second argument. Such MC are studied within the framework of the functional operator treatment.…
Duality transformations reveal unexpected equivalences between seemingly distinct models. We introduce an out-of-equilibrium generalisation of matrix product operators to implement duality transformations in one-dimensional boundary-driven…
We construct path integrals for stochastic hybrid reaction-diffusion (RD) processes, in which the reaction terms depend on the discrete state of a randomly switching environment. We proceed by spatially discretizing a given RD system and…
Markov combination is an operation that takes two statistical models and produces a third whose marginal distributions include those of the original models. Building upon and extending existing work in the Gaussian case, we develop Markov…
Lattice systems with certain Lie algebraic or quantum Lie algebraic symmetries are constructed. These symmetric models give rise to series of integrable systems. As examples the $A_n$-symmetric chain models and the SU(2)-invariant ladder…
Markov decision process over vector addition system with states (VASS MDP) is a finite state model combining non-deterministic and probabilistic behavior, augmented with non-negative integer counters that can be incremented or decremented…
We develop the theory linking 'E-separation' in directed mixed graphs (DMGs) with conditional independence relations among coordinate processes in stochastic differential equations (SDEs), where causal relationships are determined by "which…
We describe an approach for exploiting structure in Markov Decision Processes with continuous state variables. At each step of the dynamic programming, the state space is dynamically partitioned into regions where the value function is the…
Finite order Markov models are theoretically well-studied models for dependent discrete data. Despite their generality, application in empirical work when the order is large is rare. Practitioners avoid using higher order Markov models…
We develop an exact framework to describe the non-Markovian dynamics of an open quantum system interacting with an environment modeled by a generalized spectral density function. The approach relies on mapping the initial system onto an…
We are studying stationary random processes with conditional polynomial moments that allow a continuous path modification. Processes with continuous path modification, are important because they are relatively easy to simulate. One does not…
The infinite discrete stable Boltzmann maps are "heavy-tailed" generalisations of the well-known Uniform Infinite Planar Quadrangulation. Very efficient tools to study these objects are Markovian step-by-step explorations of the lattice…
We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the…
We construct and study branching Markov processes on the space of finite configurations of the state space of a given standard process, controlled by a branching kernel and a killing one. In particular, we may start with a superprocess,…