Related papers: From Markov Processes to Semimartingales
We develop a martingale approach for studying continuous-time stochastic differential games of control and stopping, in a non-Markovian framework and with the control affecting only the drift term of the state-process. Under appropriate…
Using a newly introduced connection between the local and non-local description of open quantum system dynamics, we investigate the relationship between these two characterisations in the case of quantum semi-Markov processes. This class of…
The literature on Bayesian methods for the analysis of discrete-time semi-Markov processes is sparse. In this paper, we introduce the semi-Markov beta-Stacy process, a stochastic process useful for the Bayesian non-parametric analysis of…
In the paper we consider some piecewise deterministic Markov process whose continuous component evolves according to semiflows, which are switched at the jump times of a Poisson process. The associated Markov chain describes the states of…
We consider a Markovian evolution on point processes, the $\Psi$--process, on the unit interval in which points are added according to a rule that depends only on the spacings of the existing point configuration. Having chosen a spacing, a…
We construct a large class of non-Markovian master equations that describe the dynamics of open quantum systems featuring strong memory effects, which relies on a quantum generalization of the concept of classical semi-Markov processes.…
This manuscript contributes a general and practical framework for casting a Markov process model of a system at equilibrium as a structural causal model, and carrying out counterfactual inference. Markov processes mathematically describe…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
We develop an approach to time-consistent risk evaluation of continuous-time processes in Markov systems. Our analysis is based on dual representation of coherent risk measures, differentiability concepts for multivalued mappings, and a…
We show that a substantial portion of stochastic calculus can be developed along similar lines to ordinary calculus, with derivative-based concepts driving the development. We define a notion of stopping derivative, which is a form of right…
Markov decision processes continue to gain in popularity for modeling a wide range of applications ranging from analysis of supply chains and queuing networks to cognitive science and control of autonomous vehicles. Nonetheless, they tend…
In this note we connect the notion of solutions of a martingale problem to the notion of a strongly continuous and locally equi-continuous semigroup on the space of bounded continuous functions equipped with the strict topology. This…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…
We provide a new proof for regularity of affine processes on general state spaces by methods from the theory of Markovian semimartingales. On the way to this result we also show that the definition of an affine process, namely as…
New algorithms for computing power moments of hitting times and accumulated rewards of hitting type for semi-Markov processes. The algorithms are based on special techniques of sequential phase space reduction and recurrence relations…
An estimation method is proposed for a wide variety of discrete time stochastic processes that have an intractable likelihood function but are otherwise conveniently specified by an integral transform such as the characteristic function,…
In this paper, we introduce a class of processes that contains many natural examples. The interesting feature of such type processes lays on its infinite memory that allows it to record a quite ancient history. Then, using the martingale…
Markov models are often used to capture the temporal patterns of sequential data for statistical learning applications. While the Hidden Markov modeling-based learning mechanisms are well studied in literature, we analyze a…
We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators…
We discuss the conceptually different definitions used for the non-Markovianity of classical and quantum processes. The well-established definition for non-Markovianity of a classical stochastic process represents a condition on the…