Related papers: Practical Criterion for Uniqueness of $Q$-processe…

Continuous-time Markov decision processes are an important class of models in a wide range of applications, ranging from cyber-physical systems to synthetic biology. A central problem is how to devise a policy to control the system in order…

We consider a system of $N$ particles interacting through their empirical distribution on a finite state space in continuous time. In the formal limit as $N\to\infty$, the system takes the form of a nonlinear (McKean--Vlasov) Markov chain.…

We consider a Markov process $ X(t) $ on the nonnegative integers $E= S \cup \{0\}$, where $S=\{1,2,...\}$ is an irreducible class and 0 is an absorbing state. In this paper, we investigate conditions under which the quasi-stationary…

Non-Markovian processes may arise in physics due to memory effects of environmental degrees of freedom. For quantum non-Markovianity, it is an ongoing debate to clarify whether such memory effects have a verifiable quantum origin, or…

We consider a stochastic control problem where the set of strict (classical) controls is not necessarily convex and the the variable control has two components, the first being absolutely continuous and the second singular. The system is…

We consider a class of finite Markov moment problems with arbitrary number of positive and negative branches. We show criteria for the existence and uniqueness of solutions, and we characterize in detail the non-unique solution families.…

We study the long time behaviour of a Markov process evolving in $\mathbb{N}$ and conditioned not to hit 0. Assuming that the process comes back quickly from infinity, we prove that the process admits a unique quasi-stationary distribution…

We consider a class of pure jump Markov processes in $\rr^d$ whose jump kernels are comparable to those of symmetric stable processes. We prove a support theorem, a lower bound on the occupation times of sets, and show that we can…

We extend Rice Formula to a process which is the sum of two independent processes: a smooth process and a pure jump process with finitely many jumps. Formulas for the mean number of both continuous and discontinuous crossings through a…

We present sufficient conditions, in terms of the jumping kernels, for two large classes of conservative Markov processes of pure-jump type to be purely discontinuous martingales with finite second moment. As an application, we establish…

In this paper we give general criteria on tightness and weak convergence of discrete Markov chains to symmetric jump processes on metric measure spaces under mild conditions. As an application, we investigate discrete approximation for a…

We discuss the uniqueness of quantum states compatible with given results for measuring a set of observables. For a given pure state, we consider two different types of uniqueness: (1) no other pure state is compatible with the same…

We introduce an statistical mechanical formalism for the study of discrete-time stochastic processes with which we prove: (i) General properties of extremal chains, including triviality on the tail $\sigma$-algebra, short-range…

The Principle of Sufficient Reason implies determinism. An explicit indeterministic quantum jump dynamics is constructed, which may be naturally transformed into a deterministic one. A consistent application of the Principle of Sufficient…

When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This is done by considering as basic uncertainty models the so-called credal sets that…

An extension of the conditional expectations (those under a given subalgebra of events and not the simple ones under a single event) from the classical to the quantum case is presented. In the classical case, the conditional expectations…

Decision-theoretic planning with risk-sensitive planning objectives is important for building autonomous agents or decision-support systems for real-world applications. However, this line of research has been largely ignored in the…

Density dependent families of Markov chains, such as the stochastic models of mass-action chemical kinetics, converge for large values of the indexing parameter $N$ to deterministic systems of differential equations (Kurtz, 1970). Moreover…

In [ABM07], Abdulla et al. introduced the concept of decisiveness, an interesting tool for lifting good properties of finite Markov chains to denumerable ones. Later, this concept was extended to more general stochastic transition systems…

For Markov jump processes on irreducible networks with finite number of sites, we derive a general and explicit expression of the squared coefficient of variation for the net number of transitions from one site to a connected site in a…