Related papers: Revisiting the forward equations for inhomogeneous…
New algorithms for computing power moments of hitting times and accumulated rewards of hitting type for semi-Markov processes. The algorithms are based on special techniques of sequential phase space reduction and recurrence relations…
In this short note we consider semi-Markov processes satisfying the condition of direction-time independence (Markov renewal processes). We derive large deviation principles and fluctuation theorems for the empirical current and the…
The paper studies a class of critical Markov branching processes with infinite variance of the offspring distribution. The processes admit also an immigration component at the jump-points of a non-homogeneous Poisson process, assuming that…
The discrete class algorithm presented in this paper is an efficient simulation tool for stochastic processes governed by a reasonably small set of transition rates. The algorithm is presented, its performance compared to prevailing methods…
We provide probabilistic and computational results on Markovian multivariate Hawkes processes and induced population processes. By applying the Markov property, we characterize in closed form a joint transform, bijective to the probability…
In this paper, we consider a modified version of a well-known submartingale condition fortheweak convergence of probabilitymeasures, adapted to the semi-Markov case. In this setting, it is convenient to work with an embedded Markov chain…
In this paper we study a family of nonlinear (conditional) expectations that can be understood as a semimartingale with uncertain local characteristics. Here, the differential characteristics are prescribed by a time and path-dependent…
We provide a criterion for establishing lower bounds on the rate of convergence in $f$-variation of a continuous-time ergodic Markov process to its invariant measure. The criterion consists of novel super- and submartingale conditions for…
The representation theorem is obtained for functionals of non-Markov processes and their first exit times from bounded domains. These functionals are represented via solutions of backward parabolic Ito equations. As an example of…
Many integrable stochastic particle systems in one space dimension (such as TASEP - Totally Asymmetric Simple Exclusion Process - and its $q$-deformation, the $q$-TASEP) remain integrable if we equip each particle with its own speed…
For near-critical, transient Markov chains on the non-negative integers in the Lamperti regime, where the mean drift at $x$ decays as $1/x$ as $x \to \infty$, we quantify degree of transience via existence of moments for conditional return…
This paper provides full classification of dynamics for continuous time Markov chains (CTMCs) on the non-negative integers with polynomial transition rate functions. Such stochastic processes are abundant in applications, in particular in…
In this work we investigate the long-time behavior, that is the existence and characterization of invariant measures as well as convergence of transition probabilities, for Markov processes obtained as the unique mild solution to stochastic…
We study discrete-time discounted constrained Markov decision processes (CMDPs) on Borel spaces with unbounded reward functions. In our approach the transition probability functions are weakly or set-wise continuous. The reward functions…
Multi-state models are frequently applied for representing processes evolving through a discrete set of state. Important classes of multi-state models arise when transitions between states may depend on the time since entry into the current…
Consider a continuous time particle system $\eta^t=(\eta^t(k),k\in \mathbb{L})$, indexed by a lattice $\mathbb{L}$ which will be either $\mathbb{Z}$, $\mathbb{Z}/n\mathbb{Z}$, a segment $\{1,\cdots, n\}$, or $\mathbb{Z}^d$, and taking its…
An extension of the conditional expectations (those under a given subalgebra of events and not the simple ones under a single event) from the classical to the quantum case is presented. In the classical case, the conditional expectations…
Modeling the dynamics of non-stationary stochastic systems requires balancing the representational power of deep learning with the mathematical transparency of classical models. While classical Markov transition operators provide explicit,…
We present a general framework for applying learning algorithms and heuristical guidance to the verification of Markov decision processes (MDPs). The primary goal of our techniques is to improve performance by avoiding an exhaustive…
Necessary and sufficient conditions for convergence in distribution of first-rare-event times and convergence in Skorokhod J-topology of first-rare-event-time processes for perturbed semi-Markov processes with finite phase space are…