Related papers: On discrete-time semi-Markov processes
The article presents a general discrete time dividend valuation model when the dividend growth rate is a general continuous variable. The main assumption is that the dividend growth rate follows a discrete time semi-Markov chain with…
In this note, we present few examples of Piecewise Deterministic Markov Processes and their long time behavior. They share two important features: they are related to concrete models (in biology, networks, chemistry,. . .) and they are…
A canonical formalism and constraint analysis for discrete systems subject to a variational action principle are devised. The formalism is equivalent to the covariant formulation, encompasses global and local discrete time evolution moves…
In this paper, we outline a model of graph (or network) dynamics based on two ingredients. The first ingredient is a Markov chain on the space of possible graphs. The second ingredient is a semi-Markov counting process of renewal type. The…
Changing time of simple continuous-time Markov counting processes by independent unit-rate Poisson processes results in Markov counting processes for which we provide closed-form transition rates via composition of trajectories and with…
In this paper continuous time random walk models approximating fractional space-time diffusion processes are studied. Stochastic processes associated with the considered equations represent time-changed processes, where the time-change…
A discrete-time method for solving problems in optimal quantum control is presented. Controlling the time discretized markovian dynamics of a quantum system can be reduced to a Markov-decision process. We demonstrate this method in this…
Using a newly introduced connection between the local and non-local description of open quantum system dynamics, we investigate the relationship between these two characterisations in the case of quantum semi-Markov processes. This class of…
We develop an Onsager-Machlup-type theory for nonequilibrium semi-Markov processes. Our main result is an exact large time asymptotics for the joint probability of the occupation times and the currents in the system, establishing some…
In this article we study a class of stochastic functional differential equations driven by L\'{e}vy processes (in particular, $\alpha$-stable processes), and obtain the existence and uniqueness of Markov solutions in small time intervals.…
There is a growing interest in methods for detecting and interpreting changes in experimental time evolution data. Based on measured time series, the quantitative characterization of dynamical phase transitions at bifurcation points of the…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…
We study stochastic differential equations (SDEs) whose drift and diffusion coefficients are path-dependent and controlled. We construct a value process on the canonical path space, considered simultaneously under a family of singular…
Piecewise Deterministic Markov Processes (PDMPs) are studied in a general framework. First, different constructions are proven to be equivalent. Second, we introduce a coupling between two PDMPs following the same differential flow which…
Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…
In this research paper, weighted / unweighted, directed / undirected graphs are associated with interesting Discrete Time Markov Chains (DTMCs) as well as Continuous Time Markov Chains (CTMCs). The equilibrium / transient behaviour of such…
The time reversal of a completely-positive, nonequilibrium discrete-time quantum Markov evolution is derived via a suitable adjointness relation. Space-time harmonic processes are introduced for the forward and reverse-time transition…
This paper examines a discrete-time queuing system with applications to telecommunications traffic. The arrival process is a particular Markov modulated process which belongs to the class of discrete batched Markovian arrival processes. The…
We study normal approximations for a class of discrete-time occupancy processes, namely, Markov chains with transition kernels of product Bernoulli form. This class encompasses numerous models which appear in the complex networks…
We have shown recently that a Markov process conditioned on rare events involving time-integrated random variables can be described in the long-time limit by an effective Markov process, called the driven process, which is given…