Related papers: Semimartingale dynamics and estimation for a semi-…
We consider a strictly substochastic matrix or an stochastic matrix with absorbing states. By using quasi-stationary distributions one shows there is a canonical associated stationary Markov chain. Based upon $2-$stringing representation of…
Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical…
We develop a systematic matrix-analytic approach, based on intertwinings of Markov semigroups, for proving theorems about hitting-time distributions for finite-state Markov chains -- an approach that (sometimes) deepens understanding of the…
We analyse dynamical large deviations of quantum trajectories in Markovian open quantum systems in their full generality. We derive a {\em quantum level-2.5 large deviation principle} for these systems, which describes the joint…
We estimate the distance in total variation between the law of a finite state Markov process at time t, starting from a given initial measure, and its unique invariant measure. We derive upper bounds for the time to reach the equilibrium.…
At present, the problem to steer a non-Markovian process with minimum energy between specified end-point marginal distributions remains unsolved. Herein, we consider the special case for a non-Markovian process y(t) which, however, assumes…
We propose a new approach for estimating the finite dimensional transition matrix of a Markov chain using a large number of independent sample paths observed at random times. The sample paths may be observed as few as two times, and the…
Motivated by wide-ranging applications such as video delivery over networks using Multiple Description Codes, congestion control, and inventory management, we study the state-tracking of a Markovian random process with a known transition…
We address a class of Markov jump linear systems that are characterized by the underlying Markov process being time-inhomogeneous with a priori unknown transition probabilities. Necessary and sufficient conditions for uniform stochastic…
In this work, we characterise the statistics of Markov chains by constructing an associated sequence of periodic differential operators. Studying the density of states of these operators reveals the absolutely continuous invariant measure…
A discrete rate theory for general multi-ion channels is presented, in which the continuous dynamics of ion diffusion is reduced to transitions between Markovian discrete states. In an open channel, the ion permeation process involves three…
Consider a continuous time Markov chain with rates Q in the state space \Lambda\cup\{0\} with 0 as an absorbing state. In the associated Fleming-Viot process N particles evolve independently in \Lambda with rates Q until one of them…
We present and explore a general method for deriving a Lie-Markov model from a finite semigroup. If the degree of the semigroup is $k$, the resulting model is a continuous-time Markov chain on $k$ states and, as a consequence of the product…
To describe and analyze the dynamics of Self-Organized Criticality (SOC) systems, a four-state continuous-time Markov model is proposed in this paper. Different to computer simulation or numeric experimental approaches commonly employed for…
We consider the almost semi-continuous processes defined on a finite Markov chain. The representation of the moment generating functions for the absolute maximum after achievement positive level and for the recovery time are obtained.…
For an infinite system of particles arriving in and departing from a habitat $X$ -- a locally compact Polish space with a positive Radon measure $\chi$ -- a Markov process is constructed in an explicit way. Along with its location $x\in X$,…
We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion…
Generators of Markov processes on a countable state space can be represented as finite or infinite matrices. One key property is that the off-diagonal entries corresponding to jump rates of the Markov process are non-negative. Here we…
Optimal sensor scheduling with applications to networked estimation and control systems is considered. We model sensor measurement and transmission instances using jumps between states of a continuous-time Markov chain. We introduce a cost…
A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…