Related papers: Realizable monotonicity for continuous-time Markov…
Comparison results are given for time-inhomogeneous Markov processes with respect to function classes induced stochastic orderings. The main result states comparison of two processes, provided that the comparability of their infinitesimal…
The invariant measure is a fundamental object in the theory of Markov processes. In finite dimensions a Markov process is defined by transition rates of the corresponding stochastic matrix. The Markov tree theorem provides an explicit…
We introduce an statistical mechanical formalism for the study of discrete-time stochastic processes with which we prove: (i) General properties of extremal chains, including triviality on the tail $\sigma$-algebra, short-range…
We study a variable length Markov chain model associated with a group of stationary processes that share the same context tree but each process has potentially different conditional probabilities. We propose a new model selection and…
This article presents several results establishing connections be- tween Markov chains and dynamical systems, from the point of view of open systems in physics. We show how all Markov chains can be understood as the information on one…
A constructive proof is given to the fact that any ergodic Markov chain can be realized as a random walk subject to a synchronizing road coloring. Redundancy (ratio of extra entropy) in such a realization is also studied.
We give the first algorithm for testing the feasibility of a system of sporadic real-time tasks on a set of identical processors, solving one major open problem in the area of multiprocessor real-time scheduling [S.K. Baruah and K. Pruhs,…
This paper compiles several aspects of the dynamics of stochastic approximation algorithms with Markov iterate-dependent noise when the iterates are not known to be stable beforehand. We achieve the same by extending the lock-in probability…
In this short paper, we consider discrete-time Markov chains on lattices as approximations to continuous-time diffusion processes. The approximations can be interpreted as finite difference schemes for the generator of the process. We…
We study discrete time Markov processes with periodic or open boundary conditions and with inhomogeneous rates in the bulk. The Markov matrices are given by the inhomogeneous transfer matrices introduced previously to prove the…
We introduce a simple approach for testing the reliability of homogeneous generators and the Markov property of the stochastic processes underlying empirical time series of credit ratings. We analyze open access data provided by Moody's and…
We consider continuous-time Markov chain on a finite state space X. We assume X can be clustered into several subsets such that the intra-transition rates within these subsets are of order $\mathcal{O}(\frac{1}{\epsilon})$ comparing to the…
We study the phenomenon of entrainment in processor sharing networks, whereby, while individual network resources have sufficient capacity to met demand, the requirement for simultaneous availability of resources means that a network may…
We study the limiting object of a sequence of Markov chains analogous to the limits of graphs, hypergraphs, and other objects which have been studied. Following a suggestion of Aldous, we assign to a sequence of finite Markov chains with…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
We consider killed Markov decision processes for countable models on a finite time-interval. Existence of a uniform $\varepsilon$-optimal policy is proven. We show the correctness of the fundamental equation. The optimal control problem is…
Markov models are often used to capture the temporal patterns of sequential data for statistical learning applications. While the Hidden Markov modeling-based learning mechanisms are well studied in literature, we analyze a…
We introduce a new class of stochastic processes which are stationary, Markovian and characterized by an infinite range of time-scales. By transforming the Fokker-Planck equation of the process into a Schrodinger equation with an…
We consider the problem of determining the existence of a sequence of matrices driving a discrete-time consensus system to consensus. We transform this problem into one of the existence of a product of the transition (stochastic) matrices…
We study stochastic choice across decision problems, each represented as a menu of action labels paired with observable outcome vectors. We propose a consistency condition for behavior in decision problems composed of two separable…