Related papers: Markov loops in discrete spaces

We axiomatize and generalize Markov's approach to the continuity problem for Type 1 computable functions, i.e. the problem of finding sufficient conditions on a computable topological space to obtain a theorem of the form "computable…

Markov decision processes (MDPs) in queues and networks have been an interesting topic in many practical areas since the 1960s. This paper provides a detailed overview on this topic and tracks the evolution of many basic results. Also, this…

In this paper we study the spectral properties of Markov-operator on $L^{2}$-spaces. Lawler and Sokal (Trans. Amer. Math. Soc., 1988, 309, pp. 557-580) used isoperimetric constants for discrete and continuous time Markov chains to obtain a…

This note studies monotone Markov chains, a subclass of Markov chains with extensive applications in operations research and economics. While the properties that ensure the global stability of these chains are well studied, their…

A non-symplectic generalization of Hamiltonian mechanics is considered. It allows include into consideration "non-Lagrange" systems, such as theory of charged particle in the field of magnetic monopole. The corresponding generalization for…

In this note we connect the notion of solutions of a martingale problem to the notion of a strongly continuous and locally equi-continuous semigroup on the space of bounded continuous functions equipped with the strict topology. This…

Markov jump processes (or continuous-time Markov chains) are a simple and important class of continuous-time dynamical systems. In this paper, we tackle the problem of simulating from the posterior distribution over paths in these models,…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

A function on the state space of a Markov chain is a "lumping" if observing only the function values gives a Markov chain. We give very general conditions for lumpings of a large class of algebraically-defined Markov chains, which include…

Random walks find applications in many areas of science and are the heart of essential network analytic tools. When defined on temporal networks, even basic random walk models may exhibit a rich spectrum of behaviours, due to the…

In this paper, we study one dimensional Markov processes with spatial delay. Since the seminal work of Feller, we know that virtually any one dimensional, strong, homogeneous, continuous Markov process can be uniquely characterized via its…

Using two simple examples, the continuous-time random walk as well as a two state Markov chain, the relation between generalized anomalous relaxation equations and semi-Markov processes is illustrated. This relation is then used to discuss…

We abstract and generalize homotopical monadicity statements, placing in a single conceptual framework a range of old and recent recognition and characterization principles in iterated loop space theory in classical, equivariant, and…

This survey is based on a series of lectures that we gave at MSRI in Spring 2015 and on a series of papers, mostly written jointly with Joan Porti. Our goal here is to: 1. Describe a class of discrete subgroups $\Gamma<G$ of higher rank…

A novel framework for the analysis of observation statistics on time discrete linear evolutions in Banach space is presented. The model differs from traditional models for stochastic processes and, in particular, clearly distinguishes…

We show that any loop-free Markov chain on a discrete space can be viewed as a determinantal point process. As an application, we prove central limit theorems for the number of particles in a window for renewal processes and Markov renewal…

We review a recent development at the interface between discrete mathematics on one hand and probability theory and statistics on the other, specifically the use of Markov chains and their boundary theory in connection with the asymptotics…

Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…

We show that the stationary distribution of a finite Markov chain can be expressed as the sum of certain normal distributions. These normal distributions are associated to planar graphs consisting of a straight line with attached loops. The…

In this paper we discuss four problems regarding Markov equivalences for subclasses of loopless mixed graphs. We classify these four problems as finding conditions for internal Markov equivalence, which is Markov equivalence within a…