Related papers: T. E. Harris's contributions to recurrent Markov p…

In this paper we study various properties of finite stochastic systems or hidden Markov chains as they are alternatively called. We discuss their construction following different approaches and we also derive recursive filtering formulas…

In this invited contribution, we revisit the stochastic shortest path problem, and show how recent results allow one to improve over the classical solutions: we present algorithms to synthesize strategies with multiple guarantees on the…

These notes were used in a short graduate course on branching processes the author gave in Beijing Normal University. The following main topics are covered: scaling limits of Galton--Watson processes, continuous-state branching processes,…

Recent work in dynamical systems theory has shown that many properties that are associated with irreversible processes in fluids can be understood in terms of the dynamical properties of reversible, Hamiltonian systems. That is,…

An overview is given of recent advances in nonequilibrium statistical mechanics about the statistics of random paths and current fluctuations. Although statistics is carried out in space for equilibrium statistical mechanics, statistics is…

This paper generalizes the notion of stochastic order to a relation between probability measures over arbitrary measurable spaces. This generalization is motivated by the observation that for the stochastic ordering of two stationary Markov…

This project is going to work with one example of stochastic matrix to understand how Markov chains evolve and how to use them to make faster and better decisions only looking to the present state of the system.

In this paper, we investigate a special class of stochastic Markov processes, known as Glauber dynamics. Markov processes are importance, for example, in the study of complex systems. For this, we present the basic theory of Glauber…

We develop a new methodology for the fluctuation theory of continuous-time skip-free Markov chains, extending the recent work of Choi and Patie [5] for discrete-time skip-free Markov chains. As the main application we use it to derive a…

This paper investigates stochastic finite matrices and the corresponding finite Markov chains constructed using recurrence matrices for general families of orthogonal polynomials and multiple orthogonal polynomials. The paper explores the…

Many regenerative arguments in stochastic processes use random times which are akin to stopping times, but which are determined by the future as well as the past behaviour of the process of interest. Such arguments based on "conditioning on…

The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive n-th order…

We consider linear hyperbolic balance law that describe gas flow. Stochastic influences are introduced by series of orthogonal functions. A deterministic stabilization concept, which makes deviations at steady states decay exponentially…

We study continuous time Markov processes on graphs. The notion of frequency is introduced, which serves well as a scaling factor between any Markov time of a continuous time Markov process and that of its jump chain. As an application, we…

We propose an extension to Hawkes processes by treating the levels of self-excitation as a stochastic differential equation. Our new point process allows better approximation in application domains where events and intensities accelerate…

Markov processes are popular mathematical models, studied by theoreticians for their intriguing properties, and applied by practitioners for their flexible structure. With this book we teach how to model and analyze Markov processes. We…

Dynamics of information flow in adaptively interacting stochastic processes is studied. We give an extended form of game dynamics for Markovian processes and study its behavior to observe information flow through the system. Examples of the…

Markov matrices have an important role in the filed of stochastic processes. In this paper, we will show and prove a series of conclusions on Markov matrices and transformations rather than pay attention to stochastic processes although…

Since the introduction of Dyson's Brownian motion in early 1960's, there have been a lot of developments in the investigation of stochastic processes on the space of Hermitian matrices. Their properties, especially, the properties of their…

In the paper we consider some piecewise deterministic Markov process whose continuous component evolves according to semiflows, which are switched at the jump times of a Poisson process. The associated Markov chain describes the states of…