Related papers: The elapsed time between two transient state obser…

When is keeping a memory of observations worthwhile? We use hidden Markov models to look at phase transitions that emerge when comparing state estimates in systems with discrete states and noisy observations. We infer the underlying state…

Markov state modeling has gained popularity in various scientific fields since it reduces complex time-series data sets into transitions between a few states. Yet common Markov state modeling frameworks assume a single Markov chain…

We consider a discrete-time Markov chain, called fragmentation process, that describes a specific way of successively removing objects from a linear arrangement. The process arises in population genetics and describes the ancestry of the…

For an infinite system of particles arriving in and departing from a habitat $X$ -- a locally compact Polish space with a positive Radon measure $\chi$ -- a Markov process is constructed in an explicit way. Along with its location $x\in X$,…

Interpreting partial information collected from systems subject to noise is a key problem across scientific disciplines. Theoretical frameworks often focus on the dynamics of variables that result from coarse-graining the internal states of…

Status update systems require the timely collection of sensing information for which deploying multiple sensors/servers to obtain diversity gains is considered as a promising solution. In this work, we construct an absorbing Markov chain…

This paper deals with the state estimation of linear time-invariant systems using distributed observers with local sampled-data measurement and aperiodic communication. Each observer agent perceives partial information of the system to be…

Aging is a universal consequence of life, yet researchers have identified no universal theme. This manuscript considers aging from the perspective of entropy, wherein things fall apart. We first examine biological information change as a…

We consider the modeling of data generated by a latent continuous-time Markov jump process with a state space of finite but unknown dimensions. Typically in such models, the number of states has to be pre-specified, and Bayesian inference…

In this work, we consider a finite-state inhomogeneous-time Markov chain whose probabilities of transition from one state to another tend to decrease over time. This can be seen as a cooling of the dynamics of an underlying Markov chain. We…

We study properties and parameter estimation of finite-state homogeneous continuous-time bivariate Markov chains. Only one of the two processes of the bivariate Markov chain is observable. The general form of the bivariate Markov chain…

Whereas the importance of transient dynamics to the functionality and management of complex systems has been increasingly recognized, most of the studies are based on models. Yet in realistic situations the models are often unknown and what…

A discrete-time Markov chain can be transformed into a new Markov chain by looking at its states along iterations of an almost surely finite stopping time. By the optional stopping theorem, any bounded harmonic function with respect to the…

Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…

There is a growing interest in methods for detecting and interpreting changes in experimental time evolution data. Based on measured time series, the quantitative characterization of dynamical phase transitions at bifurcation points of the…

This paper gives a method for computing distributions associated with patterns in the state sequence of a hidden Markov model, conditional on observing all or part of the observation sequence. Probabilities are computed for very general…

We obtain universal estimates on the convergence to equilibrium and the times of coupling for continuous time irreducible reversible finite-state Markov chains, both in the total variation and in the L^2 norms. The estimates in total…

We consider a finite state discrete time process X. Without loss of generality the finite state space can be identified with the set of unit vectors {e1, e2, . . . , eN} with ei = (0, . . . , 0, 1, 0, . . . , 0)0 2 RN. For a Markov chain…

This project is going to work with one example of stochastic matrix to understand how Markov chains evolve and how to use them to make faster and better decisions only looking to the present state of the system.

We investigate recurrence and transience of Branching Markov Chains (BMC) in discrete time. Branching Markov Chains are clouds of particles which move (according to an irreducible underlying Markov Chain) and produce offspring…