Related papers: Markov Switching
Scaled type Markov renewal processes generalize classical renewal processes: renewal times come from a one parameter family of probability laws and the sequence of the parameters is the trajectory of an ergodic Markov chain. Our primary…
Motivated by applications in movement ecology, in this paper I propose a new class of integrated continuous-time hidden Markov models in which each observation depends on the underlying state of the process over the whole interval since the…
Datasets involving sequences of different types of events without meaningful time stamps are prevalent in many applications, for instance when extracted from textual corpora. We propose a family of models for such event sequences -- summary…
We propose a novel method to directly learn a stochastic transition operator whose repeated application provides generated samples. Traditional undirected graphical models approach this problem indirectly by learning a Markov chain model…
Phase-type distribution has been an important probabilistic tool in the analysis of complex stochastic system evolution. It was introduced by Neuts \cite{Neuts1975} in 1975. The model describes the lifetime distribution of a finite-state…
The spatial autoregressive (SAR) model is extended by introducing a Markov switching dynamics for the weight matrix and spatial autoregressive parameter. The framework enables the identification of regime-specific connectivity patterns and…
This paper deals with optimal prediction in a regime-switching model driven by a continuous-time Markov chain. We extend existing results for geometric Brownian motion by deriving optimal stopping strategies that depend on the current…
We consider a node-monitor pair, where the node's state varies with time. The monitor needs to track the node's state at all times; however, there is a fixed cost for each state query. So the monitor may instead predict the state using…
We consider the synthesis of control policies for probabilistic systems, modeled by Markov decision processes, operating in partially known environments with temporal logic specifications. The environment is modeled by a set of Markov…
The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…
A decade ago, Abdulla, Ben Henda and Mayr introduced the elegant concept of decisiveness for denumerable Markov chains [1]. Roughly speaking, decisiveness allows one to lift most good properties from finite Markov chains to denumerable…
This paper introduces a novel methodology for the identification of switching dynamics for switched autoregressive linear models. Switching behavior is assumed to follow a Markov model. The system's outputs are contaminated by possibly…
We study the performance of a stochastic algorithm based on the power method that adaptively learns the large deviation functions characterizing the fluctuations of additive functionals of Markov processes, used in physics to model…
In this letter we announce rigorous results that elucidate the relation between metastable states and low-lying eigenvalues in Markov chains in a much more general setting and with considerable greater precision as was so far available.…
The deterministic dynamics of randomly connected neural networks are studied, where a state of binary neurons evolves according to a discreet-time synchronous update rule. We give a theoretical support that the overlap of systems' states…
Immune events such as infection, vaccination, and a combination of the two result in distinct time-dependent antibody responses in affected individuals. These responses and event prevalences combine non-trivially to govern antibody levels…
We consider the fluctuations of a time-integrated particle current around an atypical value in a generic stochastic Markov process involving classical particles with two-site interaction and hardcore repulsion on a finite one-dimensional…
Random walks are a fundamental model in applied mathematics and are a common example of a Markov chain. The limiting stationary distribution of the Markov chain represents the fraction of the time spent in each state during the stochastic…
Markov processes are shown to be consistent with metastable states seen in pulsar phenomena, including intensity nulling, pulse-shape mode changes, subpulse drift rates, spindown rates, and X-ray emission, based on the typically broad and…
Biological systems need to react to stimuli over a broad spectrum of timescales. If and how this ability can emerge without external fine-tuning is a puzzle. We consider here this problem in discrete Markovian systems, where we can leverage…