Related papers: Estimating Parameters of Large CTMP from Single Tr…
This paper contributes an in-depth study of properties of continuous time Markov chains (CTMCs) on non-negative integer lattices $\N_0^d$, with particular interest in one-dimensional CTMCs with polynomial transitions rates. Such stochastic…
We consider the task of learning a parametric Continuous Time Markov Chain (CTMC) sequence model without examples of sequences, where the training data consists entirely of aggregate steady-state statistics. Making the problem harder, we…
Analyses of serially-sampled data often begin with the assumption that the observations represent discrete samples from a latent continuous-time stochastic process. The continuous-time Markov chain (CTMC) is one such generative model whose…
This paper provides full classification of dynamics for continuous time Markov chains (CTMCs) on the non-negative integers with polynomial transition rate functions. Such stochastic processes are abundant in applications, in particular in…
We are interested in the analysis of very large continuous-time Markov chains (CTMCs) with many distinct rates. Such models arise naturally in the context of reliability analysis, e.g., of computer network performability analysis, of power…
Labeled continuous-time Markov chains (CTMCs) describe processes subject to random timing and partial observability. In applications such as runtime monitoring, we must incorporate past observations. The timing of these observations matters…
More than ever, today we are left with the abundance of molecular data outpaced by the advancements of the phylogenomic methods. Especially in the case of presence of many genes over a set of species under the phylogeny question, more…
Epidemic processes are common out-of-equilibrium phenomena of broad interdisciplinary interest. Recently, dynamic message-passing (DMP) has been proposed as an efficient algorithm for simulating epidemic models on networks, and in…
Finding the correct encoding for a generic dynamical system's trajectory is a complicated task: the symbolic sequence needs to preserve the invariant properties from the system's trajectory. In theory, the solution to this problem is found…
Transition Path Theory (TPT) provides a rigorous statistical characterization of the ensemble of trajectories connecting directly, i.e., without detours, two disconnected (sets of) states in a Markov chain, a stochastic process that…
This paper investigates the effect of the structure of the contact network on the dynamics of the epidemic outbreak. In particular, we focus on the peak number of critically infected nodes (PCIN), determining the maximum workload of…
We study continuous time Markov processes on graphs. The notion of frequency is introduced, which serves well as a scaling factor between any Markov time of a continuous time Markov process and that of its jump chain. As an application, we…
Many epidemic processes in networks spread by stochastic contacts among their connected vertices. There are two limiting cases widely analyzed in the physics literature, the so-called contact process (CP) where the contagion is expanded at…
Piecewise-deterministic Markov processes (PDMPs) offer a powerful stochastic modeling framework that combines deterministic trajectories with random perturbations at random times. Estimating their local characteristics (particularly the…
This study introduces a comparative modeling framework using stationary and non-stationary transition probabilities within a Markov Decision Process (MDP) to assess COVID-19 disease dynamics. Stationary transition probabilities assume…
We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the…
Markov decision process over vector addition system with states (VASS MDP) is a finite state model combining non-deterministic and probabilistic behavior, augmented with non-negative integer counters that can be incremented or decremented…
Continuous Time Markov Chain (CMTC) is widely used to describe and analyze systems in several knowledge areas. Steady state availability is one important analysis that can be made through Markov chain formalism that allows researchers…
We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…
We study a coupled epidemic-mobility model in which, at each time, individuals move in a network of spatially-distributed regions (sub-populations) according to a Continuous Time Markov Chain (CTMC) and subsequently interact with the local…