Related papers: A partition method for bounding continuous-time Ma…
Dynamical systems with large state-spaces are often expensive to thoroughly explore experimentally. Coarse-graining methods aim to define simpler systems which are more amenable to analysis and exploration; most current methods, however,…
To understand the long-run behavior of Markov population models, the computation of the stationary distribution is often a crucial part. We propose a truncation-based approximation that employs a state-space lumping scheme, aggregating…
We consider state-aggregation schemes for Markov chains from an information-theoretic perspective. Specifically, we consider aggregating the states of a Markov chain such that the mutual information of the aggregated states separated by T…
The asymptotic properties of some Markov processes associated to stochastic chemical reaction networks (CRNs) driven by the kinetics of the law of mass action are analyzed. The scaling regime introduced in the paper assumes that the norm of…
We address the problem of Lyapunov function construction for a class of continuous-time Markov chains with affine transition rates, typically encountered in stochastic chemical kinetics. Following an optimization approach, we take advantage…
Stochastic modeling of chemical reaction systems based on master equations has been an indispensable tool in physical sciences. In the long-time limit, the properties of these systems are characterized by stationary distributions of…
We construct a new Markov chain Monte Carlo method on finite states with optimal choices of acceptance-rejection ratio functions. We prove that the constructed continuous time Markov jumping process has a global in-time convergence rate in…
Probabilistic model checking for systems with large or unbounded state space is a challenging computational problem in formal modelling and its applications. Numerical algorithms require an explicit representation of the state space, while…
Analyses of serially-sampled data often begin with the assumption that the observations represent discrete samples from a latent continuous-time stochastic process. The continuous-time Markov chain (CTMC) is one such generative model whose…
After collecting data from observations or experiments, the next step is to build an appropriate mathematical or stochastic model to describe the data so that further studies can be done with the help of the models. In this article, the…
Product-form stationary distributions in Markov chains have been a foundational advance and driving force in our understanding of stochastic systems. In this paper, we introduce a new product-form relationship that we call "graph-based…
Throughout physics Brownian dynamics are used to describe the behaviour of molecular systems. When the Brownian particle is confined to a bounded domain, a particularly important question arises around determining how long it takes the…
The paper deals with a certain class of random evolutions. We develop a construction that yields an invariant measure for a continuous-time Markov process with random transitions. The approach is based on a particular way of constructing…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
We explore a Markov model used in the analysis of gene expression, involving the bursty production of pre-mRNA, its conversion to mature mRNA, and its consequent degradation. We demonstrate that the integration used to compute the solution…
We consider discrete-space continuous-time Markov models of reaction networks and provide sufficient conditions for the following stability condition to hold: each state in a closed, irreducible component of the state space is positive…
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…
We first derive the Hamilton-Jacobi theory underlying continuous-time Markov processes, and then use the construction to develop a variational algorithm for estimating escape (least improbable or first passage) paths for a generic…
We consider the problem of controlling a Markov decision process (MDP) with a large state space, so as to minimize average cost. Since it is intractable to compete with the optimal policy for large scale problems, we pursue the more modest…
The asymptotic behavior of a stochastic network represented by a birth and death processes of particles on a compact state space is analyzed. Births: Particles are created at rate $\lambda_+$ and their location is independent of the current…