Related papers: Explicit estimates in the Bramson-Kalikow model
A new approach for optimal estimation of Markov chains with sparse transition matrices is presented.
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…
In this article we consider Bayesian estimation of static parameters for a class of partially observed McKean-Vlasov diffusion processes with discrete-time observations over a fixed time interval. This problem features several obstacles to…
Mathematical modelling has become an established tool for studying the dynamics of biological systems. Current applications range from building models that reproduce quantitative data to identifying systems with predefined qualitative…
Direct simulation of biomolecular dynamics in thermal equilibrium is challenging due to the metastable nature of conformation dynamics and the computational cost of molecular dynamics. Biased or enhanced sampling methods may improve the…
Markov chains provide a foundational framework for modeling sequential stochastic processes, with the transition probability matrix characterizing the dynamics of state evolution. While classical estimation methods such as maximum…
We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…
Models for open quantum systems, which play important roles in electron transport problems and quantum computing, must take into account the interaction of the quantum system with the surrounding environment. Although such models can be…
General Markov chains in an arbitrary phase space are considered in the framework of the operator treatment. Markov operators continue from the space of countably additive measures to the space of finitely additive measures. Cycles of…
We consider the problem of computing the satisfaction probability of a formula for stochastic models with parametric uncertainty. We show that this satisfaction probability is a smooth function of the model parameters. This enables us to…
We introduce a new tool for the quantitative characterisation of the departure form Markovianity of a given dynamical process. Our tool can be applied to a generic $N$-level system and extended straightforwardly to Gaussian…
We consider killed Markov decision processes for countable models on a finite time-interval. Existence of a uniform $\varepsilon$-optimal policy is proven. We show the correctness of the fundamental equation. The optimal control problem is…
We consider the problem of estimating the transition rate matrix of a continuous-time Markov chain from a finite-duration realisation of this process. We approach this problem in an imprecise probabilistic framework, using a set of prior…
Motivated by applications in Markov chain Monte Carlo, we discuss what it means for one Markov chain to be an approximation to another. Specifically included in that discussion are situations in which a Markov chain with continuous state…
Continuous-time Markov processes over finite state-spaces are widely used to model dynamical processes in many fields of natural and social science. Here, we introduce an maximum likelihood estimator for constructing such models from data…
Cross-sectional observations from a dynamical system can be modeled via steady-state distributions of Markov processes. The major challenge is then to determine whether the process parameters can be identified and estimated from the…
Consider a one-sided Markov additive process with an upper and a lower barrier, where each can be either reflecting or terminating. For both defective and non-defective processes and all possible scenarios we identify the corresponding…
The two-parameter Macdonald polynomials are a central object of algebraic combinatorics and representation theory. We give a Markov chain on partitions of k with eigenfunctions the coefficients of the Macdonald polynomials when expanded in…
We introduce $(\varepsilon, \delta)$-bisimulation, a novel type of approximate probabilistic bisimulation for continuous-time Markov chains. In contrast to related notions, $(\varepsilon, \delta)$-bisimulation allows the use of different…
We propose a distance between continuous-time Markov chains (CTMCs) and study the problem of computing it by comparing three different algorithmic methodologies: iterative, linear program, and on-the-fly. In a work presented at FoSSaCS'12,…