Related papers: Flowgraph Models and Analysis for Markov Jump Proc…
We study the performance of a stochastic algorithm based on the power method that adaptively learns the large deviation functions characterizing the fluctuations of additive functionals of Markov processes, used in physics to model…
Many biological and medical questions can be modeled using time-to-event data in finite-state Markov chains, with the phase-type distribution describing intervals between events. We solve the inverse problem: given a phase-type…
In this work, we present a general method to establish properties of multi-dimensional continuous-time Markov chains representing stochastic reaction networks. This method consists of grouping states together (via a partition of the state…
A study of time homogeneous, real valued Markov processes with a special property and a non-atomic initial distribution is provided. The new notion of a function of evolution of distribution which determines the dependency between one…
Markov switching models are a popular family of models that introduces time-variation in the parameters in the form of their state- or regime-specific values. Importantly, this time-variation is governed by a discrete-valued latent…
This paper discusses tractable development and statistical estimation of a continuous time stochastic process with a finite state space having non-Markov property. The process is formed by a finite mixture of right-continuous Markov jump…
We consider Markov jump processes on a graph described by a rate matrix that depends on various control parameters. We derive explicit expressions for the static responses of edge currents and steady-state probabilities. We show that they…
Machine learning techniques not only offer efficient tools for modelling dynamical systems from data, but can also be employed as frontline investigative instruments for the underlying physics. Nontrivial information about the original…
Markov-modulated fluids have a long history. They form a simple class of Markov additive processes, and were initially developed in the 1950s as models for dams and reservoirs, before gaining much popularity in the 1980s as models for…
Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…
To ensure the effective and objective development of transportation networks, it is crucial to identify performance limitations across various subsystems. A timetable-independent assessment of infrastructure capacity at railway junctions is…
These lecture notes introduce the statistical analysis of continuous-time generative models built from Markov dynamics. We begin with the stochastic-calculus foundations of score-based diffusion models, including time reversal, score…
Stochastic processes find applications in modelling systems in a variety of disciplines. A large number of stochastic models considered are Markovian in nature. It is often observed that higher order Markov processes can model the data…
We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…
We present an approach for flux analysis in process algebra models of biological systems. We perceive flux as the flow of resources in stochastic simulations. We resort to an established correspondence between event structures, a broadly…
Markov processes are popular mathematical models, studied by theoreticians for their intriguing properties, and applied by practitioners for their flexible structure. With this book we teach how to model and analyze Markov processes. We…
We propose a new stochastic model for streamflow discharge timeseries as a jump-driven process, called a superposition of continuous-state branching processes with immigration (a supCBI process). It is a non-Markovian model having the…
This project is going to work with one example of stochastic matrix to understand how Markov chains evolve and how to use them to make faster and better decisions only looking to the present state of the system.
The conventional perspective on Markov chains considers decision problems concerning the probabilities of temporal properties being satisfied by traces of visited states. However, consider the following query made of a stochastic system…
Advanced measurement techniques and high performance computing have made large data sets available for a wide range of turbulent flows that arise in engineering applications. Drawing on this abundance of data, dynamical models can be…