Related papers: Marginal process framework: A model reduction tool…
We consider Piecewise Deterministic Markov Processes (PDMPs) with a finite set of discrete states. In the regime of fast jumps between discrete states, we prove a law of large number and a large deviation principle. In the regime of fast…
Stochastic processes abound in nature and accurately modeling them is essential across the quantitative sciences. They can be described by hidden Markov models (HMMs) or by their quantum extensions (QHMMs). These models explain and give…
In many operations management problems, we need to make decisions sequentially to minimize the cost while satisfying certain constraints. One modeling approach to study such problems is constrained Markov decision process (CMDP). When…
Branching processes are a class of continuous-time Markov chains (CTMCs) with ubiquitous applications. A general difficulty in statistical inference under partially observed CTMC models arises in computing transition probabilities when the…
This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of…
This paper introduces a mathematical framework of a stochastic process model as a generalization of diffusion stochastic processes to model latent variables in categorical responses given unobserved random effects and maximum likelihood…
We study approximations of evolving probability measures by an interacting particle system. The particle system dynamics is a combination of independent Markov chain moves and importance sampling/resampling steps. Under global regularity…
Revision of the paper previously entitled "Learning a Machine for the Decision in a Partially Observable Markov Universe" In this paper, we are interested in optimal decisions in a partially observable universe. Our approach is to directly…
In this paper, we consider the finite-state approximation of a discrete-time constrained Markov decision process (MDP) under the discounted and average cost criteria. Using the linear programming formulation of the constrained discounted…
Consider a system of interacting particles indexed by the nodes of a graph whose vertices are equipped with marks representing parameters of the model such as the environment or initial data. Each particle takes values in a countable state…
We use Markov categories to generalize the basic theory of Markov chains and hidden Markov models to an abstract setting. This comprises characterizations of hidden Markov models in terms of conditional independences and algorithms for…
Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition…
We consider chemical reaction networks modeled by a discrete state and continuous in time Markov process for the vector copy number of the species and provide a novel particle filter method for state and parameter estimation based on exact…
We introduce a class of Markov processes conditioned to avoid intersection over a moving time window of length T>0, a setting we refer to as myopic non-intersection. In particular, we study a system of myopic non-intersecting Brownian…
Inference for Dirichlet process hierarchical models is typically performed using Markov chain Monte Carlo methods, which can be roughly categorised into marginal and conditional methods. The former integrate out analytically the…
We present a novel multiscale simulation approach for modeling stochasticity in chemical reaction networks. The approach seamlessly integrates exact-stochastic and "leaping" methodologies into a single "partitioned leaping" algorithmic…
We investigate the Poisson regression method for Markov and semi-Markov jump processes from a nonparametric angle, allowing the lengths of the time and duration intervals in the partition to vary with the number of observations. Imposing no…
This paper describes a data reduction technique in case of a markov chain of specified order. Instead of observing all the transitions in a markov chain we record only a few of them and treat the remaining part as missing. The decision…
The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…
Motivated by entropic optimal transport, time reversal of Markov jump processes in $\mathbb{R}^n$ is investigated. Relying on an abstract integration by parts formula for the carr\'e du champ of a Markov process recently obtained by…