Related papers: Almost sure convergence of numerical approximation…
We review some recent results of quantitative long-time convergence for the law of a killed Markov process conditioned to survival toward a quasi-stationary distribution, and on the analogous question for the particle systems used in…
We study a Q learning algorithm for continuous time stochastic control problems. The proposed algorithm uses the sampled state process by discretizing the state and control action spaces under piece-wise constant control processes. We show…
We develop algorithms for the numerical computation of the quadratic hedging strategy in incomplete markets modeled by pure jump Markov process. Using the Hamilton-Jacobi-Bellman approach, the value function of the quadratic hedging problem…
We develop an algorithm for computing bounded reachability probability for hybrid systems, i.e., the probability that the system reaches an unsafe region within a finite number of discrete transitions. In particular, we focus on hybrid…
This article presents a short and concise description of stochastic approximation algorithms in reinforcement learning of Markov decision processes. The algorithms can also be used as a suboptimal method for partially observed Markov…
Biochemical reactions can happen on different time scales and also the abundance of species in these reactions can be very different from each other. Classical approaches, such as deterministic or stochastic approach, fail to account for or…
The molecular evolution in a gene regulatory network is classically modeled by Markov jump processes. However, the direct simulation of such models is extremely time consuming. Indeed, even the simplest Markovian model, such as the…
We study a class of Piecewise Deterministic Markov Processes with state space Rd x E where E is a finite set. The continuous component evolves according to a smooth vector field that is switched at the jump times of the discrete coordinate.…
This paper presents a rigorous mathematical analysis, alongside simulation studies, of a spatially extended stochastic electrophysiology model, the Hodgkin-Huxley model of the squid giant axon being a classical example. Although most…
Gradient-based algorithms are one of the methods of choice for the optimisation of Markov Decision Processes. In this article we will present a novel approximate Newton algorithm for the optimisation of such models. The algorithm has…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
In the continuity of a recent paper ([6]), dealing with finite Markov chains, this paper proposes and analyzes a recursive algorithm for the approximation of the quasi-stationary distribution of a general Markov chain living on a compact…
In the paper we consider some piecewise deterministic Markov process whose continuous component evolves according to semiflows, which are switched at the jump times of a Poisson process. The associated Markov chain describes the states of…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
The probability distribution describing the state of a Stochastic Reaction Network evolves according to the Chemical Master Equation (CME). It is common to estimated its solution using Monte Carlo methods such as the Stochastic Simulation…
In this paper, we compute finite sample bounds for data-driven approximations of the solution to stochastic reachability problems. Our approach uses a nonparametric technique known as kernel distribution embeddings, and provides…
In piecewise-deterministic Markov processes (PDMPs) the state of a finite-dimensional system evolves continuously, but the evolutive equation may change randomly as a result of discrete switches. A running cost is integrated along the…
Recently there have been exciting developments in Monte Carlo methods, with the development of new MCMC and sequential Monte Carlo (SMC) algorithms which are based on continuous-time, rather than discrete-time, Markov processes. This has…
Stochastic approximation algorithm is a useful technique which has been exploited successfully in probability theory and statistics for a long time. The step sizes used in stochastic approximation are generally taken to be deterministic and…
Consider a probability measure supported by a regular geodesic ball in a manifold. For any p larger than or equal to 1 we define a stochastic algorithm which converges almost surely to the p-mean of the measure. Assuming furthermore that…