Related papers: The theory of individual based discrete-time proce…
The mathematical model of a linear system with the short memory about own stochastic behavior is proposed. It is assumed that the system is under a continual influence of independent stochastic impulses. In a short memory approximation the…
This paper is concerned with a compositional approach for constructing finite Markov decision processes of interconnected discrete-time stochastic control systems. The proposed approach leverages the interconnection topology and a notion of…
We prove a general existence result in stochastic optimal control in discrete time where controls take values in conditional metric spaces, and depend on the current state and the information of past decisions through the evolution of a…
In this paper, a general stochastic model with controls applied at the moments when the random process hits the boundary of a given subset of the state set is proposed and studied. The general concept of the model is formulated and its…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
For controlled discrete-time stochastic processes we introduce a new class of dynamic risk measures, which we call process-based. Their main features are that they measure risk of processes that are functions of the history of a base…
We present an approximate analytical expression for the escape rate of time-dependent driven stochastic processes with an absorbing boundary such as the driven leaky integrate-and-fire model for neural spiking. The novel approximation is…
In the present work we derive a Central Limit Theorem for sequences of Hilbert-valued Piecewise Deterministic Markov process models and their global fluctuations around their deterministic limit identified by the Law of Large Numbers. We…
We consider a stochastic differential equation in a Hilbert space with time-dependent coefficients for which no general existence and uniqueness results are known. We prove, under suitable assumptions, existence and uniqueness of a measure…
These notes give a summary of techniques used in large deviation theory to study the fluctuations of time-additive quantities, called dynamical observables, defined in the context of Langevin-type equations, which model equilibrium and…
We consider a locally regulated spatial population model introduced by Bolker and Pacala. Based on the deterministic approximation studied by Fournier and M\'el\'eard, we prove that the fluctuation theorem holds under some mild moment…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
A recent paper of Melbourne & Stuart, A note on diffusion limits of chaotic skew product flows, Nonlinearity 24 (2011) 1361-1367, gives a rigorous proof of convergence of a fast-slow deterministic system to a stochastic differential…
We present a general method to identify an arbitrary number of fluctuating quantities which satisfy a detailed fluctuation theorem for all times within the framework of time-inhomogeneous Markovian jump processes. In doing so we provide a…
Recent non-asymptotic analyses have substantially advanced the theory of distributional policy evaluation, but they largely concern synchronous full-state updates under a generative model, model-based estimators, accelerated variants, or…
This article present a continuous cascade model of volatility formulated as a stochastic differential equation. Two independent Brownian motions are introduced as random sources triggering the volatility cascade. One multiplicatively…
We consider the fluctuations of a time-integrated particle current around an atypical value in a generic stochastic Markov process involving classical particles with two-site interaction and hardcore repulsion on a finite one-dimensional…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
We study discrete time Markov processes with periodic or open boundary conditions and with inhomogeneous rates in the bulk. The Markov matrices are given by the inhomogeneous transfer matrices introduced previously to prove the…
A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…