Related papers: Bounds for the expected value of one-step processe…
We study the mean-field limit of a generic class of dynamic co-evolving latent space networks motivated by the social and opinion dynamics literature. Such models include $n$ agents, whose opinions are given by latent stochastic processes,…
The mean-field analysis of a multi-population agent-based model is performed. The model couples a particle dynamics driven by a nonlocal velocity with a Markow-type jump process on the probability that each agent has of belonging to a given…
We consider finite model approximations of discrete-time partially observed Markov decision processes (POMDPs) under the discounted cost criterion. After converting the original partially observed stochastic control problem to a fully…
The purpose of this paper is to study a Markovian metapopulation model on a directed graph with edge-supported transfers and deterministic intra-nodal population dynamics. We first state tractable stability conditions for two typical…
Density dependent Markov population processes in large populations of size $N$ were shown by Kurtz (1970, 1971) to be well approximated over finite time intervals by the solution of the differential equations that describe their average…
We consider a stochastic spatial point process with births and deaths on $\mathbb{R}^d$, with the hard-core property that at any time the balls of radius half of any two points do not overlap. We give explicit construction of the process.…
We consider the problem of computing minimum and maximum probabilities of satisfying an $\omega$-regular property in a bounded-parameter Markov decision process (BMDP). BMDP arise from Markov decision processes (MDP) by allowing for…
Epidemics are inherently stochastic, and stochastic models provide an appropriate way to describe and analyse such phenomena. Given temporal incidence data consisting of, for example, the number of new infections or removals in a given time…
Non-equilibrium steady states (NESS) of Markov processes give rise to non-trivial cyclic probability fluxes. Cycle decompositions of the steady state offer an effective description of such fluxes. Here, we present an iterative cycle…
A dynamic mean field theory is developed for finite state and action Bayesian reinforcement learning in the large state space limit. In an analogy with statistical physics, the Bellman equation is studied as a disordered dynamical system;…
Gradient matching with Gaussian processes is a promising tool for learning parameters of ordinary differential equations (ODE's). The essence of gradient matching is to model the prior over state variables as a Gaussian process which…
We formulate a class of mean field games on a finite state space with variational principles resembling those in continuous-state mean field games. We construct a controlled continuity equation featuring a nonlinear activation function on…
Markov processes with stochastic resetting towards the origin generically converge towards non-equilibrium steady-states. Long dynamical trajectories can be thus analyzed via the large deviations at Level 2.5 for the joint probability of…
We consider the linear programming approach for constrained and unconstrained Markov decision processes (MDPs) under the long-run average cost criterion, where the class of MDPs in our study have Borel state spaces and discrete countable…
A definition of metastable states applicable to arbitrary finite state Markov processes satisfying detailed balance is discussed. In particular, we identify a crucial condition that distinguishes genuine metastable states from other types…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
Markov jump process models have many applications across science. Often, these models are defined on a state-space of product form and only one of the components of the process is of direct interest. In this paper, we extend the marginal…
For algorithms based on interacting particle systems that admit a mean-field description, convergence analysis is often more accessible at the mean-field level. In order to transfer convergence results obtained at the mean-field level to…
We consider general Markov processes with absorption and provide criteria ensuring the exponential convergence in total variation of the distribution of the process conditioned not to be absorbed. The first one is based on two-sided…
The transition between low and high density phases is a typical feature of systems with social interactions. This contribution focuses on simple evacuation design of one room with one entrance and one exit; four passing-through experiments…