Related papers: Self-interacting processes via Doob conditioning
We report on self-induced switchings between multiple distinct space--time patterns in the dynamics of a spatially extended excitable system. These switchings between low-amplitude oscillations, nonlinear waves, and extreme events strongly…
In multi-period stochastic optimization problems, the future optimal decision is a random variable whose distribution depends on the parameters of the optimization problem. We analyze how the expected value of this random variable changes…
This paper presents a new condition for the existence of optimal stationary policies in average-cost continuous-time Markov decision processes with unbounded cost and transition rates, arising from controlled queueing systems. This…
It has been proposed that adaptation in complex systems is optimized at the critical boundary between ordered and disordered dynamical regimes. Here, we review models of evolving dynamical networks that lead to self-organization of network…
For a network of discrete states with a periodically driven Markovian dynamics, we develop an inference scheme for an external observer who has access to some transitions. Based on waiting-time distributions between these transitions, the…
A continuous-time Markov process is proposed to analyze how a group of humans solves a complex task, consisting in the search of the optimal set of decisions on a fitness landscape. Individuals change their opinions driven by two different…
We consider the fluctuations of a time-integrated particle current around an atypical value in a generic stochastic Markov process involving classical particles with two-site interaction and hardcore repulsion on a finite one-dimensional…
In stochastic systems, numerically sampling the relevant trajectories for the estimation of the large deviation statistics of time-extensive observables requires overcoming their exponential (in space and time) scarcity. The optimal way to…
Motivated by the Internet-of-things and sensor networks for cyberphysical systems, the problem of dynamic sensor activation for the tracking of a time-varying process is examined. The tradeoff is between energy efficiency, which decreases…
Motivated by applications to mathematical biology, we study the averaging problem for slow-fast systems, {\em in the case in which the fast dynamics is a stochastic process with multiple invariant measures}. We consider both the case in…
We consider stochastic systems of interacting particles or agents, with dynamics determined by an interaction kernel which only depends on pairwise distances. We study the problem of inferring this interaction kernel from observations of…
In this paper we consider the problem of parameter inference for Markov jump process (MJP) representations of stochastic kinetic models. Since transition probabilities are intractable for most processes of interest yet forward simulation is…
Dynamic processes in complex networks are crucial for better understanding collective behavior in human societies, biological systems, and the internet. In this paper, we first focus on the continuous Markov-based modeling of evolving…
We introduce a class of Markov processes conditioned to avoid intersection over a moving time window of length T>0, a setting we refer to as myopic non-intersection. In particular, we study a system of myopic non-intersecting Brownian…
In this paper, we study discrete-time absorbing Markov Decision Processes (MDP) with measurable state space and Borel action space with a given initial distribution. For such models, solutions to the characteristic equation that are not…
Reinforcement learning algorithms are typically designed for generic Markov Decision Processes (MDPs), where any state-action pair can lead to an arbitrary transition distribution. In many practical systems, however, only a subset of the…
We study discrete-time predictable forward processes when trading times do not coincide with performance evaluation times in a binomial tree model for the financial market. The key step in the construction of these processes is to solve a…
We study a system of $N$ interacting particles on $\bf{Z}$. The stochastic dynamics consists of two components: a free motion of each particle (independent random walks) and a pair-wise interaction between particles. The interaction belongs…
In this paper, we study pinning control problem of coupled dynamical systems with stochastically switching couplings and stochastically selected controller-node set. Here, the coupling matrices and the controller-node sets change with time,…
Constrained Markov processes, such as reflecting diffusions, behave as an unconstrained process in the interior of a domain but upon reaching the boundary are controlled in some way so that they do not leave the closure of the domain. In…