Related papers: Discrete Time Markovian Agents Interacting Through…
In this paper, we equip the conventional discrete-time queueing network with a Markovian input process, that, in addition to the usual short-term stochastics, governs the mid- to long-term behavior of the links between the network nodes.…
We study the large deviations of the time-integrated current for a self-propelled particle moving within a confined environment. The dynamics is modeled as a semi-Markovian process, where the transitions between a \textit{normal running…
Data-based inference of directed interactions in complex dynamical systems is a problem common to many disciplines of science. In this work, we study networks of spatially separate dynamical entities, which could represent physical systems…
Modelling the evolution of a system using stochastic dynamics typically implies a greater subjective uncertainty in the adopted system coordinates as time progresses, and stochastic entropy production has been developed as a measure of this…
We derive Markovian master equations of single and interacting harmonic systems in different scenarios, including strong internal coupling. By comparing the dynamics resulting from the corresponding Markovian master equations with exact…
Multi-agent reinforcement learning methods have shown remarkable potential in solving complex multi-agent problems but mostly lack theoretical guarantees. Recently, mean field control and mean field games have been established as a…
In this paper we study multiagent models with time-varying type change. Assume that there exist a closed system of $N$ agents classified into $r$ types according to their states of an internal system; each agent changes its type by an…
We consider the problem of parameter estimation for a stochastic McKean-Vlasov equation, and the associated system of weakly interacting particles. We study two cases: one in which we observe multiple independent trajectories of the…
Decentralized stochastic control problems with local information involve problems where multiple agents and subsystems which are coupled via dynamics and/or cost are present. Typically, however, the dynamics of such couplings is complex and…
We review recent work on systems with multiple interacting-particles having the dynamical feature of stochastic resetting. The interplay of time scales related to inter-particle interactions and resetting leads to a rich behavior, both…
We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion…
Advances in experimental techniques allow the collection of high-resolution spatio-temporal data that track individual motile entities over time. These tracking data motivate the use of mathematical models to characterise the motion…
The linear response of a dynamical system refers to changes to properties of the system when small external perturbations are applied. We consider the little-studied question of selecting an optimal perturbation so as to (i) maximise the…
We study discrete-time Markov Decision Processes (MDPs) on finite state-action spaces and analyze the stability of optimal policies and value functions in the long-run discounted risk-sensitive objective setting. Our analysis addresses…
Continuous-time Markov decision processes are an important class of models in a wide range of applications, ranging from cyber-physical systems to synthetic biology. A central problem is how to devise a policy to control the system in order…
Stochastic reaction networks, which are usually modeled as continuous-time Markov chains on $\mathbb Z^d_{\ge 0}$, and simulated via a version of the "Gillespie algorithm," have proven to be a useful tool for the understanding of processes,…
The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…
We study recursive maximum likelihood estimation for stochastic interacting particle systems based on continuous observation of a single particle. In this regime, consistent estimation of the finite-particle log-likelihood is not possible,…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
A classical approach to formal policy synthesis in stochastic dynamical systems is to construct a finite-state abstraction, often represented as a Markov decision process (MDP). The correctness of these approaches hinges on a behavioural…