Related papers: Variational and optimal control representations of…
The long-timescale behavior of complex dynamical systems can be described by linear Markov or Koopman models in a suitable latent space. Recent variational approaches allow the latent space representation and the linear dynamical model to…
We study optimal control of Markov processes with age-dependent transition rates. The control policy is chosen continuously over time based on the state of the process and its age. We study infinite horizon discounted cost and infinite…
We study the performance of a stochastic algorithm based on the power method that adaptively learns the large deviation functions characterizing the fluctuations of additive functionals of Markov processes, used in physics to model…
We calculate the large deviation functions characterizing the long-time fluctuations of the occupation of drifted Brownian motion and show that these functions have non-analytic points. This provides the first example of dynamical phase…
We obtain the posterior distribution of a random process conditioned on observing the empirical frequencies of a finite sample path. We find under a rather broad assumption on the "dependence structure" of the process, {\em c.f.}…
The problem of constrained Markov decision process is considered. An agent aims to maximize the expected accumulated discounted reward subject to multiple constraints on its costs (the number of constraints is relatively small). A new dual…
These notes give a summary of techniques used in large deviation theory to study the fluctuations of time-additive quantities, called dynamical observables, defined in the context of Langevin-type equations, which model equilibrium and…
Planning for distributed agents with partial state information is considered from a decision- theoretic perspective. We describe generalizations of both the MDP and POMDP models that allow for decentralized control. For even a small number…
Driven Langevin processes have appeared in a variety of fields due to the relevance of natural phenomena having both deterministic and stochastic effects. The stochastic currents and fluxes in these systems provide a convenient set of…
Trajectory optimization is a fundamental stochastic optimal control problem. This paper deals with a trajectory optimization approach for dynamical systems subject to measurement noise that can be fitted into linear time-varying stochastic…
The thermodynamic formalism, which was first developed for dynamical systems and then applied to discrete Markov processes, turns out to be well suited for continuous time Markov processes as well, provided the definitions are interpreted…
The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…
We investigated a cost-constrained static ergodic control problem of the variance of measure-valued affine processes and its application in streamflow management. The controlled system is a jump-driven mixed moving average process that…
We consider the stochastic optimal control problem of nonlinear mean-field systems in discrete time. We reformulate the problem into a deterministic control problem with marginal distribution as controlled state variable, and prove that…
We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…
For Markov processes evolving on multiple time-scales a combination of large component scalings and averaging of rapid fluctuations can lead to useful limits for model approximation. A general approach to proving a law of large numbers to a…
Semi-Markov processes play an important role in the effective description of partially accessible systems in stochastic thermodynamics. They occur, for instance, in coarse-graining procedures such as state lumping and when analyzing waiting…
We consider the problem of optimally controlling stochastic, Markovian systems subject to joint chance constraints over a finite-time horizon. For such problems, standard Dynamic Programming is inapplicable due to the time correlation of…
In the present work we employ, for the first time, backward stochastic differential equations (BSDEs) to study the optimal control of semi-Markov processes on finite horizon, with general state and action spaces. More precisely, we prove…
The reduction of a continuous Markov process with multiple metastable states to a discrete rate process is investigated in the presence of slow time dependent parameters such as periodic external forces or slowly fluctuating barrier…