Related papers: Variational and optimal control representations of…
Consider an N-dimensional Markov chain obtained from N one-dimensional random walks by Doob h-transform with the q-Vandermonde determinant. We prove that as N becomes large, these Markov chains converge to an infinite-dimensional Feller…
Optimal control synthesis in stochastic systems with respect to quantitative temporal logic constraints can be formulated as linear programming problems. However, centralized synthesis algorithms do not scale to many practical systems. To…
Time-reversal symmetry of microscopic laws dictates that the equilibrium distribution of a stochastic process must obey the detailed balance. On the other hand, cyclic Markov processes that do not admit equilibrium distributions with…
Inference, prediction and control of complex dynamical systems from time series is important in many areas, including financial markets, power grid management, climate and weather modeling, or molecular dynamics. The analysis of such highly…
We show the variational convergence of an irreversible Markov jump process describing a finite stochastic particle system to the solution of a countable infinite system of deterministic time-inhomogeneous quadratic differential equations…
In this paper, we consider a modified version of the control problem in a model free Markov decision process (MDP) setting with large state and action spaces. The control problem most commonly addressed in the contemporary literature is to…
We investigate the optimal control of open quantum systems, in particular, the mutual influence of driving and dissipation. A stochastic approach to open-system control is developed, using a generalized version of Krotov's iterative…
We study the large deviations of time-integrated observables of Markov diffusions that have perfectly reflecting boundaries. We discuss how the standard spectral approach to dynamical large deviations must be modified to account for such…
Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…
Controlling stochastic systems with unknown dynamics and under complex specifications is specially challenging in safety-critical settings, where performance guarantees are essential. We propose a data-driven policy synthesis framework that…
When analysing statistical systems or stochastic processes, it is often interesting to ask how they behave given that some observable takes some prescribed value. This conditioning problem is well understood within the linear operator…
In the paper we study continuous time controlled Markov processes using discrete time controlled Markov processes. We consider long run functionals: average reward per unit time or long run risk sensitive functional. We also investigate…
We establish an abstract, effective, exponential large deviations type estimate for Markov systems satisfying a weaker form of mixing. We employ this result to derive such estimates, as well as a central limit theorem, for the skew product…
We consider a class of stochastic impulse control problems of general stochastic processes i.e. not necessarily Markovian. Under fairly general conditions we establish existence of an optimal impulse control. We also prove existence of…
We revisit classic ergodic large-deviation principles: for the occupation measure (Donsker-Varadhan), and for the empirical flux. We show that these problems can be embedded into a more general, discrete-time framework. A conditioning and…
In this work we provide a computationally tractable procedure for designing affine control policies, applied to constrained, discrete-time, partially observable, linear systems subject to set bounded disturbances, stochastic noise and…
We present a new, tractable method for solving and analyzing risk-aware control problems over finite and infinite, discounted time-horizons where the dynamics of the controlled process are described as a martingale problem. Supposing…
In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Building upon recent advances in Markov chain approximation methods and sampling-based algorithms for deterministic path planning,…
In this paper, we study a mean-variance optimization problem in an infinite horizon discrete time discounted Markov decision process (MDP). The objective is to minimize the variance of system rewards with the constraint of mean performance.…
There are multiple ways in which a stochastic system can be out of statistical equilibrium. It might be subject to time-varying forcing; or be in a transient phase on its way towards equilibrium; it might even be in equilibrium without us…