Related papers: Optimal linear responses for Markov chains and sto…

Stabilization of linear systems with unknown dynamics is a canonical problem in adaptive control. Since the lack of knowledge of system parameters can cause it to become destabilized, an adaptive stabilization procedure is needed prior to…

Predicting how systems respond to external perturbations far from equilibrium remains a fundamental challenge across physics, chemistry, and biology. We present a unified response framework for stochastic Markov dynamics that integrates…

This paper considers an optimal impulse control problem of dynamical systems generated by a flow. The performance criteria are total costs over the infinite time horizon. Apart from the main performance to be minimized, there are multiple…

For a discrete time Markov chain and in line with Strotz' consistent planning we develop a framework for problems of optimal stopping that are time-inconsistent due to the consideration of a non-linear function of an expected reward. We…

In this article a new algorithm for the design of stationary input sequences for system identification is presented. The stationary input signal is generated by optimizing an approximation of a scalar function of the information matrix,…

This article presents several results establishing connections be- tween Markov chains and dynamical systems, from the point of view of open systems in physics. We show how all Markov chains can be understood as the information on one…

In this paper we set up an optimal control framework for a hybrid stochastic system with dual or multiple Markov switching diffusion processes, while Markov chains governing these switching diffusions are not identical as assumed by the…

There are multiple ways in which a stochastic system can be out of statistical equilibrium. It might be subject to time-varying forcing; or be in a transient phase on its way towards equilibrium; it might even be in equilibrium without us…

Finite-time coherent sets represent minimally mixing objects in general nonlinear dynamics, and are spatially mobile features that are the most predictable in the medium term. When the dynamical system is subjected to small parameter…

We study the problem of learning the transition matrices of a set of Markov chains from a single stream of observations on each chain. We assume that the Markov chains are ergodic but otherwise unknown. The learner can sample Markov chains…

Large deviation results are given for a class of perturbed nonhomogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {P_n} be a sequence of transition matrices on a…

In this paper, we establish a general stochastic maximum principle for optimal control for systems described by a continuous-time Markov regime-switching stochastic recursive utilities model. The control domain is postulated not to be…

In this paper we study the bicausal optimal transport problem for Markov chains, an optimal transport formulation suitable for stochastic processes which takes into consideration the accumulation of information as time evolves. Our analysis…

The conventional perspective on Markov chains considers decision problems concerning the probabilities of temporal properties being satisfied by traces of visited states. However, consider the following query made of a stochastic system…

Efficiently computing the optimal control policy concerning a complicated future with stochastic disturbance has always been a challenge. The predicted stochastic future disturbance can be represented by a scenario tree, but solving the…

This paper deals with a distributed implementation of minimax adaptive control algorithm for networked dynamical systems modeled by a finite set of linear models. To hedge against the uncertainty arising out of finite number of possible…

Switched linear systems are time-varying nonlinear systems whose dynamics switch between different modes, where each mode corresponds to different linear dynamics. They arise naturally to model unexpected failures, environment uncertainties…

Scenario reduction algorithms can be an effective means to provide a tractable description of the uncertainty in optimal control problems. However, they might significantly compromise the performance of the controlled system. In this paper,…

We propose a novel randomized linear programming algorithm for approximating the optimal policy of the discounted Markov decision problem. By leveraging the value-policy duality and binary-tree data structures, the algorithm adaptively…

We consider the optimal control design problem for discrete-time LTI systems with state feedback, when the actuation signal is subject to unmeasurable switching propagation delays, due to e.g. the routing in a multi-hop communication…