Related papers: Optimal linear responses for Markov chains and sto…

We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the…

A general theory is developed to study individual based models which are discrete in time. We begin by constructing a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are…

Stochastic optimization problems often involve data distributions that change in reaction to the decision variables. This is the case for example when members of the population respond to a deployed classifier by manipulating their features…

Motivated by the Internet-of-things and sensor networks for cyberphysical systems, the problem of dynamic sensor activation for the tracking of a time-varying process is examined. The tradeoff is between energy efficiency, which decreases…

Perturbation analysis of Markov chains provides bounds on the effect that a change in a Markov transition matrix has on the corresponding stationary distribution. This paper compares and analyzes bounds found in the literature for finite…

This article is concerned with stability analysis and stabilization of randomly switched nonlinear systems. These systems may be regarded as piecewise deterministic stochastic systems: the discrete switches are triggered by a stochastic…

The problem of optimal real-time transmission of a Markov source under constraints on the expected number of transmissions is considered, both for the discounted and long term average cases. This setup is motivated by applications where…

We propose a method for approximating solutions to optimization problems involving the global stability properties of parameter-dependent continuous-time autonomous dynamical systems. The method relies on an approximation of the…

This paper investigates optimal portfolio strategies in a market where the drift is driven by an unobserved Markov chain. Information on the state of this chain is obtained from stock prices and expert opinions in the form of signals at…

Applications of stochastic models often involve the evaluation of steady-state performance, which requires solving a set of balance equations. In most cases of interest, the number of equations is infinite or even uncountable. As a result,…

We introduce and solve a general model of dynamic response under external perturbations. This model captures a wide range of systems out of equilibrium including Ising models of physical systems, social opinions, and population genetics.…

We present here a real-time control model for the train dynamics in a linear metro line system. The model describes the train dynamics taking into account average passenger arrival rates on platforms, including control laws for train dwell…

Model Predictive Control is an extremely effective control method for systems with input and state constraints. Model Predictive Control performance heavily depends on the accuracy of the open-loop prediction. For systems with uncertainty…

We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…

We construct optimally robust realizations of a given rational transfer function that represents a passive discrete-time system. We link it to the solution set of linear matrix inequalities defining passive transfer functions. We also…

In this paper, we consider a discrete-time stochastic control problem with uncertain initial and target states. We first discuss the connection between optimal transport and stochastic control problems of this form. Next, we formulate a…

Understanding how systems respond to external perturbations is a fundamental challenge in physics, particularly for non-equilibrium and non-stationary processes. The fluctuation-dissipation theorem provides a complete framework for…

Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…

Stochastic reaction networks are mathematical models with a wide range of applications in biochemistry, ecology, and epidemiology, and are often complex to analyze. Except for some special cases, it is generally difficult to predict how the…

A general formalism is developed to construct a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are therefore internal to the system and not externally specified. For…