Related papers: Markov matrix perturbations to optimize dynamical …
The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…
An optimal feedback controller for a given Markov decision process (MDP) can in principle be synthesized by value or policy iteration. However, if the system dynamics and the reward function are unknown, a learning agent must discover an…
This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization…
Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past…
Estimation of Markov Random Field and covariance models from high-dimensional data represents a canonical problem that has received a lot of attention in the literature. A key assumption, widely employed, is that of {\em sparsity} of the…
Modeling and simulating movement of vehicles in established transportation infrastructures, especially in large urban road networks is an important task. It helps with understanding and handling traffic problems, optimizing traffic…
We analyse diffusion dynamics on weakly-coupled networks (interconnected networks) by means of separation of time scales. Using an adiabatic approximation we reduced the system dynamics to a Markov chain with aggregated variables and…
We develop a system-theoretic framework for the structured analysis of distributed optimization algorithms with decomposable cost functions. We model such algorithms as a network of interacting dynamical systems and derive tests for…
This statistical physics thesis focuses on the study of three kinds of systems which display repulsive interactions: eigenvalues of random matrices, non-crossing random walks and trapped fermions. These systems share many links, which can…
The separation of the Schr\"{o}dinger equation into a Markovian and an interference term provides a new insight in the quantum dynamics of classically chaotic systems. The competition between these two terms determines the localized or…
Due to uncertainties and the complicated intrinsic dynamics of power systems, it is difficult to predict the cascading failure paths once the cascades occur. This makes it challenging to achieve the effective power system protection against…
This paper proposes a new algorithm for compensating external disturbances for class of multi-channel linear systems. The solution to this problem is based on the use of the internal model principle and the extended error adaptation…
Stochastic processes find applications in modelling systems in a variety of disciplines. A large number of stochastic models considered are Markovian in nature. It is often observed that higher order Markov processes can model the data…
We consider a Markov jump process on a general state space to which we apply a time-dependent weak perturbation over a finite time interval. By martingale-based stochastic calculus, under a suitable exponential moment bound for the…
We study the performance of a stochastic algorithm based on the power method that adaptively learns the large deviation functions characterizing the fluctuations of additive functionals of Markov processes, used in physics to model…
Optimal control synthesis in stochastic systems with respect to quantitative temporal logic constraints can be formulated as linear programming problems. However, centralized synthesis algorithms do not scale to many practical systems. To…
Given a quantum many-body system and the expectation-value dynamics of some operator, we study how this reference dynamics is altered due to a perturbation of the system's Hamiltonian. Based on projection operator techniques, we unveil that…
These notes give a summary of techniques used in large deviation theory to study the fluctuations of time-additive quantities, called dynamical observables, defined in the context of Langevin-type equations, which model equilibrium and…
The linear response of non-equilibrium systems with Markovian dynamics satisfies a generalized fluctuation-dissipation relation derived from time symmetry and antisymmetry properties of the fluctuations. The relation involves the sum of two…
Many complex systems have natural representations as multi-layer networks. While these formulations retain more information than standard single-layer network models, there is not yet a fully developed theory for computing network metrics…