Related papers: Markov matrix perturbations to optimize dynamical …
Jump Markov linear systems (JMLS) are a useful class which can be used to model processes which exhibit random changes in behavior during operation. This paper presents a numerically stable method for learning the parameters of jump Markov…
Upper bounds are derived on the total variation distance between the invariant distributions of two stochastic matrices differing on a subset W of rows. Such bounds depend on three parameters: the mixing time and the minimal expected…
Strong and weak simulation relations have been proposed for Markov chains, while strong simulation and strong probabilistic simulation relations have been proposed for probabilistic automata. However, decision algorithms for strong and weak…
Exact unstable solutions of the Navier-Stokes equations are thought to underpin the dynamics of turbulence, but are usually computed in minimal computational domains. Here, we extend this dynamical systems approach to spatially extended…
This paper addresses the problem of planning under uncertainty in large Markov Decision Processes (MDPs). Factored MDPs represent a complex state space using state variables and the transition model using a dynamic Bayesian network. This…
We show that nonequilibrium dynamics can play a constructive role in unsupervised machine learning by inducing the spontaneous emergence of latent-state cycles. We introduce a model in which visible and hidden variables interact through two…
Regularization of control policies using entropy can be instrumental in adjusting predictability of real-world systems. Applications benefiting from such approaches range from, e.g., cybersecurity, which aims at maximal unpredictability, to…
Predictability of behavior has emerged an an important characteristic in many fields including biology, medicine, and marketing. Behavior can be recorded as a sequence of actions performed by an individual over a given time period. This…
Parameter estimation in Markov random fields (MRFs) is a difficult task, in which inference over the network is run in the inner loop of a gradient descent procedure. Replacing exact inference with approximate methods such as loopy belief…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…
We have developed a steady state theory of complex transport networks used to model the flow of commodity, information, viruses, opinions, or traffic. Our approach is based on the use of the Markov chains defined on the graph…
Stochastic and soft optimal policies resulting from entropy-regularized Markov decision processes (ER-MDP) are desirable for exploration and imitation learning applications. Motivated by the fact that such policies are sensitive with…
The Euler and Navier-Stokes fluid mechanics equations are derived using a modified statistical mechanical approach using theory taken from the Chapman-Enskog perturbation analysis used to support the lattice Boltzmann method. Additional…
In a series of works published in the 1990-s, Kerov put forth various applications of the circle of ideas centred at the Markov moment problem to the limiting shape of random continual diagrams arising in representation theory and spectral…
In this paper, we study the problem of optimizing the stability of positive semi-Markov jump linear systems. We specifically consider the problem of tuning the coefficients of the system matrices for maximizing the exponential decay rate of…
Distributed algorithms for both discrete-time and continuous-time linearly solvable optimal control (LSOC) problems of networked multi-agent systems (MASs) are investigated in this paper. A distributed framework is proposed to partition the…
This paper investigates the distributed computation of the well-known linear matrix equation in the form of $AXB = F$, with the matrices A, B, X, and F of appropriate dimensions, over multi-agent networks from an optimization perspective.…
Using equilibrium fluctuations to understand the response of a physical system to an externally imposed perturbation is the basis for linear response theory, which is widely used to interpret experiments and shed light on microscopic…
We introduce a scheme for deriving an optimally-parametrised Langevin dynamics of few collective variables from data generated in molecular dynamics simulations. The drift and the position-dependent diffusion profiles governing the Langevin…
Relational Markov Decision Processes are a useful abstraction for complex reinforcement learning problems and stochastic planning problems. Recent work developed representation schemes and algorithms for planning in such problems using the…