Related papers: A method for inferring hierarchical dynamics in st…

Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical…

In this paper we explore the mathematical structure of hierarchical organization in smooth dynamical systems. We start by making precise what we mean by a level in a hierarchy, and how the higher le vels need to respect the dynamics on the…

Markov models are widely used to describe processes of stochastic dynamics. Here, we show that Markov models are a natural consequence of the dynamical principle of Maximum Caliber. First, we show that when there are different possible…

This paper studies the dynamic programming principle using the measurable selection method for stochastic control of continuous processes. The novelty of this work is to incorporate intermediate expectation constraints on the canonical…

We study the problem of finite-horizon probabilistic invariance for discrete-time Markov processes over general (uncountable) state spaces. We compute discrete-time, finite-state Markov chains as formal abstractions of general Markov…

I propose a large class of stochastic Markov processes associated with probability distributions analogous to that of lattice gauge theory with dynamical fermions. The construction incorporates the idea of approximate spectral split of the…

We investigate planning in time-critical domains represented as Markov Decision Processes, showing that search based techniques can be a very powerful method for finding close to optimal plans. To reduce the computational cost of planning…

We study the performance of a stochastic algorithm based on the power method that adaptively learns the large deviation functions characterizing the fluctuations of additive functionals of Markov processes, used in physics to model…

Complex systems made of interacting elements are commonly abstracted as networks, in which nodes are associated with dynamic state variables, whose evolution is driven by interactions mediated by the edges. Markov processes have been the…

We develop an approach for Bayesian learning of spatiotemporal dynamical mechanistic models. Such learning consists of statistical emulation of the mechanistic system that can efficiently interpolate the output of the system from arbitrary…

The behavior and architecture of large scale discrete state systems found in computer software and hardware can be specified and analyzed using a particular class of primitive recursive functions. This paper begins with an illustration of…

Complex systems are characterized by a huge number of degrees of freedom often interacting in a non-linear manner. In many cases macroscopic states, however, can be characterized by a small number of order parameters that obey stochastic…

Stochastic thermodynamics provides the framework to analyze thermodynamic laws and quantities along individual trajectories of small but fully observable systems. If the observable level fails to capture all relevant degrees of freedom,…

We describe an approach for exploiting structure in Markov Decision Processes with continuous state variables. At each step of the dynamic programming, the state space is dynamically partitioned into regions where the value function is the…

The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…

Discrete-space kinetic models, i.e., Markov state models, have emerged as powerful tools for reducing the complexity of trajectories generated from molecular dynamics simulations. These models require configuration-space representations…

In present paper we suggest a new universal approach to study complex systems by microscopic, mesoscopic and macroscopic methods. We discuss new possibilities of extracting information on nonstationarity, unsteadiness and non-Markovity of…

Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…

This paper presents a general approach to linear stochastic processes driven by various random noises. Mathematically, such processes are described by linear stochastic differential equations of arbitrary order (the simplest non-trivial…

Stochastic gradient methods are the workhorse (algorithms) of large-scale optimization problems in machine learning, signal processing, and other computational sciences and engineering. This paper studies Markov chain gradient descent, a…