Related papers: Stochastic Dynamics II: Finite Random Dynamical Sy…

Stochastic dynamics is generated by a matrix of transition probabilities. Certain eigenvectors of this matrix provide observables, and when these are plotted in the appropriate multi-dimensional space the phases (in the sense of phase…

Maximum entropy models provide the least constrained probability distributions that reproduce statistical properties of experimental datasets. In this work we characterize the learning dynamics that maximizes the log-likelihood in the case…

We analyze the probability distribution for entropy production rates of trajectories evolving on a class of out-of-equilibrium kinetic networks. These networks can serve as simple models for driven dynamical systems, which are of particular…

Markov decision processes (MDPs) are a popular model for decision-making in the presence of uncertainty. The conventional view of MDPs in verification treats them as state transformers with probabilities defined over sequences of states and…

We propose a new, nonparametric approach to learning and representing transition dynamics in Markov decision processes (MDPs), which can be combined easily with dynamic programming methods for policy optimisation and value estimation. This…

We study infinite-horizon robust Markov decision processes (MDPs) on continuous state spaces with structured rectangular ambiguity set. The proposed ambiguity set falls within the convex hull of unknown generating kernels. We utilize the…

The entropy production is one of the most essential features for systems operating out of equilibrium. The formulation for discrete-state systems goes back to the celebrated Schnakenberg's work and hitherto can be carried out when for each…

Controllers for dynamical systems that operate in safety-critical settings must account for stochastic disturbances. Such disturbances are often modeled as process noise in a dynamical system, and common assumptions are that the underlying…

Stochastic model-predictive control (SMPC) has evolved to a powerful framework for the control of stochastic dynamical systems. SMPC utilizes a probabilistic uncertainty description to provide a systematic trade-off between the control…

We obtain the posterior distribution of a random process conditioned on observing the empirical frequencies of a finite sample path. We find under a rather broad assumption on the "dependence structure" of the process, {\em c.f.}…

Markov processes with stochastic resetting towards the origin generically converge towards non-equilibrium steady-states. Long dynamical trajectories can be thus analyzed via the large deviations at Level 2.5 for the joint probability of…

The Markov decision process (MDP) formulation used to model many real-world sequential decision making problems does not efficiently capture the setting where the set of available decisions (actions) at each time step is stochastic.…

We review the behaviour of the Gibbs' and conditional entropies in deterministic and stochastic systems and continue to a formulation appropriate for a stochastically perturbed system with delayed dynamics. The underlying question driving…

For a network of discrete states with a periodically driven Markovian dynamics, we develop an inference scheme for an external observer who has access to some transitions. Based on waiting-time distributions between these transitions, the…

We consider a class of piecewise-deterministic Markov processes where the state evolves according to a linear dynamical system. This continuous time evolution is interspersed by discrete events that occur at random times and change (reset)…

We study the relation between different Markov Decision Process (MDP) frameworks in the machine learning and econometrics literatures, including the standard MDP, the entropy and general regularized MDP, and stochastic MDP, where the latter…

A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…

Markov decision processes (MDPs) are formal models commonly used in sequential decision-making. MDPs capture the stochasticity that may arise, for instance, from imprecise actuators via probabilities in the transition function. However, in…

Filyokov and Karpov [Inzhenerno-Fizicheskii Zhurnal 13, 624 (1967)] have proposed a theory of non-equilibrium steady states in direct analogy with the theory of equilibrium states : the principle is to maximize the Shannon entropy…

Entropic dynamics is a framework in which the laws of dynamics are derived as an application of entropic methods of inference. Its successes include the derivation of quantum mechanics and quantum field theory from probabilistic principles.…