Related papers: On Markovian behaviour of $p$-adic random dynamica…

Machine learning methods have proved to be useful for the recognition of patterns in statistical data. The measurement outcomes are intrinsically random in quantum physics, however, they do have a pattern when the measurements are performed…

Noisy sensing, imperfect control, and environment changes are defining characteristics of many real-world robot tasks. The partially observable Markov decision process (POMDP) provides a principled mathematical framework for modeling and…

The so-called Maryland model is a linear version of the quantum kicked rotor; it exhibits Anderson localization in momentum space. By turning the kicks into a Markovian stochastic process, the dynamics becomes a dissipative quantum process…

Continuous-time Markov chains are used to model stochastic systems where transitions can occur at irregular times, e.g., birth-death processes, chemical reaction networks, population dynamics, and gene regulatory networks. We develop a…

We analyze the properties of degree-preserving Markov chains based on elementary edge switchings in undirected and directed graphs. We give exact yet simple formulas for the mobility of a graph (the number of possible moves) in terms of its…

Online learning algorithms are designed to perform in non-stationary environments, but generally there is no notion of a dynamic state to model constraints on current and future actions as a function of past actions. State-based models are…

We define indexed categories of (open) dynamical system and random dynamical system over polynomial interfaces, where time is given by an arbitrary monoid $\mathbb{T}$. We consider the case of open random dynamical systems over both open…

Large deviation results are given for a class of perturbed nonhomogeneous Markov chains on finite state space which formally includes some stochastic optimization algorithms. Specifically, let {P_n} be a sequence of transition matrices on a…

Using random matrices, we study the reduced dynamics of a two level system interacting with a generic environment. In the weak coupling limit, the result can be obtained directly from known results for purity decay, and result in Markovian…

We consider Markov Decision Processes (MDPs) in which every stationary policy induces the same graph structure for the underlying Markov chain and further, the graph has the following property: if we replace each recurrent class by a node,…

We present a class of stochastic processes in which the large deviation functions of time-integrated observables exhibit singularities that relate to dynamical phase transitions of trajectories. These illustrative examples include Brownian…

Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…

We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…

Stochastic approximation is a framework unifying many random iterative algorithms occurring in a diverse range of applications. The stability of the process is often difficult to verify in practical applications and the process may even be…

Realistic quantum mechanical systems are always exposed to an external environment. The presence of the environment often gives rise to a Markovian process in which the system loses information to its surroundings. However, many quantum…

This paper contributes an in-depth study of properties of continuous time Markov chains (CTMCs) on non-negative integer lattices $\N_0^d$, with particular interest in one-dimensional CTMCs with polynomial transitions rates. Such stochastic…

The first motivation of this paper is to study stationarity and ergodic properties for a general class of time series models defined conditional on an exogenous covariates process. The dynamic of these models is given by an autoregressive…

Convex combinations of the completely positive (CP) as well as CP-divisible, continuous time dynamical maps arising from collision models are investigated. While the individual maps are both CP and Markovian we find that convex combinations…

Conserved dynamical systems are generally considered to be critical. We study a class of critical routing models, equivalent to random maps, which can be solved rigorously in the thermodynamic limit. The information flow is conserved for…

The work treats systems combining slow and fast motions depending on each other where fast motions are perturbations of families of either dynamical systems or Markov processes with freezed slow variable. In the first case we consider…