Related papers: Periodic-orbit determination of dynamical correlat…
Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…
The thermodynamic formalism, which was first developed for dynamical systems and then applied to discrete Markov processes, turns out to be well suited for continuous time Markov processes as well, provided the definitions are interpreted…
This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of…
We present an algorithm for finding the probabilities of rare events in nonequilibrium processes. The algorithm consists of evolving the system with a modified dynamics for which the required event occurs more frequently. By keeping track…
We consider the noise-induced transitions in the randomly perturbed discrete logistic map from a linearly stable periodic orbit consisting of T periodic points. The traditional large deviation theory and asymptotic analysis for small noise…
A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…
The Riemann surface associated with counting the current between two states of an underlying Markov process is hyperelliptic. We explore the consequences of this property for the time-dependent probability of that current for Markov…
A shift-periodic map is a one-dimensional map from the real line to itself which is periodic up to a linear translation and allowed to have singularities. It is shown that iterative sequences $x_{n+1}=F(x_n)$ generated by such maps display…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
We study discrete-time, discrete-state multistate Markov models from the perspective of algebraic statistics. These models are widely studied in event history analysis, and are characterized by the state space, the initial distribution and…
Direct simulation of biomolecular dynamics in thermal equilibrium is challenging due to the metastable nature of conformation dynamics and the computational cost of molecular dynamics. Biased or enhanced sampling methods may improve the…
We derive a finite-sample probabilistic bound on the parameter estimation error of a system identification algorithm for Linear Switched Systems. The algorithm estimates Markov parameters from a single trajectory and applies a variant of…
For a stochastic system, its evolution from one state to another can have a large number of possible paths. Non-uniformity in the field of system variables leads the local dynamics in state transition varies considerably from path to path…
Non-linear Hawkes processes with memory kernels given by the sum of Erlang kernels are considered. It is shown that their stability properties can be studied in terms of an associated class of piecewise deterministic Markov processes,…
Piecewise-deterministic Markov processes (PDMPs) are often used to model abrupt changes in the global environment or capabilities of a controlled system. This is typically done by considering a set of "operating modes" (each with its own…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
We address the general problem of formulating the dynamical large deviations of non-Markovian systems in a closed form. Specifically, we consider a broad class of ``self-interacting'' jump processes whose dynamics depends on the past…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
We introduce discrete time Markov chains that preserve uniform measures on boxed plane partitions. Elementary Markov steps change the size of the box from (a x b x c) to ((a-1) x (b+1) x c) or ((a+1) x (b-1) x c). Algorithmic realization of…
By varying a parameter of a one-dimensional piecewise smooth map, stable periodic orbits are observed. In this paper, complete analytic characterization of these stable periodic orbits is obtained. An interesting relationship between the…