Related papers: Periodic-orbit determination of dynamical correlat…
This study introduces a comparative modeling framework using stationary and non-stationary transition probabilities within a Markov Decision Process (MDP) to assess COVID-19 disease dynamics. Stationary transition probabilities assume…
This paper studies function approximation for finite horizon discrete time Markov decision processes under certain convexity assumptions. Uniform convergence of these approximations on compact sets is proved under several sampling schemes…
In this work, we consider a finite-state inhomogeneous-time Markov chain whose probabilities of transition from one state to another tend to decrease over time. This can be seen as a cooling of the dynamics of an underlying Markov chain. We…
We develop the connection between large deviation theory and more applied approaches to stochastic hybrid systems by highlighting a common underlying Hamiltonian structure. A stochastic hybrid system involves the coupling between a…
Periodic orbit action correlations are studied for the piecewise linear, area-preserving Baker map. Semiclassical periodic orbit formulae together with universal spectral statistics in the corresponding quantum Baker map suggest the…
Periodic orbits and cycles, respectively, play a significant role in discrete- and continuous-time dynamical systems (i.e. maps and flows). To succinctly describe their shifts when the system is applied perturbation, the notions of…
Lumping a Markov process introduces a coarser level of description that is useful in many contexts and applications. The dynamics on the coarse grained states is often approximated by its Markovian component. In this letter we derive…
This paper introduces two new algorithms to accurately estimate the process noise covariance of a discrete-time Kalman filter online for robust orbit determination in the presence of dynamics model uncertainties. Common orbit determination…
We establish new conditions for obtaining uniform bounds on the moments of discrete-time stochastic processes. Our results require a weak negative drift criterion along with a state-dependent restriction on the sizes of the one-step jumps…
Piecewise deterministic Markov processes (PDMPs) are a class of stochastic processes with applications in several fields of applied mathematics spanning from mathematical modeling of physical phenomena to computational methods. A PDMP is…
New results on conditional joint probability distributions of first exit times are presented for a continuous-time stochastic process defined as the mixture of Markov jump processes moving at different speeds on the same finite state space,…
The notion of a successful coupling of Markov processes, based on the idea that both components of the coupled system ``intersect'' in finite time with probability one, is extended to cover situations when the coupling is unnecessarily…
Markov models are often used to capture the temporal patterns of sequential data for statistical learning applications. While the Hidden Markov modeling-based learning mechanisms are well studied in literature, we analyze a…
We consider stochastic dynamical systems defined by differential equations with a uniform random time delay. The latter equations are shown to be equivalent to deterministic higher-order differential equations: for an $n$-th order equation…
In this paper, we seek to understand the behavior of dynamical systems that are perturbed by a parameter that changes discretely in time. If we impose certain conditions, we can study certain embedded systems within a hybrid system as…
We study a family of Markov processes on $\mathcal{P}^{(k)}$, the space of partitions of the natural numbers with at most $k$ blocks. The process can be constructed from a Poisson point process on…
We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the…
Safety-critical navigation applications require that estimation errors be reliably quantified and bounded. This can be challenging for linear dynamic systems if the process noise or measurement errors have uncertain time correlation. In…
We study the Classical Probability analogue of the dilations of a quantum dynamical semigroup in Quantum Probability. Given a (not necessarily homogeneous) Markov chain in discrete time in a finite state space E, we introduce a second…
The probabilistic description of the time evolution of a physical system can take two conceptually distinct forms: a trajectory of probabilities, which specifies how probabilities evolve over time, and a probability on trajectories, which…