Related papers: MSM lag time cannot be used for variational model …
The purpose of this paper is to study the time average behavior of Markov chains with transition probabilities being kernels of completely continuous operators, and therefore to provide a sufficient condition for a class of Markov chains…
The optimal control for mobile agents is an important and challenging issue. Recent work shows that using randomized mechanism in agents' control can make the state unpredictable, and thus improve the security of agents. However, the…
We propose certain conditions which are sufficient for the functional law of the iterated logarithm (the Strassen invariance principle) for some general class of non-stationary Markov-Feller chains. This class may be briefly specified by…
A comparison theorem for state-dependent regime-switching diffusion processes is established, which enables us to control pathwisely the evolution of the state-dependent switching component simply by Markov chains. Moreover, a sharp…
We consider a two-dimensional Hamiltonian system perturbed by a small diffusion term, whose coefficient is state-dependent and non-degenerate. As a result, the process consists of the fast motion along the level curves and slow motion…
The problem of minimizing queueing delay of opportunistic access of multiple continuous time Markov channels is considered. A new access policy based on myopic sensing and adaptive transmission (MS-AT) is proposed. Under the framework of…
Using a dynamical model to make predictions about a system has many sources of error. These can include errors in how the model was initialised but also errors in the dynamics of the model itself. For many applications in data assimilation,…
Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…
A large deviations principle is established for the joint law of the empirical measure and the flow measure of a renewal Markov process on a finite graph. We do not assume any bound on the arrival times, allowing heavy tailed distributions.…
Long-run average optimization problems for Markov decision processes (MDPs) require constructing policies with optimal steady-state behavior, i.e., optimal limit frequency of visits to the states. However, such policies may suffer from…
Multi-state models are commonly used for intermittent observations of a state over time, but these are generally based on the Markov assumption, that transition rates are independent of the time spent in current and previous states. In a…
Safety-critical navigation applications require that estimation errors be reliably quantified and bounded. This can be challenging for linear dynamic systems if the process noise or measurement errors have uncertain time correlation. In…
Predictive statistical mechanics is a form of inference from available data, without additional assumptions, for predicting reproducible phenomena. By applying it to systems with Hamiltonian dynamics, a problem of predicting the macroscopic…
We study a finite-horizon covariance steering problem for discrete-time Markov jump linear systems (MJLS) with both state- and control-dependent multiplicative noise. The objective is to minimize a quadratic running cost while steering the…
Adaptive Markov chain Monte Carlo (MCMC) algorithms, which automatically tune their parameters based on past samples, have proved extremely useful in practice. The self-tuning mechanism makes them `non-Markovian', which means that their…
In this paper, we analyze the dynamics of spreading processes taking place over time-varying networks. A common approach to model time-varying networks is via Markovian random graph processes. This modeling approach presents the following…
Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `$r$' of a discrete time Markov chain with `$\mu_{\pi}$' as invariant ergodic measure, possibly having support on an unbounded state space. The…
Current molecular dynamic simulations of biomolecules using multiple time steps to update the slowingly changing force are hampered by an instability occuring at time step equal to half the period of the fastest vibrating mode. This has…
We prove a large deviation principle on path space for a class of discrete time Markov processes whose state space is the intersection of a regular domain $\L\subset \R^d$ with some lattice of spacing $\e$. Transitions from $x$ to $y$ are…
We have shown recently that a Markov process conditioned on rare events involving time-integrated random variables can be described in the long-time limit by an effective Markov process, called the driven process, which is given…