Related papers: Sharp entrywise perturbation bounds for Markov cha…
When we speak about parametric programming, sensitivity analysis, or related topics, we usually mean the problem of studying specified perturbations of the data such that for a given optimization problem some optimality criterion remains…
Markov chains and diffusion processes are indispensable tools in machine learning and statistics that are used for inference, sampling, and modeling. With the growth of large-scale datasets, the computational cost associated with simulating…
We prove an invariance principle for non-stationary random processes and establish a rate of convergence under a new type of mixing condition. The dependence is exponentially decaying in the gap between the past and the future and is…
We study discrete time Markov processes with periodic or open boundary conditions and with inhomogeneous rates in the bulk. The Markov matrices are given by the inhomogeneous transfer matrices introduced previously to prove the…
In this paper a new estimator for the transition density $\pi$ of an homogeneous Markov chain is considered. We introduce an original contrast derived from regression framework and we use a model selection method to estimate $\pi$ under…
A continuous-time random walk in the quarter plane with homogeneous transition rates is considered. Given a non-negative reward function on the state space, we are interested in the expected stationary performance. Since a direct derivation…
We suggest an approach to obtaining general two-sided bounds on the rate of convergence in terms of special "weighted" norms related to total variation. Some important classes of continuous-time Markov chains are considered:…
The question of recurrence and transience of branching Markov chains is more subtle than for ordinary Markov chains; they can be classified in transience, weak recurrence, and strong recurrence. We review criteria for transience and weak…
The time variation of contacts in a networked system may fundamentally alter the properties of spreading processes and affect the condition for large-scale propagation, as encoded in the epidemic threshold. Despite the great interest in the…
We consider the problem of adaptive stabilization for discrete-time, multi-dimensional linear systems with bounded control input constraints and unbounded stochastic disturbances, where the parameters of the true system are unknown. To…
Perturbations in complex media, due to their own dynamical evolution or to external effects, are often seen as detrimental. Therefore, a common strategy, especially for telecommunication and imaging applications, is to limit the sensitivity…
Classical chaotic systems are distinguished by their sensitive dependence on initial conditions. The absence of this property in quantum systems has lead to a number of proposals for perturbation-based characterizations of quantum chaos,…
We characterize the reachability probabilities in stochastic directed graphs by means of reinforcement learning methods. In particular, we show that the dynamics of the transition probabilities in a stochastic digraph can be modeled via a…
It has been well known for some time that for strictly stationary Markov chains that are ``reversible'', that special symmetry provides special extra features in the mathematical theory. This paper here is primarily a purely expository…
We investigate recurrence and transience of Branching Markov Chains (BMC) in discrete time. Branching Markov Chains are clouds of particles which move (according to an irreducible underlying Markov Chain) and produce offspring…
In this paper we develop a statistical estimation technique to recover the transition kernel $P$ of a Markov chain $X=(X_m)_{m \in \mathbb N}$ in presence of censored data. We consider the situation where only a sub-sequence of $X$ is…
We study the following model of hidden Markov chain: $Y_i=X_i+\epsilon_i$, $ i=1,...,n+1$ with $(X_i)$ a real-valued positive recurrent and stationary Markov chain and $(\epsilon_i)_{1\leq i\leq n+1}$ a noise independent of the sequence…
We present a new algorithm for the statistical model checking of Markov chains with respect to unbounded temporal properties, such as reachability and full linear temporal logic. The main idea is that we monitor each simulation run on the…
In this paper we establish a diffusion limit for a multivariate continuous time Markov chain whose components are indexed by vertices of a finite graph. The components take values in a common finite set of non-negative integers and evolve…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…