Related papers: Limit Behaviour of Upper and Lower Expected Time A…
In this letter we announce rigorous results that elucidate the relation between metastable states and low-lying eigenvalues in Markov chains in a much more general setting and with considerable greater precision as was so far available.…
This paper studies finite-time safety and reach-avoid verification for stochastic discrete-time dynamical systems. The aim is to ascertain lower and upper bounds of the probability that, within a predefined finite-time horizon, a system…
This article is concerned with stability analysis and stabilization of randomly switched nonlinear systems. These systems may be regarded as piecewise deterministic stochastic systems: the discrete switches are triggered by a stochastic…
A continuous-time random walk in the quarter plane with homogeneous transition rates is considered. Given a non-negative reward function on the state space, we are interested in the expected stationary performance. Since a direct derivation…
We consider a discrete-time Markov decision process with Borel state and action spaces. The performance criterion is to maximize a total expected {utility determined by unbounded return function. It is shown the existence of optimal…
The work treats systems combining slow and fast motions depending on each other where fast motions are perturbations of families of either dynamical systems or Markov processes with freezed slow variable. In the first case we consider…
The class of nonlinear Markov processes is characterized by the dependence of the current state of the process on its current distribution in addition to the dependence on the previous state. Due to this feature, these processes are…
The extremal behaviour of a Markov chain is typically characterized by its tail chain. For asymptotically dependent Markov chains existing formulations fail to capture the full evolution of the extreme event when the chain moves out of the…
Conditions on the generator of a Markov process to control the fluctuations of its bridges are found. In particular, continuous time random walks on graphs and gradient diffusions are considered. Under these conditions, a concentration of…
We apply the method of differential inequalities for the computation of upper bounds for the rate of convergence to the limiting regime for one specific class of (in)homogeneous continuous-time Markov chains. To obtain these estimates, we…
We consider a Markov chain on $\mathbb{R}^d$ with invariant measure $\mu$. We are interested in the rate of convergence of the empirical measures towards the invariant measure with respect to various dual distances, including in particular…
This paper considers stochastic-constrained stochastic optimization where the stochastic constraint is to satisfy that the expectation of a random function is below a certain threshold. In particular, we study the setting where data samples…
Continuous time financial market models are often motivated as scaling limits of discrete time models. The objective of this paper is to establish such a connection for a robust framework. More specifically, we consider discrete time models…
The limiting probability distribution is one of the key characteristics of a Markov chain since it shows its long-term behavior. In this paper, for a higher order Markov chain, we establish some properties related to its exact limiting…
We present a learning model predictive control (MPC) scheme for chance-constrained Markov jump systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are…
To avoid poor empirical performance in Metropolis-Hastings and other accept-reject-based algorithms practitioners often tune them by trial and error. Lower bounds on the convergence rate are developed in both total variation and Wasserstein…
This simple note lays out a few observations which are well known in many ways but may not have been said in quite this way before. The basic idea is that when comparing two different Markov chains it is useful to couple them is such a way…
We consider three different types of global uncertainty models for discrete-time stochastic processes: measure-theoretic upper expectations, game-theoretic upper expectations and axiomatic upper expectations. The last two are known to be…
Machine learning algorithms have been applied to predict agent behaviors in real-world dynamic systems, such as advertiser behaviors in sponsored search and worker behaviors in crowdsourcing. The behavior data in these systems are generated…
In this paper we investigate the behavior of the bridges of a Markov counting process in several directions. We first characterize convexity(concavity) in time of the mean value in terms of lower (upper) bounds on the so called…