Related papers: Limit Behaviour of Upper and Lower Expected Time A…
We propose a model of random walks on weighted graphs where the weights are interval valued, and connect it to reversible imprecise Markov chains. While the theory of imprecise Markov chains is now well established, this is a first attempt…
In this paper, we study the dynamics of epidemic processes taking place in temporal and adaptive networks. Building on the activity-driven network model, we propose an adaptive model of epidemic processes, where the network topology…
This work is devoted to examining qualitative properties of dynamic systems, in particular, limit cycles of stochastic differential equations with both rapid switching and small diffusion. The systems are featured by multi-scale…
Quantitative understanding of stochastic dynamics in limit order price changes is essential for execution strategy design. We analyze intraday transition dynamics of ask and bid orders across market capitalization tiers using high-frequency…
A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be…
We consider a system of $N$ particles interacting through their empirical distribution on a finite state space in continuous time. In the formal limit as $N\to\infty$, the system takes the form of a nonlinear (McKean--Vlasov) Markov chain.…
We consider Markov chains on general state spaces in stationary random environment which are defined by a random mapping that is contractive up to a bounded perturbation. We prove their convergence to a limiting law, providing convergence…
In this paper, selection of an active sensor subset for tracking a discrete time, finite state Markov chain having an unknown transition probability matrix (TPM) is considered. A total of N sensors are available for making observations of…
We find upper bounds for the probability of underestimation and overestimation errors in penalized likelihood context tree estimation. The bounds are explicit and applies to processes of not necessarily finite memory. We allow for general…
We obtain non-Gaussian limit laws for one-dimensional random walk in a random environment assuming that the environment is a function of a stationary Markov process. This is an extension of the work of Kesten, M. Kozlov and Spitzer for…
In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior…
Excited random walks (ERWs) are a self-interacting non-Markovian random walk in which the future behavior of the walk is influenced by the number of times the walk has previously visited its current site. We study the speed of the walk,…
We study stochastic approximation procedures for approximately solving a $d$-dimensional linear fixed point equation based on observing a trajectory of length $n$ from an ergodic Markov chain. We first exhibit a non-asymptotic bound of the…
We study long time behavior of a discrete time weakly interacting particle system, and the corresponding nonlinear Markov process in $\mathbb{R}^d$, described in terms of a general stochastic evolution equation. In a setting where the state…
The problem of constrained Markov decision process is considered. An agent aims to maximize the expected accumulated discounted reward subject to multiple constraints on its costs (the number of constraints is relatively small). A new dual…
The paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term 'risk-sensitive' refers here to the use of the Optimized Certainty Equivalent as a means to measure expectation and risk.…
We propose a discrete time discrete space Markov chain approximation with a Brownian bridge correction for computing curvilinear boundary crossing probabilities of a general diffusion process on a finite time interval. For broad classes of…
Consider longitudinal networks whose edges turn on and off according to a discrete-time Markov chain with exponential-family transition probabilities. We characterize when their joint distributions are also exponential families with the…
We consider a risk-sensitive continuous-time Markov decision process over a finite time duration. Under the conditions that can be satisfied by unbounded transition and cost rates, we show the existence of an optimal policy, and the…
Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…