Related papers: Refined large deviations asymptotics for Markov-mo…
We propose a computational method for large deviation statistics of time-averaged quantities in general Markov processes. In our proposed method, we repeat a response measurement against external forces, where the forces are determined by…
Motivated by a real failure dataset in a two-dimensional context, this paper presents an extension of the Markov modulated Poisson process (MMPP) to two dimensions. The one-dimensional MMPP has been proposed for the modeling of dependent…
Motivated by applications in large-scale and multi-agent reinforcement learning, we study the non-asymptotic performance of stochastic approximation (SA) schemes with delayed updates under Markovian sampling. While the effect of delays has…
We analyze a model of relay-augmented cellular wireless networks. The network users, who move according to a general mobility model based on a Poisson point process of continuous trajectories in a bounded domain, try to communicate with a…
We investigate the parameter recovery of Markov-switching ordinary differential processes from discrete observations, where the differential equations are nonlinear additive models. This framework has been widely applied in biological…
Using a result of Blanchet and Wallwater (2015: Exact sampling of stationary and time-reversed queues. ACM TOMACS, 25, 26) for exactly simulating the maximum of a negative drift random walk queue endowed with independent and identically…
The majority of existing probabilistic model checking case studies are based on well understood theoretical models and distributions. However, real-life probabilistic systems usually involve distribution parameters whose values are obtained…
This paper studies a scheduling control problem for a single-server multiclass queueing network in heavy traffic, operating in a changing environment. The changing environment is modeled as a finite state Markov process that modulates the…
We consider a Markovian load balancing model on a fully-connected network, where calls have Poisson arrivals and exponential durations. The endpoints of each call are uniform over all the links of the network. Each call is routed either…
We study the asymptotics of large, moderate and normal deviations for the connected components of the sparse random graph by the method of stochastic processes. We obtain the logarithmic asymptotics of large deviations of the joint…
This work proposes a compositional data-driven technique for the construction of finite Markov decision processes (MDPs) for large-scale stochastic networks with unknown mathematical models. Our proposed framework leverages dissipativity…
In this paper we study limit behavior for a Markov-modulated (MM) binomial counting process, also called a binomial counting process under regime switching. Such a process naturally appears in the context of credit risk when multiple…
This paper is devoted to the problem of sample path large deviations for the Markov processes on R_+^N having a constant but different transition mechanism on each boundary set {x:x_i=0 for i\notin\Lambda, x_i>0 for i\in\Lambda}. The global…
Suppose $ E$ is a space with a null-recurrent Markov kernel $ P$. Furthermore, suppose there are infinite particles with variable weights on $ E$ performing a random walk following $ P$. Let $ X_{t}$ be a weighted functional of the position…
We present existence and discrete-time approximation results on optimal control policies for continuous-time stochastic control problems under a variety of information structures. These include fully observed models, partially observed…
Rare trajectories of stochastic systems are important to understand -- because of their potential impact. However, their properties are by definition difficult to sample directly. Population dynamics provides a numerical tool allowing their…
This paper introduces a periodic multivariate Poisson autoregression with potentially infinite memory, with a special focus on the network setting. Using contraction techniques, we study the stability of such a process and provide upper…
We prove a complete class theorem that characterizes \emph{all} stationary time reversible Markov processes whose finite dimensional marginal distributions (of all orders) are infinitely divisible. Aside from two degenerate cases (iid and…
In this paper, a simple transient Markov process with an absorbing point is used to investigate the qualitative behavior of a large scale storage network of non reliable file servers where files can be duplicated. When the size of the…
The theory of stochastic approximations form the theoretical foundation for studying convergence properties of many popular recursive learning algorithms in statistics, machine learning and statistical physics. Large deviations for…