Related papers: On Markov-dependent reflected autoregressive proce…

For Markov jump processes on irreducible networks with finite number of sites, we derive a general and explicit expression of the squared coefficient of variation for the net number of transitions from one site to a connected site in a…

Arrivals in queueing systems are typically assumed to be independent and exponentially distributed. Our analysis of an online bookshop, however, shows that there is an autocorrelation structure present. First, we adjust the inter-arrival…

Knowledge on the dynamics of standard epidemic models and their variants over complex networks has been well-established primarily in the stationary regime, with relatively little light shed on their transient behavior. In this paper, we…

We consider a system of N queues with decentralized load balancing such as power-of-D strategies(where D may depend on N) and generic scheduling disciplines. To measure the dependence of the queues, we use the clan of ancestors, a technique…

This paper considers the queueing performance of a system that transmits coded data over a time-varying erasure channel. In our model, the queue length and channel state together form a Markov chain that depends on the system parameters.…

In this paper, the recurrent events that can occur more than one over the follow-up time have been modeled by phase-type distributions. We use the finite-state continuous-time Markov process with multi states for patients with recurrent…

A study of time homogeneous, real valued Markov processes with a special property and a non-atomic initial distribution is provided. The new notion of a function of evolution of distribution which determines the dependency between one…

This paper studies some temporal dependence properties and addresses the issue of parametric estimation for a class of state-dependent autoregressive models for nonlinear time series in which we assume a stochastic autoregressive…

We present for the first time an asymptotic convergence analysis of two time-scale stochastic approximation driven by "controlled" Markov noise. In particular, the faster and slower recursions have non-additive controlled Markov noise…

We consider a stationary Markov process that models certain queues with a bulk service of a fixed number $m$ of admitted customers. We find an integral expression of its transition probability function in terms of certain multi-orthogonal…

For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one…

Linear fractional Galton-Watson branching processes in i.i.d.~random environment are, on the quenched level, intimately connected to random difference equations by the evolution of the random parameters of their linear fractional marginals.…

In many important real-world queueing settings, arrival and service rates fluctuate over time. We consider the MAMS system, where the arrival and service rates each vary according to an arbitrary finite-state Markov chain, allowing…

We give Hoeffding and Bernstein-type concentration inequalities for the largest eigenvalue of sums of random matrices arising from a Markov chain. We consider time-dependent matrix-valued functions on a general state space, generalizing…

We consider a Markovian load balancing model on a fully-connected network, where calls have Poisson arrivals and exponential durations. The endpoints of each call are uniform over all the links of the network. Each call is routed either…

In this paper, we develop a more general framework of block-structured Markov processes in the queueing study of blockchain systems, which can provide analysis both for the stationary performance measures and for the sojourn times of any…

We consider a tandem queue with coupled processors, which is subject to global breakdowns. When the network is in the operating mode and both queues are non empty, the total service capacity is shared among the stations according to fixed…

We review the theory of regenerative processes, which are processes that can be intuitively seen as comprising of i.i.d.\ cycles. Although we focus on the classical definition, we present a more general definition that allows for some form…

Motivated by applications in movement ecology, in this paper I propose a new class of integrated continuous-time hidden Markov models in which each observation depends on the underlying state of the process over the whole interval since the…

Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…