Related papers: On the occupancy problem for a regime switching mo…
We apply Doeblin's ergodicity coefficient as a computational tool to approximate the occupancy distribution of a set of states in a homogeneous but possibly non-stationary finite Markov chain. Our approximation is based on new properties…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
In this work we introduce a new and richer class of finite order Markov chain models and address the following model selection problem: find the Markov model with the minimal set of parameters (minimal Markov model) which is necessary to…
This paper proposes a new type of recurrence where we divide the Markov chains into intervals that start when the chain enters into a subset A, then sample another subset B far away from A and end when the chain again return to A. The…
A random sequence having two segments being the homogeneous Markov processes is registered. Each segment has his own transition probability law and the length of the segment is unknown and random. The transition probabilities of each…
We study random walk on complex networks with transition probabilities which depend on the current and previously visited nodes. By using an absorbing Markov chain we derive an exact expression for the mean first passage time between pairs…
The insurance model when the amount of claims depends on the state of the insured person (healthy, ill, or dead) and claims are connected in a Markov chain is investigated. The signed compound Poisson approximation is applied to the…
We present a model for the evolution of networks of occupied sites on undirected regular graphs. At every iteration step in a parallel update I randomly chosen empty sites are occupied and occupied sites having degree outside of a given…
We investigate the stability problem for discrete-time stochastic switched linear systems under the specific scenarios where information about the switching patterns and the probability of switches are not available. Our analysis focuses on…
This article is concerned with stability analysis and stabilization of randomly switched systems under a class of switching signals. The switching signal is modeled as a jump stochastic (not necessarily Markovian) process independent of the…
In this paper, we investigated the effect on extreme of random replacing for a stationary sequence satisfying a type of long dependent condition and a local dependent condition, and derived the joint asymptotic distribution of maximum from…
This article is concerned with stability analysis and stabilization of randomly switched nonlinear systems. These systems may be regarded as piecewise deterministic stochastic systems: the discrete switches are triggered by a stochastic…
This paper presents a new condition for the existence of optimal stationary policies in average-cost continuous-time Markov decision processes with unbounded cost and transition rates, arising from controlled queueing systems. This…
Estimating the entropy based on data is one of the prototypical problems in distribution property testing and estimation. For estimating the Shannon entropy of a distribution on $S$ elements with independent samples, [Paninski2004] showed…
We consider a type of optimal switching problems with non-uniform execution delays and ramping. Such problems frequently occur in the operation of economical and engineering systems. We first provide a solution to the problem by applying a…
In this paper, we consider an optimal control problem for a prescription opioid epidemic model that describes the interaction between the regular prescription or addictive use of opioid drugs, and the process of rehabilitation and that of…
We extend stochastic network optimization theory to treat networks with arbitrary sample paths for arrivals, channels, and mobility. The network can experience unexpected link or node failures, traffic bursts, and topology changes, and…
We study one-sided and $\alpha$-correct sequential hypothesis testing for data generated by an ergodic Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the…
We propose experimentally feasible ways to probe universal features of absorbing phase transitions from two different approaches, both based on numerical validations. On one hand, we numerically study a probability distribution of…
We investigate the scheduling of a common resource between several concurrent users when the feasible transmission rate of each user varies randomly over time. Time is slotted and users arrive and depart upon service completion. This may…