Related papers: On the occupancy problem for a regime switching mo…
We investigate absorption, i.e., almost sure convergence to an absorbing state, in time-varying (non-homogeneous) discrete-time Markov chains with finite state space. We consider systems that can switch among a finite set of transition…
Dynamic and evolving operational and economic environments present significant challenges for decision-making. We explore a simulation optimization problem characterized by non-stationary input distributions with regime-switching dynamics…
Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past…
In this paper, we discuss long-time behavior of sample paths for a wide range of regime-switching diffusions. Firstly, almost sure asymptotic stability is concerned (i) for regime-switching diffusions with finite state spaces by the…
We consider finite-state time-nonhomogeneous Markov chains where the probability of moving from state $i$ to state $j\neq i$ at time $n$ is $G(i,j)/n^\zeta$ for a ``generator'' matrix $G$ and strength parameter $\zeta>0$. In these chains,…
We consider a particle moving in a one dimensional potential which has a symmetric deterministic part and a quenched random part. We study analytically the probability distributions of the local time (spent by the particle around its mean…
We study a multi-armed bandit problem where the rewards exhibit regime switching. Specifically, the distributions of the random rewards generated from all arms are modulated by a common underlying state modeled as a finite-state Markov…
A semi-Markov process is one that changes states in accordance with a Markov chain but takes a random amount of time between changes. We consider the generalisation to semi-Markov processes of the classical Lamperti law for the occupation…
In this paper, we study an optimal stopping problem in the presence of model uncertainty and regime switching. The max-min formulation for robust control and the dynamic programming approach are adopted to establish a general theoretical…
We consider a class of discrete time Markov chains with state space [0,1] and the following dynamics. At each time step, first the direction of the next transition is chosen at random with probability depending on the current location. Then…
In this paper, we propose the first exact Markov model for connection blocking analysis in elastic optical networks, based on the occupancy status of spectrum slices on all links due to arrivals and departures of various classes of…
In multi-period stochastic optimization problems, the future optimal decision is a random variable whose distribution depends on the parameters of the optimization problem. We analyze how the expected value of this random variable changes…
We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the…
We consider the problem of maximizing the expected average reward obtained over an infinite time horizon by $n$ weakly coupled Markov decision processes. Our setup is a substantial generalization of the multi-armed restless bandit problem…
This paper is concerned with the stability analysis of continuous-time switched systems with a random switching signal. The switching signal manifests its characteristics with that the dwell time in each subsystem consists of a fixed part…
We investigate an optimal stopping problem for the expected value of a discounted payoff on a regime-switching geometric Brownian motion under two constraints on the possible stopping times: only at exogenous random times and only during a…
We investigate an M/M/1 queue operating in two switching environments, where the switch is governed by a two-state time-homogeneous Markov chain. This model allows to describe a system that is subject to regular operating phases alternating…
We explore the concept of a consistent exchangeable survival process - a joint distribution of survival times in which the risk set evolves as a continuous-time Markov process with homogeneous transition rates. We show a correspondence with…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…
We consider a finite number of $N$ statistically equal agents, each moving on a finite set of states according to a continuous-time Markov Decision Process (MDP). Transition intensities of the agents and generated rewards depend not only on…