Related papers: Hitting time, access time and optimal transport on…
This note considers a variation of the full-information secretary problem where the random variables to be observed are independent and identically distributed. Consider $X_1,\dots,X_n$ to be an independent sequence of random variables, let…
In this paper, we study a continuous-time discounted jump Markov decision process with both controlled actions and observations. The observation is only available for a discrete set of time instances. At each time of observation, one has to…
We consider reversible ergodic Markov chains with finite state space, and we introduce a new notion of quasi-stationary distribution that does not require the presence of any absorbing state. In our setting, the hitting time of the…
We study fundamental graph parameters such as the Diameter and Radius in directed graphs, when distances are measured using a somewhat unorthodox but natural measure: the distance between $u$ and $v$ is the minimum of the shortest path…
We address the problem of identifying the dynamical law governing the evolution of a population of indistinguishable particles, when only aggregate distributions at successive times are observed. Assuming a Markovian evolution on a discrete…
We consider a random model for directed graphs whereby an arc is placed from one vertex to another with a prescribed probability which may vary from arc to arc. Using perturbation bounds as well as Chernoff inequalities, we show that the…
In a recent paper, Shah [arXiv:2502.03073] derived an explicit expression for the distribution of occupancy times for a two-state Markov chain, using a method based on enumerating sample paths. We consider here the more general problem of…
For a random walk on a network, the mean first-passage time from a node $i$ to another node $j$ chosen stochastically according to the equilibrium distribution of Markov chain representing the random walk is called Kemeny constant, which is…
We present the analytical and numerical results of a random walk on the family of small-world graphs. The average access time shows a crossover from the regular to random behavior with increasing distance from the starting point of the…
We characterize the extremal structure for the exact mixing time for random walks on trees $T_{n,d}$ of order $n$ with diameter $d$. Given a graph $G=(V,E)$, let $H(v,\pi)$ denote the expected length of an optimal stopping rule from vertex…
The mixer chain on a graph G is the following Markov chain. Place tiles on the vertices of G, each tile labeled by its corresponding vertex. A "mixer" moves randomly on the graph, at each step either moving to a randomly chosen neighbor, or…
This paper studies the network throughput and transport delay of a multihop wireless random access network based on a Markov renewal model of packet transportation. We show that the distribution of the source-to-destination (SD) distance…
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…
We consider irreducible reversible discrete time Markov chains on a finite state space. Mixing times and hitting times are fundamental parameters of the chain. We relate them by showing that the mixing time of the lazy chain is equivalent…
Discrete-time discrete-state finite Markov chains are versatile mathematical models for a wide range of real-life stochastic processes. One of most common tasks in studies of Markov chains is computation of the stationary distribution.…
Let $(X,\p_x)$ be a continuous time Markov chain with finite or countable state space $S$ and let $T$ be its first passage time in a subset $D$ of $S$. It is well known that if $\mu$ is a quasi-stationary distribution relatively to $T$,…
We study the complexity of the directed periodic temporal graph realization problem. This work is motivated by the design of periodic schedules in public transport with constraints on the quality of service. Namely, we require that the…
This paper considers the problem of randomized influence maximization over a Markovian graph process: given a fixed set of nodes whose connectivity graph is evolving as a Markov chain, estimate the probability distribution (over this fixed…
The trapping problem on graph (or network) as a typical focus of great interest has attracted more attention from various science fields, including applied mathematics and theoretical computer science, in the past. Here, we first study this…
We consider the problem of defining and fitting models of autoregressive time series of probability distributions on a compact interval of $\mathbb{R}$. An order-$1$ autoregressive model in this context is to be understood as a Markov…