相关论文: Hitting time for Markov decision process

Advances in mobile computing technologies have made it possible to monitor and apply data-driven interventions across complex systems in real time. Markov decision processes (MDPs) are the primary model for sequential decision problems with…

We consider a simple but important class of metastable discrete time Markov chains, which we call perturbed Markov chains. Basically, we assume that the transition matrices depend on a parameter $\varepsilon$, and converge as $\varepsilon$.…

In this paper, we consider reinforcement learning of Markov Decision Processes (MDP) with peak constraints, where an agent chooses a policy to optimize an objective and at the same time satisfy additional constraints. The agent has to take…

A determinantal point process (DPP) is a random process useful for modeling the combinatorial problem of subset selection. In particular, DPPs encourage a random subset Y to contain a diverse set of items selected from a base set Y. For…

We consider a distributionally robust Partially Observable Markov Decision Process (DR-POMDP), where the distribution of the transition-observation probabilities is unknown at the beginning of each decision period, but their realizations…

Consider a continuous time particle system $\eta^t=(\eta^t(k),k\in \mathbb{L})$, indexed by a lattice $\mathbb{L}$ which will be either $\mathbb{Z}$, $\mathbb{Z}/n\mathbb{Z}$, a segment $\{1,\cdots, n\}$, or $\mathbb{Z}^d$, and taking its…

It is well known that the distributions of hitting times in Markov chains are quite irregular, unless the limit as time tends to infinity is considered. We show that nevertheless for a typical finite irreducible Markov chain and for…

We study a model of a polling system, that is, a collection of $d$ queues with a single server that switches from queue to queue. The service time distribution and arrival rates change randomly every time a queue is emptied. This model is…

This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be…

This paper studies optimal market making for large-tick assets in the presence of latency. We consider a random walk model for the asset price, and formulate the market maker's optimization problem using Markov Decision Processes (MDP). We…

Our work addresses a fundamental problem in the context of counterfactual inference for Markov Decision Processes (MDPs). Given an MDP path $\tau$, this kind of inference allows us to derive counterfactual paths $\tau'$ describing what-if…

We consider a Markov jump process on a general state space to which we apply a time-dependent weak perturbation over a finite time interval. By martingale-based stochastic calculus, under a suitable exponential moment bound for the…

In this paper, we study Markov Decision Processes (MDPs) with self-triggered strategies, where the idea of self-triggered control is extended to more generic MDP models. This extension broadens the application of self-triggering policies to…

The cover time of a Markov chain on a finite state space is the expected time until all states are visited. We show that if the cover time of a discrete-time Markov chain with rational transitions probabilities is bounded, then it is a…

Robust Markov decision processes (MDPs) aim to handle changing or partially known system dynamics. To solve them, one typically resorts to robust optimization methods. However, this significantly increases computational complexity and…

In this paper, we focus on the hitting times of a stochastic epidemic model presented by \cite{Gray}. Under the help of the auxiliary stopping times, we investigate the asymptotic limits of the hitting times by the variations of calculus…

We study a class of dissipative PDE's perturbed by a bounded random kick force. It is assumed that the random force is non-degenerate, so that the Markov process obtained by the restriction of solutions to integer times has a unique…

Hitting times for discrete quantum walks on graphs give an average time before the walk reaches an ending condition. To be analogous to the hitting time for a classical walk, the quantum hitting time must involve repeated measurements as…

Markov decision processes (MDP) are finite-state systems with both strategic and probabilistic choices. After fixing a strategy, an MDP produces a sequence of probability distributions over states. The sequence is eventually synchronizing…

In this work we focus on the notion of quantum hitting time for discrete-time Szegedy quantum walks, compared to its classical counterpart. Under suitable hypotheses, quantum hitting time is known to be of the order of the square root of…