Related papers: On Occupation Time for On-Off Processes with Multi…
We revisit the work of Dhar and Majumdar [Phys. Rev. E 59, 6413 (1999)] on the limiting distribution of the temporal mean $M_{t}=t^{-1}\int_{0}^{t}du \sign y_{u}$, for a Gaussian Markovian process $y_{t}$ depending on a parameter $\alpha $,…
Dynamic Programming suffers from the curse of dimensionality due to large state and action spaces, a challenge further compounded by uncertainties in the environment. To mitigate these issue, we explore an off-policy based Temporal…
In this article, we investigate the local behaviors of the occupation measure $\mu$ of a class of real-valued Markov processes M, defined via a SDE. This (random) measure describes the time spent in each set A $\subset$ R by the sample…
Motivated by the many real-world applications of reinforcement learning (RL) that require safe-policy iterations, we consider the problem of off-policy evaluation (OPE) -- the problem of evaluating a new policy using the historical data…
The models studied in the steady state involve two queues which are served either by a single server whose speed depends on the number of jobs present, or by several parallel servers whose number may be controlled dynamically. Job service…
We consider the connections among `clumped' residual allocation models (RAMs), a general class of stick-breaking processes including Dirichlet processes, and the occupation laws of certain discrete space time-inhomogeneous Markov chains…
Motivated by applications in telecommunications, computer scienceand physics, we consider a discrete-time Markov process withrestart. At each step the process eitherwith a positive probability restarts from a given distribution, orwith the…
In this study, a new extension of the Markov Renewal theory is introduced by allowing time to evolve in multiple dimensions. The resulting chains are referred to as multi-time Markov Renewal chains and since this extension is new, the state…
We consider discrete-time Markov chains and study large deviations of the pair empirical occupation measure, which is useful to compute fluctuations of pure-additive and jump-type observables. We provide an exact expression for the…
Multi-state models are commonly used for intermittent observations of a state over time, but these are generally based on the Markov assumption, that transition rates are independent of the time spent in current and previous states. In a…
This article studies the expected occupancy probabilities on an alphabet. Unlike the standard situation, where observations are assumed to be independent and identically distributed (iid), we assume that they follow a regime switching…
A central object of study in Reinforcement Learning (RL) is the Markovian policy, in which an agent's actions are chosen from a memoryless probability distribution, conditioned only on its current state. The family of Markovian policies is…
In this paper, we reveal a general relationship between model simplification and irreversibility based on the model of continuous-time Markov chains with time-scale separation. According to the topological structure of the fast process, we…
We investigate the distribution of occupation times for a particle undergoing a random walk among random energy traps and in the presence of a deterministic potential field $U^{{\rm det}}(x)$. When the distribution of energy traps is…
In this paper, we identify Laplace transforms of occupation times of intervals until first passage times for spectrally negative L\'evy processes. New analytical identities for scale functions are derived and therefore the results are…
We are interested in the connection between a metastable continuous state space Markov process (satisfying e.g. the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the…
In this study, two-state Markov switching multinomial logit models are proposed for statistical modeling of accident injury severities. These models assume Markov switching in time between two unobserved states of roadway safety. The states…
Many economic models feature monotone Markov dynamics on state spaces that may be noncompact. Establishing existence, uniqueness, and stability of stationary distributions in such settings has required a patchwork of sufficient conditions,…
Offline reinforcement learning seeks to utilize offline (observational) data to guide the learning of (causal) sequential decision making strategies. The hope is that offline reinforcement learning coupled with function approximation…
Motivated by the recent results of Nualart and Xu \cite{Nualart} concerning limits laws for occupation times of one dimensional symmetric stable processes, this paper proves a decomposition for functionals of one dimensional symmetric…