Related papers: On Occupation Time for On-Off Processes with Multi…
To extend the classical concept of Markovianity to an open quantum system, different notions of the divisibility of its dynamics have been introduced. Here we analyze this issue by five complementary approaches: equations of motion,…
In this paper, we derive the joint Laplace transforms of occupation times until its last passage times as well as its positions. Motivated by Baurdoux [2], the last times before an independent exponential variable are studied. By applying…
Sticky Brownian motion on the real line can be obtained as a weak solution of a system of stochastic differential equations. We find the conditional distribution of the process given the driving Brownian motion, both at an independent…
We prove exponential decay of pair correlations for 1D stationary point processes when spacings satisfy a Markov condition, geometric ergodicity, and a condition on exponential moments. The conditions are phrased for stationary sequences of…
Offline reinforcement learning (offline RL) considers problems where learning is performed using only previously collected samples and is helpful for the settings in which collecting new data is costly or risky. In model-based offline RL,…
In this paper, we demonstrate through the use of matrix calculus a transparent analysis of fractional inhomogeneous Markov models for life insurance where transition matrices commute. The resulting formulae are intuitive matrix…
The classical and extended occupancy distributions are useful for examining the number of occupied bins in problems involving random allocation of balls to bins. We examine the extended occupancy problem by framing it as a Markov chain and…
A complex multi-state redundant system undergoing preventive maintenance and experiencing multiple events is being considered in a continuous time frame. The online unit is susceptible to various types of failures, both internal and…
Using a new approach, for spectrally negative L\'evy processes we find joint Laplace transforms involving the last exit time (from a semi-infinite interval), the value of the process at the last exit time and the associated occupation time,…
This paper presents a convex optimization approach to control the density distribution of autonomous mobile agents with two control modes: ON and OFF. The main new characteristic distinguishing this model from standard Markov decision…
We provide a general framework for dual representations of Laplace transforms of Markov processes. Such representations state that the Laplace transform of a finite-dimensional distribution of a Markov process can be expressed in terms of a…
In this short communication we study a fluid queue with a finite buffer. The performance measure we are interested in is the occupation time over a finite time period, i.e., the fraction of time the workload process is below some fixed…
This work aims to study off-policy evaluation (OPE) under scenarios where two key reinforcement learning (RL) assumptions -- temporal stationarity and individual homogeneity are both violated. To handle the ``double inhomogeneities", we…
This article is devoted to some time-changed stochastic models based on multivariate stable processes. The considered models have several advantages in comparison with classical time-changed Brownian motions - for instance, it turns out…
We establish limit theorems for re-scaled occupation time fluctuations of a sequence of branching particle systems in $\R^d$ with anisotropic space motion and weakly degenerate splitting ability. In the case of large dimensions, our limit…
There is a well-established theory linking certain semi-Markov chains and continuous-time random walks to time-fractional equations and anomalous diffusion. In this work, we go beyond the semi-Markov framework by considering some…
We consider a branching system consisting of particles moving according to a Markov family in $\Rd$ and undergoing subcritical branching with a constant rate $V>0$. New particles immigrate to the system according to homogeneous space-time…
In this paper, the recurrent events that can occur more than one over the follow-up time have been modeled by phase-type distributions. We use the finite-state continuous-time Markov process with multi states for patients with recurrent…
How does a driven system with many energy levels approach its steady state? Insights are gained by studying a system with three energy levels when the ground state is excited by a laser. The time-dependent occupation probabilities of the…
The training of autonomous agents often requires expensive and unsafe trial-and-error interactions with the environment. Nowadays several data sets containing recorded experiences of intelligent agents performing various tasks, spanning…