Related papers: On Occupation Time for On-Off Processes with Multi…
We study the classical occupancy problem from the viewpoint of its embedding Markov chain. We derive new expressions for the probability mass function and (complementary) distribution function in generalized form. Furthermore, we derive a…
Renewal processes with heavy-tailed power law distributed sojourn times are commonly encountered in physical modelling and so typical fluctuations of observables of interest have been investigated in detail. To describe rare events the rate…
The mean time taken by an irreducible Markov chain on a finite state space to hit a target chosen at random according to the stationary distribution does not depend on the initial state of the chain. This mean time is known as Kemeny's…
This paper solves exit problems for spectrally negative Markov additive processes and their reflections. A so-called scale matrix, which is a generalization of the scale function of a spectrally negative \levy process, plays a central role…
This paper is concerned with combined inference for point processes on the real line observed in a broken interval. For such processes, the classic history-based approach cannot be used. Instead, we adapt tools from sequential spatial point…
This article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30,197-207, (1986)…
We study a class of systems termed Markov Machines (MM) which process job requests with exponential service times. Assuming a Poison job arrival process, these MMs oscillate between two states, free and busy. We consider the problem of…
A semi-Markov process method for obtaining general counting statistics for open quantum systems is extended to the scenario of resetting. The simultaneous presence of random resets and wave function collapses means that the quantum jump…
The capability to widely sample the state and action spaces is a key ingredient toward building effective reinforcement learning algorithms. The variational optimization principles exposed in this paper emphasize the importance of an…
We study offline reinforcement learning problems with a long-run average reward objective. The state-action pairs generated by any fixed behavioral policy thus follow a Markov chain, and the {\em empirical} state-action-next-state…
Predicting future operational risk losses gives rise to a significant challenge due to the heterogeneous and time-dependent structures present in real-world data. Furthermore, stress test exercises require examining the relationship with…
We study time-changed Markov processes to speed up the convergence of Markov chain Monte Carlo (MCMC) algorithms. The time-changed process is defined by adjusting the speed of time of a base process via a user-chosen, state-dependent…
Two complex multi-state systems subject to multiple events are built in an algorithmic and computational way by considering phase-type distributions and Markovian arrival processes with marked arrivals. The internal performance of the…
We analyze the conditions in which offloading computation reduces completion time. We extend the existing literature by deriving an inequality (Eq. 4) that relates computation offloading system parameters to the bits per instruction ratio…
The orientational memory of particles can serve as an effective measure of diffusivity, spreading, and search efficiency in complex stochastic processes. We develop a theoretical framework to describe the decay of directional correlations…
Conditioning Markov processes to avoid a set is a classical problem that has been studied in many settings. In the present article we study the question if a Levy process can be conditioned to avoid an interval and, if so, the path behavior…
The Oslo sandpile model, or if one wants to be precise, ricepile model, is a cellular automaton designed to model experiments on granular piles displaying self-organized criticality. We present an analytic treatment that allows the…
We consider a Markov additive process with a finite phase space and study its path decompositions at the times of extrema, first passage and last exit. For these three families of times we establish splitting conditional on the phase, and…
We present a time change construction of affine processes with state-space $\mathbb{R}_+^m\times \mathbb{R}^n$. These processes were systematically studied in (Duffie, Filipovi\'c and Schachermayer, 2003) since they contain interesting…
This paper considers a distributed decision-making approach for manufacturing task assignment and condition-based machine health maintenance. Our approach considers information sharing between the task assignment and health management…