Related papers: Palm problems arising in BAR approach and its appl…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
Continuous time random walks (CTRWs) are versatile models for anomalous diffusion processes that have found widespread application in the quantitative sciences. Their scaling limits are typically non-Markovian, and the computation of their…
We discuss certain renormalised first passage bridges of self-similar processes. These processes generalise the Brownian co-ascent, a term recently introduced by H. Panzo (S\'eminaire de Probabilit\'es L, 2019). Our main result states that…
We give stationary estimates for the derivative of the expectation of a non-smooth function of bounded variation f of the workload in a G/G/1/$\infty$ queue, with respect to a parameter influencing the distribu- tion of the input process.…
Motivated by a general principle governing regulation mechanisms in biological cells, we investigate a general interaction scheme between different populations of particles and specific particles, referred to as agents. Assuming that each…
We study a class of distribution-steering problems from a variational point of view. Under some differentiability assumptions, we derive necessary conditions for optimal Markov policies in the spirit of the Lagrange multiplier approach. We…
This paper considers differential problems with random switching, with specific applications to the motion of cells and centrally coordinated motion. Starting with a differential-equation model of cell motion that was proposed previously,…
Flowgraph models provide an alternative approach in modeling a multi-state stochastic process. One of the most widely used stochastic processes that have many real-world applications especially in actuarial models is the Markov jump process…
We study irreducible time-homogenous Markov chains with finite state space in discrete time. We obtain results on the sensitivity of the stationary distribution and other statistical quantities with respect to perturbations of the…
Motivated by the operational problems in click and collect systems, such as curbside pickup programs, we study a joint admission control and capacity allocation problem. We consider a system where arriving customers have preferred service…
We study the Patient Assignment Scheduling (PAS) problem in a random environment that arises in the management of patient flow in the hospital systems, due to the stochastic nature of the arrivals as well as the Length of Stay distribution.…
The problem of reservation in a large distributed system is analyzed via a new mathematical model. A typical application is a station-based car-sharing system which can be described as a closed stochastic network where the nodes are the…
A comparison theorem for state-dependent regime-switching diffusion processes is established, which enables us to control pathwisely the evolution of the state-dependent switching component simply by Markov chains. Moreover, a sharp…
Palm distributions are critical in the study of point processes. In the present paper we focus on a point process $\Phi$ defined as the superposition, i.e., sum, of two independent point processes, say $\Phi = \Phi_1 + \Phi_2$, and we…
Maximum entropy (maxEnt) inference of state probabilities using state-dependent constraints is popular in the study of complex systems. In stochastic dynamical systems, the effect of state space topology and path-dependent constraints on…
Sampling from the conditional (or posterior) probability distribution of the latent states of a Hidden Markov Model, given the realization of the observed process, is a non-trivial problem in the context of Markov Chain Monte Carlo. To do…
This article describes an accurate procedure for computing the mean first passage times of a finite irreducible Markov chain and a Markov renewal process. The method is a refinement to the Kohlas, Zeit fur Oper Res, 30,197-207, (1986)…
This paper introduces a general model of a single-lane roundabout, represented as a circular lattice that consists of $L$ cells, with Markovian traffic dynamics. Vehicles enter the roundabout via on-ramp queues that have stochastic arrival…
This paper presents a new condition for the existence of optimal stationary policies in average-cost continuous-time Markov decision processes with unbounded cost and transition rates, arising from controlled queueing systems. This…
This note presents conjectures on polynomial/algebraic/sub-exponential convergence of transition probabilities for $\lambda$-null recurrent and $\lambda$-transient Markov chains in continuous time. The only known positive examples are in…