Related papers: Reward Processes and Performance Simulation in Sup…
We consider a tandem queue with coupled processors, which is subject to global breakdowns. When the network is in the operating mode and both queues are non empty, the total service capacity is shared among the stations according to fixed…
Motivated by the growing interest in today's massive parallel computing capabilities we analyze a queueing network with many servers in parallel to which jobs arrive a according to a Poisson process. Each job, upon arrival, is split into…
A state-dependent 1-limited polling model with N queues is analyzed. The routing strategy generalizes the classical Markovian polling model, in the sense that two routing matrices are involved, the choice being made according to the state…
The problem of appropriately matching items subject to compatibility constraints arises in a number of important applications. While most of the literature on matching theory focuses on a static setting with a fixed number of items, several…
We develop a new algorithm for the estimation of rare event probabilities associated with the steady-state of a Markov stochastic process with continuous state space $\mathbb R^d$ and discrete time steps (i.e. a discrete-time $\mathbb…
We consider a time-slotted job-assignment system with a central server, N users and a machine which changes its state according to a Markov chain (hence called a Markov machine). The users submit their jobs to the central server according…
In the paper we study continuous time controlled Markov processes using discrete time controlled Markov processes. We consider long run functionals: average reward per unit time or long run risk sensitive functional. We also investigate…
We consider a finite number of $N$ statistically equal agents, each moving on a finite set of states according to a continuous-time Markov Decision Process (MDP). Transition intensities of the agents and generated rewards depend not only on…
We consider a scenario where multiple event-based systems use a wireless network to communicate with their respective controllers. These systems use a contention resolution mechanism (CRM) to arbitrate access to the network. We present a…
The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…
Service system performance depends on how participants respond to design choices, but modeling these responses is hard due to the complexity of human behavior. We introduce an LLM-powered multi-agent simulation (LLM-MAS) framework for…
We consider a queueing system consisting of two non-identical exponential servers, where each server has its own dedicated queue and serves the customers in that queue FCFS. Customers arrive according to a Poisson process and join the queue…
The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…
This paper studies statistical inference in a network of infinite-server queues, with the aim of estimating the underlying parameters (routing matrix, arrival rates, parameters pertaining to the service times) using observations of the…
In this paper, we provide a matrix-analytic solution for randomized load balancing models (also known as \emph{supermarket models}) with phase-type (PH) service times. Generalizing the service times to the phase-type distribution makes the…
We consider a matching system with random arrivals of items of different types. The items wait in queues -- one per each item type -- until they are "matched." Each matching requires certain quantities of items of different types; after a…
Traffic to any server is rarely constant over time. In addition, the workload brought by each service request is typically unknown in advance, and each request may bring a different workload to the server. Cha and Lee (2011) proposed a…
This paper introduces a novel stochastic control framework to enhance the capabilities of automated investment managers, or robo-advisors, by accurately inferring clients' investment preferences from past activities. Our approach leverages…
We address the problem of community detection in networks by introducing a general definition of Markov stability, based on the difference between the probability fluxes of a Markov chain on the network at different time scales. The…
Stochastic Processing Networks (SPNs) can be used to model communication networks, manufacturing systems, service systems, etc. We consider a real-time SPN where tasks generate jobs with strict deadlines according to their traffic patterns.…