Related papers: Reward Processes and Performance Simulation in Sup…
Process Reward Modeling (PRM) is critical for complex reasoning and decision-making tasks where the accuracy of intermediate steps significantly influences the overall outcome. Existing PRM approaches, primarily framed as classification…
In this paper, we provide a novel matrix-analytic approach for studying doubly exponential solutions of randomized load balancing models (also known as supermarket models) with Markovian arrival processes (MAPs) and phase-type (PH) service…
Queueing networks and Markov chains are widely used for conducting performance and reliability studies. In this paper we describe the queueing package, a free software package for queueing networks and Markov chain analysis for GNU Octave.…
Reward models play a critical role in guiding large language models toward outputs that align with human expectations. However, an open challenge remains in effectively utilizing test-time compute to enhance reward model performance. In…
The use of stochastic models, in effect piecewise deterministic Markov processes (PDMP), has become increasingly popular especially for the modeling of chemical reactions and cell biophysics. Yet, exact simulation methods, for the…
The success of reinforcement learning in typical settings is predicated on Markovian assumptions on the reward signal by which an agent learns optimal policies. In recent years, the use of reward machines has relaxed this assumption by…
Imagine, you enter a grocery store to buy food. How many peopledo you overlap with in this store? How much time do you overlap witheach person in the store? In this paper, we answer these questions bystudying the overlap times between…
This paper proposes a stochastic framework to evaluate the performance of public transit systems under short random service suspensions. We aim to derive closed-form formulations of the mean and variance of the queue length and waiting…
Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…
Same-day delivery for e-commerce has become a popular service. Companies usually offer several time delivery options with the earliest one being next hour delivery. Due to tight delivery deadlines and thin margins, companies often find it…
In this paper we consider a single-server, cyclic polling system with switch-over times. A distinguishing feature of the model is that the rates of the Poisson arrival processes at the various queues depend on the server location. For this…
Markov decision processes (MDPs) are standard models for probabilistic systems with non-deterministic behaviours. Mean payoff (or long-run average reward) provides a mathematically elegant formalism to express performance related…
Motivated by timeouts in Internet services, we consider networks of infinite server queues in which routing decisions are based on deadlines. Specifically, at each node in the network, the total service time equals the minimum of several…
Modern distributed systems include a class of applications in which non-functional requirements are important. In particular, these applications include multimedia facilities where real time constraints are crucial to their correct…
Stochastic processes with long memories, known as long memory processes, are ubiquitous in various science and engineering problems. Superposing Markovian stochastic processes generates a non-Markovian long memory process serving as…
Marcel F. Neuts opened a key door in numerical computation of stochastic models by means of phase-type (PH) distributions and Markovian arrival processes (MAPs). To celebrate his 75th birthday, this paper reports a more general framework of…
The dynamics in games involving multiple players, who adaptively learn from their past experience, is not yet well understood. We analyzed a class of stochastic games with Markov strategies in which players choose their actions…
This paper studies a scheduling control problem for a single-server multiclass queueing network in heavy traffic, operating in a changing environment. The changing environment is modeled as a finite state Markov process that modulates the…
Quantum repeater chains will form the backbone of future quantum networks that distribute entanglement between network nodes. Therefore, it is important to understand the entanglement distribution performance of quantum repeater chains,…
Multiclass open queueing networks find wide applications in communication, computer and fabrication networks. Often one is interested in steady-state performance measures associated with these networks. Conceptually, under mild conditions,…