Related papers: A State-Dependent Polling Model with Markovian Rou…
We consider stochastic discrete event dynamic systems that have time evolution represented with two-dimensional state vectors through a vector equation that is linear in terms of an idempotent semiring. The state transitions are governed by…
We investigate an M/M/1 queue operating in two switching environments, where the switch is governed by a two-state time-homogeneous Markov chain. This model allows to describe a system that is subject to regular operating phases alternating…
A many-server queueing system is considered in which customers arrive according to a renewal process and have service and patience times that are drawn from two independent sequences of independent, identically distributed random variables.…
We investigate the classical active pure exploration problem in Markov Decision Processes, where the agent sequentially selects actions and, from the resulting system trajectory, aims at identifying the best policy as fast as possible. We…
We provide a probabilistic analysis of the banker algorithm when transition probabilities may depend on time and space. The transition probabilities evolve, as time goes by, along the trajectory of an ergodic Markovian environment, whereas…
Robust estimates for the performance of complicated queueing networks can be obtained by showing that the number of jobs in the network is stochastically comparable to a simpler, analytically tractable reference network. Classical coupling…
Tandem queueing networks are widely used to model systems where services are provided in sequential stages. In this study, we assume that each station in the tandem system operates under a general renewal process. Additionally, we assume…
We compare different selection criteria to choose the number of latent states of a multivariate latent Markov model for longitudinal data. This model is based on an underlying Markov chain to represent the evolution of a latent…
Queueing networks are notoriously difficult to analyze sans both Markovian and stationarity assumptions. Much of the theoretical contribution towards performance analysis of time-inhomogeneous single class queueing networks has focused on…
Optimal Markov Decision Process policies for problems with finite state and action space are identified through a partial ordering by comparing the value function across states. This is referred to as state-based optimality. This paper…
The voter model with memory-dependent dynamics is theoretically and numerically studied at the mean-field level. The `internal age', or time an individual spends holding the same state, is added to the set of binary states of the…
We have developed a steady state theory of complex transport networks used to model the flow of commodity, information, viruses, opinions, or traffic. Our approach is based on the use of the Markov chains defined on the graph…
We provide a general macrostatistical formulation of nonequilibrium steady states of reservoir driven quantum systems. This formulation is centred on the large scale properties of the locally conserved hydrodynamical observables, and our…
High-dimensional dynamical systems projected onto a reduced-order model cease to be deterministic and are best described by probability distributions in state space. Their equations of motion map onto an evolution operator with a…
We consider Markov processes, which describe e.g. queueing network processes, in a random environment which influences the network by determining random breakdown of nodes, and the necessity of repair thereafter. Starting from an explicit…
Since the first paper on polling systems, written by Mack in 1957, a huge number of papers on this topic has been written. A typical polling system consists of a number of queues, attended by a single server. In several surveys, the most…
We study a stochastic spatial epidemic model where the $N$ individuals carry two features: a position and an infection state, interact and move in $\R^d$. In this Markovian model, the evolution of the infection states are described with the…
Understanding and predicting how complex systems respond to external perturbations is a central challenge in nonequilibrium statistical physics. Here we consider continuous-time Markov networks, which we subject to perturbations along a…
Stochastic systems characterised by a random driving in a form of the general stable noise are considered. The particle experiences long rests due to the traps the density of which is position-dependent and obeys a power-law form attributed…
This paper introduces a general model of a single-lane roundabout, represented as a circular lattice that consists of $L$ cells, with Markovian traffic dynamics. Vehicles enter the roundabout via on-ramp queues that have stochastic arrival…