Related papers: Stationary Markovian Arrival Processes, Results an…
This paper presents a simulation-based framework for sequential inference from partially and discretely observed point process (PP's) models with static parameters. Taking on a Bayesian perspective for the static parameters, we build upon…
We address a class of Markov jump linear systems that are characterized by the underlying Markov process being time-inhomogeneous with a priori unknown transition probabilities. Necessary and sufficient conditions for uniform stochastic…
We provide necessary and sufficient conditions for explosion and implosion of birth-and-death (non-Markov) continuous-time random walks. In other words, we obtain conditions for $\infty$ to be accessible and for it to be an entrance point.…
The stochastic shortest path problem (SSPP) asks to resolve the non-deterministic choices in a Markov decision process (MDP) such that the expected accumulated weight before reaching a target state is maximized. This paper addresses the…
Many problems of practical interest rely on Continuous-time Markov chains~(CTMCs) defined over combinatorial state spaces, rendering the computation of transition probabilities, and hence probabilistic inference, difficult or impossible…
In the development of stochastic integration and the theory of semimartingales, Markov processes have been a constant source of inspiration. Despite this historical interweaving, it turned out that semimartingales should be considered the…
Interval Markov Decision Processes (IMDPs) are finite-state uncertain Markov models, where the transition probabilities belong to intervals. Recently, there has been a surge of research on employing IMDPs as abstractions of stochastic…
The density-dependent Markov chain (DDMC) introduced in \cite{Kurtz1978} is a continuous time Markov process applied in fields such as epidemics, chemical reactions and so on. In this paper, we give moderate deviation principles of paths of…
In this paper, we present a novel iterative Monte Carlo method for approximating the stationary probability of a single state of a positive recurrent Markov chain. We utilize the characterization that the stationary probability of a state…
We study a class of stochastic models of mass transport on discrete vertex set $V$. For these models, a one-parameter family of homogeneous product measures $\otimes_{i\in V} \nu_\theta$ is reversible. We prove that the set of mixtures of…
A new mechanism for efficiently solving the Markov decision processes (MDPs) is proposed in this paper. We introduce the notion of reachability landscape where we use the Mean First Passage Time (MFPT) as a means to characterize the…
In many important real-world queueing settings, arrival and service rates fluctuate over time. We consider the MAMS system, where the arrival and service rates each vary according to an arbitrary finite-state Markov chain, allowing…
The distribution of the "mixing time" or the "time to stationarity" in a discrete time irreducible Markov chain, starting in state i, can be defined as the number of trials to reach a state sampled from the stationary distribution of the…
Motivated by a risk process with positive and negative premium rates, we consider a real-valued Markov additive process with finitely many background states. This additive process linearly increases or decreases while the background state…
In this paper, we investigate second-order necessary conditions and exact penalty of mathematical programs with switching constraints (MPSC). Some new second-order constraint qualifications and second-order quasi-normality are introduced…
We consider a two station cascade system in which waiting or externally arriving customers at station $1$ move to the station $2$ if the queue size of station $1$ including a customer being served is greater than a given threshold level…
We study a class of Markov processes with finite state space and continuous time that have product form stationary distributions. We obtain a number of examples that can generate conjectures for diffusions with inert drift.
We endow the classical stochastic fluid process with a duration-dependent Markovian arrival process (DMArP). We show that this provides a flexible model for the revenue of a solar energy generator. In particular, it allows for heavy-tailed…
We consider the equation of the Jeffreys type as the basic one in three different models of mass transfer, namely, the Jeffreys type and two-phase models, and the $D_1$ approximation to the linear Boltzmann equation. We study two classic 1…
Many networking-related settings can be modeled by Markov-modulated infinite-server systems. In such models, the customers' arrival rates and service rates are modulated by a Markovian background process, additionally, there are infinitely…