Related papers: Some notes on a semi-Markov matrix occurring in th…
Spatial birth-and-death processes with a finite number of particles are obtained as unique solutions to certain stochastic equations. Conditions are given for existence and uniqueness of such solutions, as well as for continuous dependence…
In this work we introduce a new and richer class of finite order Markov chain models and address the following model selection problem: find the Markov model with the minimal set of parameters (minimal Markov model) which is necessary to…
Queueing networks are notoriously difficult to analyze sans both Markovian and stationarity assumptions. Much of the theoretical contribution towards performance analysis of time-inhomogeneous single class queueing networks has focused on…
In this paper, the recurrent events that can occur more than one over the follow-up time have been modeled by phase-type distributions. We use the finite-state continuous-time Markov process with multi states for patients with recurrent…
Semi-Markov processes (SMPs) provide a rich framework for many real-world problems. However, due to difficulty implementing practical solutions they are rarely used with their full capability. The theory of SMPs is quite mature but was…
The Meantime to Failure is a statistic used to determine how much time a system spends to enter one of its absorption states. This statistic can be used in most areas of knowledge. In engineering, for example, can be used as a measure of…
A mean-field extension of the queueing system \(GI/GI/1\) is considered. The process is constructed as a Markov solution of a martingale problem. Uniqueness in distribution is established under a bit different sets of assumptions on…
We consider Markov basis arising from fractional factorial designs with three-level factors. Once we have a Markov basis, $p$ values for various conditional tests are estimated by the Markov chain Monte Carlo procedure. For designed…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
We consider a two-node queue modeled as a two-dimensional random walk. In particular, we consider the case that one or both queues have finite buffers. We develop an approximation scheme based on the Markov reward approach to error bounds…
We present a method for solving implicit (factored) Markov decision processes (MDPs) with very large state spaces. We introduce a property of state space partitions which we call epsilon-homogeneity. Intuitively, an epsilon-homogeneous…
In many operations management problems, we need to make decisions sequentially to minimize the cost while satisfying certain constraints. One modeling approach to study such problems is constrained Markov decision process (CMDP). When…
In this paper we propose an alternative construction of the self-similar entrance laws for positive self-similar Markov processes. The study of entrance laws has been carried out in previous papers using different techniques, depending on…
Characterizing whether a Markov process of discrete random variables has an homogeneous continuous-time realization is a hard problem. In practice, this problem reduces to deciding when a given Markov matrix can be written as the…
Closed queuing networks with finite capacity buffers and skip-over policies are fundamental models in the performance evaluation of computer and communication systems. This technical report presents the details of computational algorithms…
This thesis is devoted to the study of affine processes and their applications in financial mathematics. In the first part we consider the theory of time-inhomogeneous affine processes on general state spaces. We present a concise setup for…
This paper is dedicated to the numerical study of the optimization of an industrial launcher integration process. It is an original case of inventory-production system where a calendar plays a crucial role. The process is modeled using the…
Filtering---estimating the state of a partially observable Markov process from a sequence of observations---is one of the most widely studied problems in control theory, AI, and computational statistics. Exact computation of the posterior…
We develop an approach to time-consistent risk evaluation of continuous-time processes in Markov systems. Our analysis is based on dual representation of coherent risk measures, differentiability concepts for multivalued mappings, and a…