Related papers: Maximum likelihood estimation in the two-state Mar…

We discuss two parameterizations of models for marginal independencies for discrete distributions which are representable by bi-directed graph models, under the global Markov property. Such models are useful data analytic tools especially…

We analyse the stochastic comparison of interacting particle systems allowing for multiple arrivals, departures and non-conservative jumps of individuals between sites. That is, if $k$ individuals leave site $x$ for site $y$, a possibly…

Pathways of diffusion observed in real-world systems often require stochastic processes going beyond first-order Markov models, as implicitly assumed in network theory. In this work, we focus on second-order Markov models, and derive an…

A complex multi-state redundant system with preventive maintenance subject to multiple events is considered. The online unit can undergo several types of failures: internal and those provoked by external shocks. Multiple degradation levels…

This paper examines a discrete-time queuing system with applications to telecommunications traffic. The arrival process is a particular Markov modulated process which belongs to the class of discrete batched Markovian arrival processes. The…

This paper considers the problem of randomized influence maximization over a Markovian graph process: given a fixed set of nodes whose connectivity graph is evolving as a Markov chain, estimate the probability distribution (over this fixed…

We study a class of systems termed Markov Machines (MM) which process job requests with exponential service times. Assuming a Poison job arrival process, these MMs oscillate between two states, free and busy. We consider the problem of…

Queueing systems with batch Markovian arrival process (BMAP) have paramount applications in the domain of wireless communication. The BMAP has been used to model the superposition of video sources and to approximate the super-position of…

In this work, a single-unit multi-state system is considered. The system is subject to internal failures, as well as external shocks with multiple consequences. It also incorporates a preventive maintenance strategy and a Bernoulli vacation…

Modeling unknown systems from data is a precursor of system optimization and sequential decision making. In this paper, we focus on learning a Markov model from a single trajectory of states. Suppose that the transition model has a small…

To provide a more accurate description of the driving behaviors in vehicle queues, a namely Markov-Gap cellular automata model is proposed in this paper. It views the variation of the gap between two consequent vehicles as a Markov process…

High time resolution of ultra wideband (UWB) signals facilitates very precise positioning capabilities based on time-of-arrival (TOA) measurements. Although the theoretical lower bound for TOA estimation can be achieved by the maximum…

This report considers a fairly general model of constrained queuing networks that allows us to represent both MMBP (Markov Modulated Bernoulli Processes) arrivals and time-varying service constraints. We derive a set of sufficient…

Asymptotic properties of Markov Processes, such as steady state probabilities or hazard rate for absorbing states can be efficiently calculated by means of linear algebra even for large-scale problems. This paper discusses the methods for…

In this study, we propose a mixture logistic regression model with a Markov structure, and consider the estimation of model parameters using maximum likelihood estimation. We also provide a forward type variable selection algorithm to…

In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…

We present a systematic way to analyze and model systems having many characteristic time-scales. The method we propose is employed for a test-case of a meandering jet model manifesting chaotic tracer dispersion with long time-correlations.…

Computing optimal conditional reachability probabilities in Markov decision processes (MDPs) is tractable by a reduction to reachability probabilities. Yet, this reduction yields cyclic, challenging MDPs that are often notoriously hard to…

Semi-Markov models are widely used for survival analysis and reliability analysis. In general, there are two competing parameterizations and each entails its own interpretation and inference properties. On the one hand, a semi-Markov…

The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…