Related papers: Some notes on a semi-Markov matrix occurring in th…

Sequences of parametrized Lyapunov equations can be encountered in many application settings. Moreover, solutions of such equations are often intermediate steps of an overall procedure whose main goal is the computation of…

We introduce a Markov chain model of concurrent quantum programs. This model is a quantum generalization of Hart, Sharir and Pnueli's probabilistic concurrent programs. Some characterizations of the reachable space, uniformly repeatedly…

Bayesian inference for Markov processes has become increasingly relevant in recent years. Problems of this type often have intractable likelihoods and prior knowledge about model rate parameters is often poor. Markov Chain Monte Carlo…

Lecture notes (in French) of a master 2 level course in applied mathematics. Contents: Part I. Markov chains on a countable space. 1. Examples 2. Summary of basic properties. 3. Spectral theory and speed of convergence. 4. Lyapunov…

We discuss the efficient computation of performance, reliability, and availability measures for Markov chains; these metrics, and the ones obtained by combining them, are often called performability measures. We show that this computational…

In this article, we provide different representations for a time-fractional birth and death process $N_{\alpha}(t)$, whose transition probabilities are governed by a time-fractional system of differential equations. More specifically, we…

The main purpose of this paper is to consider the multiple birth properties for multi-type Markov branching processes. We first construct a new multi-dimensional Markov process based on the multi-type Markov branching process, which can…

We present and explore a general method for deriving a Lie-Markov model from a finite semigroup. If the degree of the semigroup is $k$, the resulting model is a continuous-time Markov chain on $k$ states and, as a consequence of the product…

We introduce Markov substitute processes, a new model at the crossroad of statistics and formal grammars, and prove its main property : Markov substitute processes with a given support form an exponential family.

This article describes a method for computing limits of a class of non-stationary Markov chains motivated by healthcare sojourn-time cycles. A mathematical validation of the computation method is also given. Applications are described that…

Sequential scaling is a prominent inference-time scaling paradigm, yet its performance improvements are typically modest and not well understood, largely due to the prevalence of heuristic, non-principled approaches that obscure clear…

This paper studies average-cost Markov decision processes with semi-uniform Feller transition probabilities. This class of MDPs was recently introduced by the authors to study MDPs with incomplete information. This paper studies the…

This is an expository review paper illustrating the ``martingale method'' for proving many-server heavy-traffic stochastic-process limits for queueing models, supporting diffusion-process approximations. Careful treatment is given to an…

In this paper we study a non-stationary Markovian queueing model of a two-processor heterogeneous system with time-varying arrival and service rates. We obtain the bounds on the rate of convergence and find the main limiting characteristics…

Given a semi-Markov law, using an additional parameter, we consider a family of stochastic flows corresponding to that law. Then we suitably select a particular flow, for which we obtain expressions of the meeting and merging probabilities…

After birth, extremely preterm infants often require specialized respiratory management in the form of invasive mechanical ventilation (IMV). Protracted IMV is associated with detrimental outcomes and morbidities. Premature extubation, on…

Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…

We introduce Multi-Environment Markov Decision Processes (MEMDPs) which are MDPs with a set of probabilistic transition functions. The goal in a MEMDP is to synthesize a single controller with guaranteed performances against all…

We develop operator renewal theory for flows and apply this to obtain results on mixing and rates of mixing for a large class of finite and infinite measure semiflows. Examples of systems covered by our results include suspensions over…

Consider a Markov chain defined on a finite state space, X, that leaves invariant the uniform distribution on X, and whose transition probabilities are integer multiples of 1/Q, for some integer Q. I show how a simulation of n transitions…