Related papers: Predictive Characterization of Mixtures of Markov …

Even simply-defined, finite-state generators produce stochastic processes that require tracking an uncountable infinity of probabilistic features for optimal prediction. For processes generated by hidden Markov chains the consequences are…

Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…

The paper deals with a certain class of random evolutions. We develop a construction that yields an invariant measure for a continuous-time Markov process with random transitions. The approach is based on a particular way of constructing…

Markov chains are a class of probabilistic models that have achieved widespread application in the quantitative sciences. This is in part due to their versatility, but is compounded by the ease with which they can be probed analytically.…

We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion…

The dynamics in games involving multiple players, who adaptively learn from their past experience, is not yet well understood. We analyzed a class of stochastic games with Markov strategies in which players choose their actions…

Predictive recursion is an accurate and computationally efficient algorithm for nonparametric estimation of mixing densities in mixture models. In semiparametric mixture models, however, the algorithm fails to account for any uncertainty in…

New results on conditional joint probability distributions of first exit times are presented for a continuous-time stochastic process defined as the mixture of Markov jump processes moving at different speeds on the same finite state space,…

We show that boundary theory for transient Markov chains, as initiated by Doob, can be used to prove de Finetti's classical representation result for exchangeable random sequences. We also include the relevant parts of the theory, with full…

Measure-valued Markov chains have raised interest in Bayesian nonparametrics since the seminal paper by (Math. Proc. Cambridge Philos. Soc. 105 (1989) 579--585) where a Markov chain having the law of the Dirichlet process as unique…

Conditional distributions, as defined by the Markov category framework, are studied in the setting of matrix algebras (quantum systems). Their construction as linear unital maps are obtained via a categorical Bayesian inversion procedure.…

While for deterministic systems, a counterexample to a property can simply be an error trace, counterexamples in probabilistic systems are necessarily more complex. For instance, a set of erroneous traces with a sufficient cumulative…

Random walks on simple graphs in connection with electrical resistor networks lead to the definition of Markov chains with transition probability matrix in terms of electrical conductances. We extend this definition to an effective…

The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…

We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…

Automata expressiveness is an essential feature in understanding which of the formalisms available should be chosen for modelling a particular problem. Probabilistic and stochastic automata are suitable for modelling systems exhibiting…

This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…

We investigate the mixing properties of a finite Markov chain in random environment defined as a mixture of a deterministic chain and a chain whose state space has been permuted uniformly at random. This work is the counterpart of a…

It is well established that gene expression can be modeled as a Markovian stochastic process and hence proper observables might be subjected to large fluctuations and rare events. Since dynamics is often more than statics, one can work with…

The main purpose of this article is to establish moderate deviation principles for additive functionals of bifurcating Markov chains. Bifurcating Markov chains are a class of processes which are indexed by a regular binary tree. They can be…