Related papers: Approximate group context tree
In this paper we propose a new method for approximating the nonstationary moment dynamics of one dimensional Markovian birth-death processes. By expanding the transition probabilities of the Markov process in terms of Poisson-Charlier…
We consider a general class of branching processes in discrete time, where particles have types belonging to a Polish space and reproduce independently according to their type. If the process is critical and the mean distribution of types…
Markov processes are used in a wide range of disciplines, including finance. The transition densities of these processes are often unknown. However, the conditional characteristic functions are more likely to be available, especially for…
We consider synthesis of control policies that maximize the probability of satisfying given temporal logic specifications in unknown, stochastic environments. We model the interaction between the system and its environment as a Markov…
Markov chain Monte Carlo is an inherently serial algorithm. Although likelihood calculations for individual steps can sometimes be parallelized, the serial evolution of the process is widely viewed as incompatible with parallelization,…
In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…
We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in $\ell$ steps after starting from a given…
For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
Continuous-time Bayesian networks is a natural structured representation language for multicomponent stochastic processes that evolve continuously over time. Despite the compact representation, inference in such models is intractable even…
We give rates of convergence in the strong invariance principle for stationary sequences satisfying some projective criteria. The conditions are expressed in terms of conditional expectations of partial sums of the initial sequence. Our…
When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This is done by considering as basic uncertainty models the so-called credal sets that…
Up to now, the nonparametric analysis of multidimensional continuous-time Markov processes has focussed strongly on specific model choices, mostly related to symmetry of the semigroup. While this approach allows to study the performance of…
We are interested in the analysis of very large continuous-time Markov chains (CTMCs) with many distinct rates. Such models arise naturally in the context of reliability analysis, e.g., of computer network performability analysis, of power…
Consideration is given to the three different analytical methods for the computation of upper bounds for the rate of convergence to the limiting regime of one specific class of (in)homogeneous continuous-time Markov chains. This class is…
We study the existence of densities for distributions of piecewise deterministic Markov processes. We also obtain relationships between invariant densities of the continuous time process and that of the process observed at jump times. In…
We show that one can approximate the least fixed point solution for a multivariate system of monotone probabilistic polynomial equations in time polynomial in both the encoding size of the system of equations and in log(1/\epsilon), where…
In this short paper, we consider discrete-time Markov chains on lattices as approximations to continuous-time diffusion processes. The approximations can be interpreted as finite difference schemes for the generator of the process. We…
Pairwise Choice Markov Chains (PCMC) have been recently introduced to overcome limitations of choice models based on traditional axioms unable to express empirical observations from modern behavior economics like context effects occurring…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…