Related papers: Stochastic processes with random contexts: a chara…
This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is…
The paper deals with a certain class of random evolutions. We develop a construction that yields an invariant measure for a continuous-time Markov process with random transitions. The approach is based on a particular way of constructing…
This paper presents a general approach to linear stochastic processes driven by various random noises. Mathematically, such processes are described by linear stochastic differential equations of arbitrary order (the simplest non-trivial…
Biochemical reaction networks frequently consist of species evolving on multiple timescales. Stochastic simulations of such networks are often computationally challenging and therefore various methods have been developed to obtain sensible…
This brief article gives an overview of quantum mechanics as a {\em quantum probability theory}. It begins with a review of the basic operator-algebraic elements that connect probability theory with quantum probability theory. Then quantum…
This paper describes the Context Tree Switching technique, a modification of Context Tree Weighting for the prediction of binary, stationary, n-Markov sources. By modifying Context Tree Weighting's recursive weighting scheme, it is possible…
Motivated by reduction of computational complexity, this work develops sign-error adaptive filtering algorithms for estimating time-varying system parameters. Different from the previous work on sign-error algorithms, the parameters are…
Estimating the transition dynamics of controlled Markov chains is crucial in fields such as time series analysis, reinforcement learning, and system exploration. Traditional non-parametric density estimation methods often assume independent…
It is well established that gene expression can be modeled as a Markovian stochastic process and hence proper observables might be subjected to large fluctuations and rare events. Since dynamics is often more than statics, one can work with…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
In multi-period stochastic optimization problems, the future optimal decision is a random variable whose distribution depends on the parameters of the optimization problem. We analyze how the expected value of this random variable changes…
We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the…
Branching processes are a class of continuous-time Markov chains (CTMCs) prevalent for modeling stochastic population dynamics in ecology, biology, epidemiology, and many other fields. The transient or finite-time behavior of these systems…
The problem of appropriately matching items subject to compatibility constraints arises in a number of important applications. While most of the literature on matching theory focuses on a static setting with a fixed number of items, several…
Predictive equivalence in discrete stochastic processes have been applied with great success to identify randomness and structure in statistical physics and chaotic dynamical systems and to inferring hidden Markov models. We examine the…
This paper considers the problem of randomized influence maximization over a Markovian graph process: given a fixed set of nodes whose connectivity graph is evolving as a Markov chain, estimate the probability distribution (over this fixed…
We show that one can approximate the least fixed point solution for a multivariate system of monotone probabilistic polynomial equations in time polynomial in both the encoding size of the system of equations and in log(1/\epsilon), where…
In this paper, we consider semi-Markov processes whose transition times and transition probabilities depend on a small parameter $\varepsilon$. Understanding the asymptotic behavior of such processes is needed in order to study the…
We develop a model for credit rating migration that accounts for the impact of economic state fluctuations on default probabilities. The joint process for the economic state and the rating is modelled as a time-homogeneous Markov chain.…
We describe an exact approach for calculating transition probabilities and waiting times in finite-state discrete-time Markov processes. All the states and the rules for transitions between them must be known in advance. We can then…