Related papers: Approximate group context tree
Markov decision processes are typically used for sequential decision making under uncertainty. For many aspects however, ranging from constrained or safe specifications to various kinds of temporal (non-Markovian) dependencies in task and…
Modeling the time evolution of discrete sets of items (e.g., genetic mutations) is a fundamental problem in many biomedical applications. We approach this problem through the lens of continuous-time Markov chains, and show that the…
We consider the problem of estimating the context tree of a stationary ergodic process with finite alphabet without imposing additional conditions on the process. As a starting point we introduce a Hamming metric in the space of irreducible…
We develop a model for credit rating migration that accounts for the impact of economic state fluctuations on default probabilities. The joint process for the economic state and the rating is modelled as a time-homogeneous Markov chain.…
Markov chains for probability distributions related to matrix product states and 1D Hamiltonians are introduced. With appropriate 'inverse temperature' schedules, these chains can be combined into a random approximation scheme for ground…
We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion…
We consider Markov chains on general state spaces in stationary random environment which are defined by a random mapping that is contractive up to a bounded perturbation. We prove their convergence to a limiting law, providing convergence…
In this paper, we develop necessary and sufficient conditions for the validity of a martingale approximation for the partial sums of a stationary process in terms of the maximum of consecutive errors. Such an approximation is useful for…
Motivated by the evident success of context-tree based methods in lossless data compression, we explore, in this paper, methods of the same spirit in universal prediction of individual sequences. By context-tree prediction, we refer to a…
We consider a general method for the approximation of the distribution of a process conditioned to not hit a given set. Existing methods are based on particle system that are failable, in the sense that, in many situations , they are not…
This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine…
We study a new technique for the asymptotic analysis of heavy-tailed systems conditioned on large deviations events. We illustrate our approach in the context of ruin events of multidimensional regularly varying random walks. Our approach…
We examine a constrained Markov decision process under uncertain transition probabilities, with the uncertainty modeled as deviations from observed transition probabilities. We construct the uncertainty set associated with the deviations…
We address the problem of community detection in networks by introducing a general definition of Markov stability, based on the difference between the probability fluxes of a Markov chain on the network at different time scales. The…
We present a scheme for sequential decision making with a risk-sensitive objective and constraints in a dynamic environment. A neural network is trained as an approximator of the mapping from parameter space to space of risk and policy with…
Motivated by applications in systems biology, we seek a probabilistic framework based on Markov processes to represent intracellular processes. We review the formal relationships between different stochastic models referred to in the…
Neural marked temporal point processes have been a valuable addition to the existing toolbox of statistical parametric models for continuous-time event data. These models are useful for sequences where each event is associated with a single…
We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean. We design a novel recursive estimator that requires $O(1)$…
This paper proposes a family of weighted batch means variance estimators, which are computationally efficient and can be conveniently applied in practice. The focus is on Markov chain Monte Carlo simulations and estimation of the asymptotic…
In this paper we present an algorithm for pricing barrier options in one-dimensional Markov models. The approach rests on the construction of an approximating continuous-time Markov chain that closely follows the dynamics of the given…