Related papers: Limit Theorems for quadratic forms of Markov Chain…

This paper derives the asymptotic behavior of realized power variation of pure-jump It\^{o} semimartingales as the sampling frequency within a fixed interval increases to infinity. We prove convergence in probability and an associated…

We consider the problem of learning a conditional Gaussian graphical model in the presence of latent variables. Building on recent advances in this field, we suggest a method that decomposes the parameters of a conditional Markov random…

In this work, we consider a finite-state inhomogeneous-time Markov chain whose probabilities of transition from one state to another tend to decrease over time. This can be seen as a cooling of the dynamics of an underlying Markov chain. We…

This paper studies the estimation of low-rank Markov chains from empirical trajectories. We propose a non-convex estimator based on rank-constrained likelihood maximization. Statistical upper bounds are provided for the Kullback-Leiber…

The goal of this paper is to highlight the almost sure central limit theorem for martingales to the control community and to show the usefulness of this result for the system identification of controllable ARX(p,q) process in adaptive…

For a relatively large class of well-behaved absorbing (or killed) finite Markov chains, we give detailed quantitative estimates regarding the behavior of the chain before it is absorbed (or killed). Typical examples are random walks on…

The central limit theorem of martingales is the fundamental tool for studying the convergence of stochastic processes, especially stochastic integrals and differential equations. In this paper, general central limit theorems and functional…

Continuous-time Markov chains describing interacting processes exhibit a state space that grows exponentially in the number of processes. This state-space explosion renders the computation or storage of the time-marginal distribution, which…

We develop a regression based primal-dual martingale approach for solving finite time horizon MDPs with general state and action space. As a result, our method allows for the construction of tight upper and lower biased approximations of…

This note presents conjectures on polynomial/algebraic/sub-exponential convergence of transition probabilities for $\lambda$-null recurrent and $\lambda$-transient Markov chains in continuous time. The only known positive examples are in…

A continuous-time Markov chain rate change formula for simulation, model selection, filtering and theory is proven. It is used to develop Markov chain importance sampling, rejection sampling, branching particle filtering algorithms and…

For Markov processes evolving on multiple time-scales a combination of large component scalings and averaging of rapid fluctuations can lead to useful limits for model approximation. A general approach to proving a law of large numbers to a…

Predictability of behavior has emerged an an important characteristic in many fields including biology, medicine, and marketing. Behavior can be recorded as a sequence of actions performed by an individual over a given time period. This…

Asymptotic properties of Markov Processes, such as steady state probabilities or hazard rate for absorbing states can be efficiently calculated by means of linear algebra even for large-scale problems. This paper discusses the methods for…

In this work we present a modified neural network model which is capable to simulate Markov Chains. We show how to express and train such a network, how to ensure given statistical properties reflected in the training data and we…

When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-called credal sets that…

This paper is concerned with a central limit theorem for quadratic variation when observations come as exit times from a regular grid. We discuss the special case of a semimartingale with deterministic characteristics and finite activity…

We consider the problem of estimating the measure of subsets in very large networks. A prime tool for this purpose is the Markov Chain Monte Carlo (MCMC) algorithm. This algorithm, while extremely useful in many cases, still often suffers…

We consider the problem of estimating the transition rate matrix of a continuous-time Markov chain from a finite-duration realisation of this process. We approach this problem in an imprecise probabilistic framework, using a set of prior…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…