Related papers: Likelihood Ratio Gradient Estimation for Steady-St…
We consider a mechanism for area preserving Hamiltonian systems which leads to the enhanced probability, $P(\lambda, t)$, to find small values of the finite time Lyapunov exponent, $\lambda$. In our investigation of chaotic dynamical…
We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this…
We examine two analytical characterisation of the metastable behavior of a Markov chain. The first one expressed in terms of its transition probabilities, and the second one in terms of its large deviations rate functional. Consider a…
As is well known, the fundamental matrix $(I - P + e \pi)^{-1}$ plays an important role in the performance analysis of Markov systems, where $P$ is the transition probability matrix, $e$ is the column vector of ones, and $\pi$ is the row…
Providing theoretical guarantees for parameter estimation in exponential random graph models is a largely open problem. While maximum likelihood estimation has theoretical guarantees in principle, verifying the assumptions for these…
Ergodic properties and asymptotic stationarity are investigated in this paper for the pseudo-covariance matrix (PCM) of a recursive state estimator which is robust against parametric uncertainties and is based on plant output measurements…
When particles/molecules diffuse in systems that contain obstacles, the steady-state regime (during which the mean-square displacement scales linearly with time, $\left< r^2 \right> \sim t$) is preceded by a transient regime. It is common…
The class of $\alpha$-stable distributions is widely used in various applications, especially for modelling heavy-tailed data. Although the $\alpha$-stable distributions have been used in practice for many years, new methods for…
We consider a family of Markov chains whose transition dynamics are affected by model parameters. Understanding the parametric dependence of (complex) performance measures of such Markov chains is often of significant interest. The…
We provide a novel method for sensitivity analysis of parametric robust Markov chains. These models incorporate parameters and sets of probability distributions to alleviate the often unrealistic assumption that precise probabilities are…
We consider discrete graphical models Markov with respect to a graph $G$ and propose two distributed marginal methods to estimate the maximum likelihood estimate of the canonical parameter of the model. Both methods are based on a…
When performing maximum-likelihood quantum-state tomography, one must find the quantum state that maximizes the likelihood of the state given observed measurements on identically prepared systems. The optimization is usually performed with…
We consider a stochastic differential equation of the form \[dX_t=\theta a(t,X_t)\,dt+\sigma_1(t,X_t)\sigma_2(t,Y_t)\,dW_t\] with multiplicative stochastic volatility, where $Y$ is some adapted stochastic process. We prove…
This paper investigates the position (state) distribution of the single step binomial (multi-nomial) process on a discrete state / time grid under the assumption that the velocity process rather than the state process is Markovian. In this…
In this article we consider the estimation of static parameters for partially observed diffusion processes with discrete-time observations over a fixed time interval. In particular, when one only has access to time-discretized solutions of…
A conditional independence graph is a concise representation of pairwise conditional independence among many variables. Graphical Random Forests (GRaFo) are a novel method for estimating pairwise conditional independence relationships among…
We present on-line policy gradient algorithms for computing the locally optimal policy of a constrained, average cost, finite state Markov Decision Process. The stochastic approximation algorithms require estimation of the gradient of the…
The methodology based on the random walk processes is adapted and applied to a comprehensive analysis of the statistical properties of the probability fluxes. To this aim we define a simple model of the Markovian stochastic dynamics on a…
We investigate the fractional Vasicek model described by the stochastic differential equation $dX_t=(\alpha -\beta X_t)\,dt+\gamma \,dB^H_t$, $X_0=x_0$, driven by the fractional Brownian motion $B^H$ with the known Hurst parameter $H\in…
Asymptotic properties of Markov Processes, such as steady state probabilities or hazard rate for absorbing states can be efficiently calculated by means of linear algebra even for large-scale problems. This paper discusses the methods for…