Related papers: Likelihood Ratio Gradient Estimation for Steady-St…
Consider a Markov process $\{\Phi(t) : t\geq 0\}$ evolving on a Polish space ${\sf X}$. A version of the $f$-Norm Ergodic Theorem is obtained: Suppose that the process is $\psi$-irreducible and aperiodic. For a given function $f\colon{\sf…
We study random graphs with latent geometric structure, where the probability of each edge depends on the underlying random positions corresponding to the two endpoints. We focus on the setting where this conditional probability is a…
A discrimination problem consists of $N$ linearly independent pure quantum states $\Phi=\{\ket{\phi_i}\}$ and the corresponding occurrence probabilities $\eta=\{\eta_i\}$. To any such problem we associate, up to a permutation over the…
We observe a length-$n$ sample generated by an unknown,stationary ergodic Markov process (\emph{model}) over a finite alphabet $\mathcal{A}$. Given any string $\bf{w}$ of symbols from $\mathcal{A}$ we want estimates of the conditional…
A model for network panel data is discussed, based on the assumption that the observed data are discrete observations of a continuous-time Markov process on the space of all directed graphs on a given node set, in which changes in tie…
Let $L:= -a(x) (-\Delta)^{\alpha/2}+ (b(x), \nabla)$, where $\alpha\in (0,2)$, and $a:\rd\to (0,\infty)$, $b: \rd\to \rd$. Under certain regularity assumptions on the coefficients $a$ and $b$, we associate with the $C_\infty(\rd)$-closure…
It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of…
We prove a metric space scaling limit for a critical random graph with independent and identically distributed degrees having power-law tail behaviour with exponent $\alpha+1$, where $\alpha \in (1,2)$. The limiting components are…
Time-homogeneous Markov chains are often used as disease progression models in studies of cost-effectiveness and optimal decision-making. Maximum likelihood estimation of these models can be challenging when data are collected at a time…
Understanding and predicting how complex systems respond to external perturbations is a central challenge in nonequilibrium statistical physics. Here we consider continuous-time Markov networks, which we subject to perturbations along a…
The spectral gap $\gamma$ of a finite, ergodic, and reversible Markov chain is an important parameter measuring the asymptotic rate of convergence. In applications, the transition matrix $P$ may be unknown, yet one sample of the chain up to…
Stochastic differential equations and stochastic dynamics are good models to describe stochastic phenomena in real world. In this paper, we study N independent stochastic processes Xi(t) with real entries and the processes are determined by…
Policy gradient methods are extensively used in reinforcement learning as a way to optimize expected return. In this paper, we explore the evolution of the policy parameters, for a special class of exactly solvable POMDPs, as a…
Multi-object state estimation is a fundamental problem for robotic applications where a robot must interact with other moving objects. Typically, other objects' relevant state features are not directly observable, and must instead be…
In this work we investigate gradient estimation for a class of contracting stochastic systems on a continuous state space. We find conditions on the one-step transitions, namely differentiability and contraction in a Wasserstein distance,…
We consider a fundamental remote state estimation problem of discrete-time linear time-invariant (LTI) systems. A smart sensor forwards its local state estimate to a remote estimator over a time-correlated $M$-state Markov fading channel,…
We consider a quantum system with a time-independent Hamiltonian parametrized by a set of unknown parameters $\alpha$. The system is prepared in a general quantum state by an evolution operator that depends on a set of unknown parameters…
This paper explores the theory behind the rich and robust family of {\alpha}-stable distributions to estimate parameters from financial asset log-returns data. We discuss four-parameter estimation methods including the quantiles,…
This work is concerned with the stability properties of linear stochastic differential equations with random (drift and diffusion) coefficient matrices, and the stability of a corresponding random transition matrix (or exponential…
We prove the first Chernoff-Hoeffding bounds for general nonreversible finite-state Markov chains based on the standard L_1 (variation distance) mixing-time of the chain. Specifically, consider an ergodic Markov chain M and a weight…