Related papers: A generalization of Doob's maximal identity
Policy Iteration (PI) is a widely used family of algorithms to compute optimal policies for Markov Decision Problems (MDPs). We derive upper bounds on the running time of PI on Deterministic MDPs (DMDPs): the class of MDPs in which every…
We consider a recurrent random walk on a rooted tree in random environment given by a branching random walk. Up to the first return to the root, its edge local times form a Multi-type Galton-Watson tree with countably infinitely many types.…
A martingale framework for concept change detection based on testing data exchangeability was recently proposed (Ho, 2005). In this paper, we describe the proposed change-detection test based on the Doob's Maximal Inequality and show that…
For the stochastic multi-armed bandit (MAB) problem from a constrained model that generalizes the classical one, we show that an asymptotic optimality is achievable by a simple strategy extended from the $\epsilon_t$-greedy strategy. We…
Given a sequence $(M^n)^{\infty}_{n=1}$ of nonnegative martingales starting at $M^n_0=1$, we find a sequence of convex combinations $(\widetilde{M}^n)^{\infty}_{n=1}$ and a limiting process $X$ such that…
We establish a rigorous duality theory, under No Unbounded Profit with Bounded Risk, for an infinite horizon problem of optimal consumption in the presence of an income stream that can terminate randomly at an exponentially distributed…
We provide sufficient conditions which ensure that the intrinsic martingale in the supercritical branching random walk converges exponentially fast to its limit. The case of Galton-Watson processes is particularly included so that our…
A general diffusion semimartingale is a one-dimensional path-continuous semimartingale that is also a regular strong Markov process. We say that a continuous semimartingale has the representation property if all local martingales w.r.t. its…
We consider the random field M(t)=\sup_{n\geq 1}\big\{-\log A_{n}+X_{n}(t)\big\}\,,\qquad t\in T\, for a set $T\subset \mathbb{R}^{m}$, where $(X_{n})$ is an iid sequence of centered Gaussian random fields on $T$ and $0<A_{1}<A_{2}<\cdots $…
We construct a class of nonnegative martingale processes that oscillate indefinitely with high probability. For these processes, we state a uniform rate of the number of oscillations and show that this rate is asymptotically close to the…
For local martingales with nonnegative jumps, we prove a sufficient criterion for the corresponding exponential martingale to be a true martingale. The criterion is in terms of exponential moments of a convex combination of the optional and…
(Jaynes') Method of (Shannon-Kullback's) Relative Entropy Maximization (REM or MaxEnt) can be - at least in the discrete case - according to the Maximum Probability Theorem (MPT) viewed as an asymptotic instance of the Maximum Probability…
We show that the maximum moments of the sum of independent positive semidefinite random matrices with given norm upper bounds and norms of expectations is attained when all the random matrices are the multiplications of certain random…
We present $\texttt{Maxent}$, a tool for performing analytic continuation of spectral functions using the maximum entropy method. The code operates on discrete imaginary axis datasets (values with uncertainties) and transforms this input to…
Mutual localization is essential for coordination and cooperation in multi-robot systems. Previous works have tackled this problem by assuming available correspondences between measurements and received odometry estimations, which are…
For integers $n\geq r$, we treat the $r$th largest of a sample of size $n$ as an $\mathbb{R}^\infty$-valued stochastic process in $r$ which we denote $\mathbf{M}^{(r)}$. We show that the sequence regarded in this way satisfies the Markov…
We consider deterministic homogenization (convergence to a stochastic differential equation) for multiscale systems of the form \[ x_{k+1} = x_k + n^{-1} a_n(x_k,y_k) + n^{-1/2} b_n(x_k,y_k), \quad y_{k+1} = T_n y_k, \] where the fast…
We develop a new method for deriving local laws for a large class of random matrices. It is applicable to many matrix models built from sums and products of deterministic or independent random matrices. In particular, it may be used to…
This paper concerns the almost sure time dependent local extinction behavior for super-coalescing Brownian motion $X$ with $(1+\beta)$-stable branching and Lebesgue initial measure on $\bR$. We first give a representation of $X$ using…
We consider the local limit of finite uniformly distributed directed animals on the square lattice viewed from the root. Two constructions of the resulting uniform infinite directed animal are given: one as a heap of dominoes, constructed…