On Randomized Approximation of Scattered Data

Let $A$ be a set and $V$ a real Hilbert space. Let $H$ be a real Hilbert space of functions $f:A\to V$ and assume $H$ is continuously embedded in the Banach space of bounded functions. For $i=1,\cdots,n$, let $(x_i,y_i)\in A\times V$ comprise our dataset. Let $0<q<1$ and $f^*\in H$ be the unique global minimizer of the functional \begin{equation*} u(f) = \frac{q}{2}\Vert f\Vert_{H}^{2} + \frac{1-q}{2n}\sum_{i=1}^{n}\Vert f(x_i)-y_i\Vert_{V}^{2}. \end{equation*} For $x\in A$ and $v\in V$ let $\Phi(x,v)\in H$ be the unique element such that $(\Phi(x,v),f)_{H}=(f(x),v)_{V}$ for all $f\in H$. In this paper we show that for each $k\in\mathbb{N}$, $k\geq 2$ one has a random function $F_{k}\in H$ with the structure \begin{equation*} F_{k} = \sum_{h=1}^{N_k} \Lambda_{k, h} \Phi(x_{I_h}, \mathcal{E}_{h}) \end{equation*} (where $0\leq N_k\leq k-1$ are Binomially distributed with success probability $1-q$, $\Lambda_{k, h}\in\mathbb{R}$ are random coefficients, $1\leq I_{h}\leq n$ are independent and uniformly distributed and $\mathcal{E}_{h}\in V$ are random vectors) such that asymptotically for large $k$ we have \begin{equation*} E\left[ \Vert F_{k}-f^*\Vert_{H}^{2} \right] = O(\frac{1}{k}). \end{equation*} Thus we achieve the Monte Carlo type error estimate with no metric or measurability structure on $A$, possibly infinite dimensional $V$ and the ingredients of approximating functions are just the Riesz representatives $\Phi(x,v)\in H$. We obtain this result by considering the stochastic gradient descent sequence in the Hilbert space $H$ to minimize the functional $u$.