English

Yule's "nonsense correlation" solved: Part II

Statistics Theory 2023-01-31 v5 Statistics Theory

Abstract

In 1926, G. Udny Yule considered the following: given a sequence of pairs of random variables {Xk,Yk}\{X_k,Y_k \} (k=1,2,,nk=1,2, \ldots, n), and letting Xi=SiX_i = S_i and Yi=SiY_ i= S'_i where SiS_i and SiS'_i are the partial sums of two independent random walks, what is the distribution of the empirical correlation coefficient \begin{equation*} \rho_n = \frac{\sum_{i=1}^n S_i S^\prime_i - \frac{1}{n}(\sum_{i=1}^n S_i)(\sum_{i=1}^n S^\prime_i)}{\sqrt{\sum_{i=1}^n S^2_i - \frac{1}{n}(\sum_{i=1}^n S_i)^2}\sqrt{\sum_{i=1}^n (S^\prime_i)^2 - \frac{1}{n}(\sum_{i=1}^n S^\prime_i)^2}}? \end{equation*} Yule empirically observed the distribution of this statistic to be heavily dispersed and frequently large in absolute value, leading him to call it "nonsense correlation." This unexpected finding led to his formulation of two concrete questions, each of which would remain open for more than ninety years: (i) Find (analytically) the variance of ρn\rho_n as nn \rightarrow \infty and (ii): Find (analytically) the higher order moments and the density of ρn\rho_n as nn \rightarrow \infty. In 2017, Ernst, Shepp, and Wyner considered the empirical correlation coefficient \begin{equation*} \rho:= \frac{\int_0^1W_1(t)W_2(t) dt - \int_0^1W_1(t) dt \int_0^1 W_2(t) dt}{\sqrt{\int_0^1 W^2_1(t) dt - (\int_0^1W_1(t) dt)^2} \sqrt{\int_0^1 W^2_2(t) dt - (\int_0^1W_2(t) dt)^2}}\end{equation*} of two independent Wiener processes W1,W2W_1,W_2, the limit to which ρn\rho_n converges weakly, as was first shown by Phillips (1986). Using tools from integral equation theory, Ernst et al. (2017) closed question (i) by explicitly calculating the second moment of ρ\rho to be .240522. This paper begins where Ernst et al. (2017) leaves off. We succeed in closing question (ii) by explicitly calculating all moments of ρ\rho (up to order 16).

Cite

@article{arxiv.1909.02546,
  title  = {Yule's "nonsense correlation" solved: Part II},
  author = {Philip A. Ernst and L. C. G. Rogers and Quan Zhou},
  journal= {arXiv preprint arXiv:1909.02546},
  year   = {2023}
}

Comments

28 pages, 2 figures

R2 v1 2026-06-23T11:07:03.226Z