English

Adjust Pearson's $r$ to Measure Arbitrary Monotone Dependence

Other Statistics 2024-10-15 v7

Abstract

Pearson's r, the most widely-used correlation coefficient, is traditionally regarded as exclusively capturing linear dependence, leading to its discouragement in contexts involving nonlinear relationships. However, recent research challenges this notion, suggesting that Pearson's r should not be ruled out a priori for measuring nonlinear monotone relationships. Pearson's r is essentially a scaled covariance, rooted in the renowned Cauchy-Schwarz Inequality. Our findings reveal that different scaling bounds yield coefficients with different capture ranges, and interestingly, tighter bounds actually expand these ranges. We derive a tighter inequality than Cauchy-Schwarz Inequality, leverage it to refine Pearson's r, and propose a new correlation coefficient, i.e., rearrangement correlation. This coefficient is able to capture arbitrary monotone relationships, both linear and nonlinear ones. It reverts to Pearson's r in linear scenarios. Simulation experiments and real-life investigations show that the rearrangement correlation is more accurate in measuring nonlinear monotone dependence than the three classical correlation coefficients, and other recently proposed dependence measures.

Keywords

Cite

@article{arxiv.2205.04571,
  title  = {Adjust Pearson's $r$ to Measure Arbitrary Monotone Dependence},
  author = {Xinbo Ai},
  journal= {arXiv preprint arXiv:2205.04571},
  year   = {2024}
}

Comments

NeurIPS 2024. Rearrangement Correlation at both population and sample levels. 16 pages, 2 theorems, 2 corollaries, and 2 propositions

R2 v1 2026-06-24T11:12:13.270Z