English

Extracting individual variable information for their decoupling, direct mutual information and multi-feature Granger causality

Machine Learning 2023-11-23 v1 Information Theory Machine Learning math.IT

Abstract

Working with multiple variables they usually contain difficult to control complex dependencies. This article proposes extraction of their individual information, e.g. XY\overline{X|Y} as random variable containing information from XX, but with removed information about YY, by using (x,y)(xˉ=CDFXY=y(x),y)(x,y) \leftrightarrow (\bar{x}=\textrm{CDF}_{X|Y=y}(x),y) reversible normalization. One application can be decoupling of individual information of variables: reversibly transform (X1,,Xn)(X~1,X~n)(X_1,\ldots,X_n)\leftrightarrow(\tilde{X}_1,\ldots \tilde{X}_n) together containing the same information, but being independent: ijX~iX~j,X~iXj\forall_{i\neq j} \tilde{X}_i\perp \tilde{X}_j, \tilde{X}_i\perp X_j. It requires detailed models of complex conditional probability distributions - it is generally a difficult task, but here can be done through multiple dependency reducing iterations, using imperfect methods (here HCR: Hierarchical Correlation Reconstruction). It could be also used for direct mutual information - evaluating direct information transfer: without use of intermediate variables. For causality direction there is discussed multi-feature Granger causality, e.g. to trace various types of individual information transfers between such decoupled variables, including propagation time (delay).

Keywords

Cite

@article{arxiv.2311.13431,
  title  = {Extracting individual variable information for their decoupling, direct mutual information and multi-feature Granger causality},
  author = {Jarek Duda},
  journal= {arXiv preprint arXiv:2311.13431},
  year   = {2023}
}

Comments

3 pages, 1 figure

R2 v1 2026-06-28T13:28:38.271Z