Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in Memoryless Transduction
Adaptation and Self-Organizing Systems
2022-02-17 v1 Disordered Systems and Neural Networks
Machine Learning
Data Analysis, Statistics and Probability
Abstract
Discovering relevant, but possibly hidden, variables is a key step in constructing useful and predictive theories about the natural world. This brief note explains the connections between three approaches to this problem: the recently introduced information-bottleneck method, the computational mechanics approach to inferring optimal models, and Salmon's statistical relevance basis.
Keywords
Cite
@article{arxiv.nlin/0006025,
title = {Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in Memoryless Transduction},
author = {Cosma Rohilla Shalizi and James P. Crutchfield},
journal= {arXiv preprint arXiv:nlin/0006025},
year = {2022}
}
Comments
3 pages, no figures, submitted to PRE as a "brief report". Revision: added an acknowledgements section originally omitted by a LaTeX bug