Renormalized Mutual Information for Artificial Scientific Discovery
Machine Learning
2021-05-26 v3 Data Analysis, Statistics and Probability
Abstract
We derive a well-defined renormalized version of mutual information that allows to estimate the dependence between continuous random variables in the important case when one is deterministically dependent on the other. This is the situation relevant for feature extraction, where the goal is to produce a low-dimensional effective description of a high-dimensional system. Our approach enables the discovery of collective variables in physical systems, thus adding to the toolbox of artificial scientific discovery, while also aiding the analysis of information flow in artificial neural networks.
Cite
@article{arxiv.2005.01912,
title = {Renormalized Mutual Information for Artificial Scientific Discovery},
author = {Leopoldo Sarra and Andrea Aiello and Florian Marquardt},
journal= {arXiv preprint arXiv:2005.01912},
year = {2021}
}
Comments
Added a more detailed introduction and link to code repository. Physics-based examples and Feature Extraction section have been updated