Finding disentangled representation plays a predominant role in the success of modern deep learning applications, but the results lack a straightforward explanation. Here we apply the information bottleneck method and its β-VAE implementation to find the disentangled low-dimensional representation of classical models. For the Ising model, our results reveal a deep connection between the disentangled features and the physical order parameters, and the widely-used Bernoulli decoder is found to be learning a mean-field Hamiltonian at fixed temperature. This analogy motivates us to extend the application of β-VAE to more complex classical models with non-binary variables using different decoder neural network and propose a modified architecture β2-VAE to enforce thermal fluctuations in generated samples. Our work provides a way to design novel physics-informed algorithm that can yield learned features in potential correspondence with real physical properties.
@article{arxiv.2110.08082,
title = {Learning disentangled representation for classical models},
author = {Dongchen Huang and Danqing Hu and Yi-feng Yang},
journal= {arXiv preprint arXiv:2110.08082},
year = {2022}
}