Training Neural Networks for Modularity aids Interpretability
Abstract
An approach to improve network interpretability is via clusterability, i.e., splitting a model into disjoint clusters that can be studied independently. We find pretrained models to be highly unclusterable and thus train models to be more modular using an ``enmeshment loss'' function that encourages the formation of non-interacting clusters. Using automated interpretability measures, we show that our method finds clusters that learn different, disjoint, and smaller circuits for CIFAR-10 labels. Our approach provides a promising direction for making neural networks easier to interpret.
Cite
@article{arxiv.2409.15747,
title = {Training Neural Networks for Modularity aids Interpretability},
author = {Satvik Golechha and Dylan Cope and Nandi Schoots},
journal= {arXiv preprint arXiv:2409.15747},
year = {2025}
}
Comments
Some of the interpretations of the results in this paper were incorrect, and we found on further experiments that the techniques did not scale well - we have the corrected results in a different submission (due to a significant change in both content and authors it needed to be a new submission). Please check https://arxiv.org/abs/2502.02470 for the updated paper.