Interpreting the learning dynamics of neural networks can provide useful insights into how networks learn and the development of better training and design approaches. We present an approach to interpret learning in neural networks by measuring relative weight change on a per layer basis and dynamically aggregating emerging trends through combination of dimensionality reduction and clustering which allows us to scale to very deep networks. We use this approach to investigate learning in the context of vision tasks across a variety of state-of-the-art networks and provide insights into the learning behavior of these networks, including how task complexity affects layer-wise learning in deeper layers of networks.
@article{arxiv.2107.07005,
title = {WeightScale: Interpreting Weight Change in Neural Networks},
author = {Ayush Manish Agrawal and Atharva Tendle and Harshvardhan Sikka and Sahib Singh},
journal= {arXiv preprint arXiv:2107.07005},
year = {2022}
}
Comments
Intelligent Computing, 2021. arXiv admin note: text overlap with arXiv:2011.06735