Network with Sub-Networks
Machine Learning
2021-10-20 v2 Computer Vision and Pattern Recognition
Abstract
We introduce network with sub-networks, a neural network which its weight layers could be detached into sub-neural networks during inference. To develop weights and biases which could be inserted in both base and sub-neural networks, firstly, the parameters are copied from sub-model to base-model. Each model is forward-propagated separately. Gradients from a pair of networks are averaged and, used to update both networks. Our base model achieves the test-accuracy which is comparable to the regularly trained models, while the model maintains the ability to detach weight layers.
Cite
@article{arxiv.1908.00763,
title = {Network with Sub-Networks},
author = {Ninnart Fuengfusin and Hakaru Tamukoh},
journal= {arXiv preprint arXiv:1908.00763},
year = {2021}
}