Mutual Information Decay Curves and Hyper-Parameter Grid Search Design for Recurrent Neural Architectures
Machine Learning
2020-12-09 v1 Information Theory
math.IT
Abstract
We present an approach to design the grid searches for hyper-parameter optimization for recurrent neural architectures. The basis for this approach is the use of mutual information to analyze long distance dependencies (LDDs) within a dataset. We also report a set of experiments that demonstrate how using this approach, we obtain state-of-the-art results for DilatedRNNs across a range of benchmark datasets.
Cite
@article{arxiv.2012.04632,
title = {Mutual Information Decay Curves and Hyper-Parameter Grid Search Design for Recurrent Neural Architectures},
author = {Abhijit Mahalunkar and John D. Kelleher},
journal= {arXiv preprint arXiv:2012.04632},
year = {2020}
}
Comments
Published at the 27th International Conference on Neural Information Processing, ICONIP 2020, Bangkok, Thailand, November 18-22, 2020. arXiv admin note: text overlap with arXiv:1810.02966