Learning Sparse Latent Representations with the Deep Copula Information Bottleneck

Aleksander Wieczorek; Mario Wieser; Damian Murezzan; Volker Roth

Learning Sparse Latent Representations with the Deep Copula Information Bottleneck

Machine Learning 2018-04-20 v2 Machine Learning

Authors: Aleksander Wieczorek , Mario Wieser , Damian Murezzan , Volker Roth

Abstract

Deep latent variable models are powerful tools for representation learning. In this paper, we adopt the deep information bottleneck model, identify its shortcomings and propose a model that circumvents them. To this end, we apply a copula transformation which, by restoring the invariance properties of the information bottleneck method, leads to disentanglement of the features in the latent space. Building on that, we show how this transformation translates to sparsity of the latent space in the new model. We evaluate our method on artificial and real data.

Keywords

deep learning copula models latent variable models

Cite

@article{arxiv.1804.06216,
  title  = {Learning Sparse Latent Representations with the Deep Copula Information Bottleneck},
  author = {Aleksander Wieczorek and Mario Wieser and Damian Murezzan and Volker Roth},
  journal= {arXiv preprint arXiv:1804.06216},
  year   = {2018}
}

Comments

Published as a conference paper at ICLR 2018. Aleksander Wieczorek and Mario Wieser contributed equally to this work

Learning Sparse Latent Representations with the Deep Copula Information Bottleneck

Abstract

Keywords

Cite

Comments

Related papers