English

Comment on Deterministic Information Bottleneck

Neurons and Cognition 2024-07-03 v1
Authors: Sarah Marzen
View on arXiv ↗ PDF ↗

Abstract

We make the case that although Deterministic Information Bottleneck may be a contribution to clustering, it should not be used to aid lossy compression without the addition of blocklength. We therefore suggest a new objective function that does so and leave its testing to future work.

Cite

@article{arxiv.2407.01786,
  title  = {Comment on Deterministic Information Bottleneck},
  author = {Sarah Marzen},
  journal= {arXiv preprint arXiv:2407.01786},
  year   = {2024}
}

Comments

2 pages; Comment on arXiv:1604.00268

Related papers

View all related →
Machine Learning · Statistics
Sparse clustering via the Deterministic Information Bottleneck algorithm
Efthymios Costa, Ioanna Papatsouma, Angelos Markos
2026-04-14
Applications · Statistics
Extracting relevant structures from self-determination theory questionnaires via Information Bottleneck method
Daniela M L Barbato, Jean-Jacques De Groote
2018-06-06
Methodology · Statistics
A Deterministic Information Bottleneck Method for Clustering Mixed-Type Data
Efthymios Costa, Ioanna Papatsouma, Angelos Markos
2026-02-02
Machine Learning · Computer Science
Drop-Bottleneck: Learning Discrete Compressed Representation for Noise-Robust Exploration
Jaekyeom Kim, Minjung Kim, Dongyeon Woo, Gunhee Kim
2021-03-24
Signal Processing · Electrical Eng. & Systems
Opportunistic Information-Bottleneck for Goal-oriented Feature Extraction and Communication
Francesco Binucci, Paolo Banelli, Paolo Di Lorenzo, Sergio Barbarossa
2024-04-17
Machine Learning · Computer Science
Unpacking Information Bottlenecks: Unifying Information-Theoretic Objectives in Deep Learning
Andreas Kirsch, Clare Lyle, Yarin Gal
2021-01-06
Machine Learning · Computer Science
Deep Deterministic Information Bottleneck with Matrix-based Entropy Functional
Xi Yu, Shujian Yu, Jose C. Principe
2021-02-02
Neurons and Cognition · Quantitative Biology
The deterministic information bottleneck
DJ Strouse, David J Schwab
2017-02-23
Machine Learning · Computer Science
Interpretability with full complexity by constraining feature information
Kieran A. Murphy, Dani S. Bassett
2023-05-11
Computer Vision and Pattern Recognition · Computer Science
Unsupervised Hashing with Contrastive Information Bottleneck
Zexuan Qiu, Qinliang Su, Zijing Ou, Jianxing Yu +1
2021-05-20
Machine Learning · Computer Science
Class-Conditional Compression and Disentanglement: Bridging the Gap between Neural Networks and Naive Bayes Classifiers
Rana Ali Amjad, Bernhard C. Geiger
2019-06-07
Machine Learning · Computer Science
How Does Information Bottleneck Help Deep Learning?
Kenji Kawaguchi, Zhun Deng, Xu Ji, Jiaoyang Huang
2023-05-31
Computer Vision and Pattern Recognition · Computer Science
Disentangled Representation Learning with Transmitted Information Bottleneck
Zhuohang Dang, Minnan Luo, Chengyou Jia, Guang Dai +3
2024-08-15
Computation and Language · Computer Science
An Information Bottleneck Approach for Controlling Conciseness in Rationale Extraction
Bhargavi Paranjape, Mandar Joshi, John Thickstun, Hannaneh Hajishirzi +1
2020-11-04
Machine Learning · Computer Science
Multivariate Information Bottleneck
Nir Friedman, Ori Mosenzon, Noam Slonim, Naftali Tishby
2013-01-14
Information Theory · Computer Science
On the Relevance-Complexity Region of Scalable Information Bottleneck
Mohammad Mahdi Mahvari, Mari Kobayashi, Abdellatif Zaidi
2020-11-04
Machine Learning · Computer Science
Bottleneck Tokens for Unified Multimodal Retrieval
Siyu Sun, Jing Ren, Zhaohe Liao, Dongxiao Mao +7
2026-04-14
Machine Learning · Computer Science
Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression
Ivan Butakov, Alexander Tolmachev, Sofia Malanchuk, Anna Neopryatnaya +2
2024-05-10
Machine Learning · Computer Science
Learning Optimal Representations with the Decodable Information Bottleneck
Yann Dubois, Douwe Kiela, David J. Schwab, Ramakrishna Vedantam
2021-07-19
Information Theory · Computer Science
The Variational Deficiency Bottleneck
Pradeep Kr. Banerjee, Guido Montúfar
2020-11-05
Information Theory · Computer Science
Nonlinear Information Bottleneck
Artemy Kolchinsky, Brendan D. Tracey, David H. Wolpert
2022-11-22
Machine Learning · Computer Science
Training Normalizing Flows with the Information Bottleneck for Competitive Generative Classification
Lynton Ardizzone, Radek Mackowiak, Carsten Rother, Ullrich Köthe
2021-01-13
Machine Learning · Computer Science
Variational Predictive Information Bottleneck
Alexander A. Alemi
2019-10-25
Machine Learning · Computer Science
Disentangled Information Bottleneck
Ziqi Pan, Li Niu, Jianfu Zhang, Liqing Zhang
2020-12-23
Information Theory · Computer Science
Distributed Information Bottleneck Theory for Multi-Modal Task-Aware Semantic Communication
Yujie Zhou, Cheng Peng, Rulong Wang, Yong Xiao +3
2026-01-01
R2 v1 2026-06-28T17:25:44.782Z