Related papers: An Information-Theoretic Approach to Network Modul…

We describe algorithms for learning Bayesian networks from a combination of user knowledge and statistical data. The algorithms have two components: a scoring metric and a search procedure. The scoring metric takes a network structure,…

We describe techniques for the robust detection of community structure in some classes of time-dependent networks. Specifically, we consider the use of statistical null models for facilitating the principled identification of structural…

We consider the problem of learning a neural network classifier. Under the information bottleneck (IB) principle, we associate with this classification problem a representation learning problem, which we call "IB learning". We show that IB…

Using an intuitive concept of what constitutes a meaningful community, a novel metric is formulated for detecting non-overlapping communities in undirected, weighted heterogeneous networks. This metric, modularity density, is shown to be…

We present an information-theoretic framework for understanding overfitting and underfitting in machine learning and prove the formal undecidability of determining whether an arbitrary classification algorithm will overfit a dataset.…

Training a Neural Network (NN) with lots of parameters or intricate architectures creates undesired phenomena that complicate the optimization process. To address this issue we propose a first modular approach to NN design, wherein the NN…

Robust and comprehensive characterization of the topological properties of complex networks requires the adoption of several respective measurements, among which the node degree has special importance. In the present work, we provide an…

This paper presents the foundation for a decomposition theory for Boolean networks, a type of discrete dynamical system that has found a wide range of applications in the life sciences, engineering, and physics. Given a Boolean network…

Detecting communities in a network, based only on the adjacency matrix, is a problem of interest to several scientific disciplines. Recently, Zhang and Moore have introduced an algorithm in [P. Zhang and C. Moore, Proceedings of the…

The most widely used techniques for community detection in networks, including methods based on modularity, statistical inference, and information theoretic arguments, all work by optimizing objective functions that measure the quality of…

We develop a modular mean field theory for a minimalistic model of the idiotypic network. The model comprises the random influx of new idiotypes and a deterministic selection. It describes the evolution of the idiotypic network towards…

In this work, we introduce InfoDisent, a hybrid approach to explainability based on the information bottleneck principle. InfoDisent enables the disentanglement of information in the final layer of any pretrained model into atomic concepts,…

In this paper, we study bottleneck identification in networks via extracting minimax paths. Many real-world networks have stochastic weights for which full knowledge is not available in advance. Therefore, we model this task as a…

Estimating individual level treatment effects (ITE) from observational data is a challenging and important area in causal machine learning and is commonly considered in diverse mission-critical applications. In this paper, we propose an…

We study the properties of metrics aimed at the characterization of grid-like ordering in complex networks. These metrics are based on the global and local behavior of cycles of order four, which are the minimal structures able to identify…

In order to identify a system (module) embedded in a dynamic network, one has to formulate a multiple-input estimation problem that necessitates certain nodes to be measured and included as predictor inputs. However, some of these nodes may…

Learning with hidden variables is a central challenge in probabilistic graphical models that has important implications for many real-life problems. The classical approach is using the Expectation Maximization (EM) algorithm. This…

In this theory paper, we investigate training deep neural networks (DNNs) for classification via minimizing the information bottleneck (IB) functional. We show that the resulting optimization problem suffers from two severe issues: First,…

Complex systems are large collections of entities that organize themselves into non-trivial structures that can be represented by networks. A key emergent property of such systems is robustness against random failures or targeted attacks…

Inferring network topology from smooth signals is a significant problem in data science and engineering. A common challenge in real-world scenarios is the availability of only partially observed nodes. While some studies have considered…