Related papers: Information-theoretic thresholds from the cavity m…
The richness in the content of various information networks such as social networks and communication networks provides the unprecedented potential for learning high-quality expressive representations without external supervision. This…
The mutual information between two jointly distributed random variables $X$ and $Y$ is a functional of the joint distribution $P_{XY},$ which is sometimes difficult to handle or estimate. A coarser description of the statistical behavior of…
We present general results for the contact process by a method which applies to all transitive graphs of bounded degree, including graphs of exponential growth. The model's infection rates are varied through a control parameter, for which…
Fact-based Visual Question Answering (FVQA) requires external knowledge beyond visible content to answer questions about an image, which is challenging but indispensable to achieve general VQA. One limitation of existing FVQA solutions is…
We define the relevant information in a signal $x\in X$ as being the information that this signal provides about another signal $y\in \Y$. Examples include the information that face images provide about the names of the people portrayed, or…
Many complex systems can be modeled by temporal networks, whose organization often evolves through distinct structural phases. Detecting the change points that delimit these phases is both important and challenging. In this work, we extend…
The mutual information of two random variables i and j with joint probabilities t_ij is commonly used in learning Bayesian nets as well as in many other fields. The chances t_ij are usually estimated by the empirical sampling frequency…
In this paper, we establish generalization bounds for transductive learning algorithms in the context of information theory and PAC-Bayes, covering both the random sampling and the random splitting setting. First, we show that the…
We study random graphs with latent geometric structure, where the probability of each edge depends on the underlying random positions corresponding to the two endpoints. We focus on the setting where this conditional probability is a…
The superfluid to Mott insulator transition and the superradiant transition are textbook examples for quantum phase transition and coherent quantum optics, respectively. Recent experiments in ETH and Hamburg succeeded in loading degenerate…
A central concern across the natural sciences is a quantitative understanding of the mechanism governing rare transitions between two metastable states. Recent research has uncovered a fundamental equality between the time-reversal…
Various phase transitions in models for coupled charge-density waves are investigated by means of the $\epsilon$-expansion, mean-field theory, and Monte Carlo simulations. At zero temperature the effective action for the system with…
Mutual information has been successfully adopted in filter feature-selection methods to assess both the relevancy of a subset of features in predicting the target variable and the redundancy with respect to other variables. However,…
Random constraint satisfaction problems play an important role in computer science and combinatorics. For example, they provide challenging benchmark instances for algorithms and they have been harnessed in probabilistic constructions of…
Information-theoretic quantities, such as entropy, are used to quantify the amount of information a given variable provides. Entropies can be used together to compute the mutual information, which quantifies the amount of information two…
"Bounds on information combining" are entropic inequalities that determine how the information (entropy) of a set of random variables can change when these are combined in certain prescribed ways. Such bounds play an important role in…
The mutual information is bounded from above by a decreasing affine function of the square of the distance between the input distribution and the set of all capacity-achieving input distributions $\Pi_{\mathcal{A}}$, on small enough…
Information theory is an outstanding framework to measure uncertainty, dependence and relevance in data and systems. It has several desirable properties for real world applications: it naturally deals with multivariate data, it can handle…
We study information flow on a weighted graph whose topology evolves according to a spectral curvature measure $\mathcal{R}$. The model (FIU) defines $\mathcal{R}$ from the diagonal of the graph Green function, propagates energy with…
The dynamics of phase transitions plays a crucial r\^ole in the so-called interface between high energy particle physics and cosmology. Many of the interesting results generated during the last fifteen years or so rely on simplified…