Related papers: Hierarchical Maximum Entropy via the Renormalizati…
Graph mining to extract interesting components has been studied in various guises, e.g., communities, dense subgraphs, cliques. However, most existing works are based on notions of frequency and connectivity and do not capture subjective…
This paper provides a mixture modeling framework using the bivariate generalized exponential distribution. We study different properties of this mixture distribution. Hierarchical EM algorithm is developed for finding the estimates of the…
We consider a set of probabilistic functions of some input variables as a representation of the inputs. We present bounds on how informative a representation is about input data. We extend these bounds to hierarchical representations so…
We derive a functional for the entropy contributed by any microscopic degrees of freedom as arising from their measurable pair correlations. Applicable both in and out of equilibrium, this functional yields the maximum entropy which a…
In this paper, it is shown that an auto-encoder using optimal reconstruction significantly outperforms a conventional auto-encoder. Optimal reconstruction uses the conditional mean of the input given the features, under a maximum entropy…
In this paper we describe a triple correspondence between graph limits, information theory and group theory. We put forward a new graph limit concept called log-convergence that is closely connected to dense graph limits but its main…
Recent work emphasizes that the maximum entropy principle provides a bridge between statistical mechanics models for collective behavior in neural networks and experiments on networks of real neurons. Most of this work has focused on…
We investigate the analogy between the renormalization group (RG) and deep neural networks, wherein subsequent layers of neurons are analogous to successive steps along the RG. In particular, we quantify the flow of information by…
Hierarchical structure and repetition are prevalent in graphs originating from nature or engineering. These patterns can be represented by a class of parametric-structure graphs, which are defined by templates that generate structure by way…
Random graphs offer a useful mathematical representation of a variety of real world complex networks. Exponential random graphs, for example, are particularly suited towards generating random graphs constrained to have specified statistical…
Many real-world networks of interest are embedded in physical space. We present a new random graph model aiming to reflect the interplay between the geometries of the graph and of the underlying space. The model favors configurations with…
Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…
Recent years have seen the rise of convolutional neural network techniques in exemplar-based image synthesis. These methods often rely on the minimization of some variational formulation on the image space for which the minimizers are…
Maximum entropy methods, rooted in the inverse Ising/Potts problem from statistical physics, are widely used to model pairwise interactions in complex systems across disciplines such as bioinformatics and neuroscience. While successful,…
ulticasting is an important communication paradigm for enabling the dissemination of information selectively. This paper considers the problem of optimal secure multicasting in a communication network captured through a graph (optimal is in…
Finding low-dimensional interpretable models of complex physical fields such as turbulence remains an open question, 80 years after the pioneer work of Kolmogorov. Estimating high-dimensional probability distributions from data samples…
Network data often arises via a series of structured interactions among a population of constituent elements. E-mail exchanges, for example, have a single sender followed by potentially multiple receivers. Scientific articles, on the other…
Hypergraphs, encoding structured interactions among any number of system units, have recently proven a successful tool to describe many real-world biological and social networks. Here we propose a framework based on statistical inference to…
The Renyi distribution ensuring the maximum of a Renyi entropy is investigated for a particular case of a power--law Hamiltonian. Both Lagrange parameters, $\alpha$ and $\beta$ can be excluded. It is found that $\beta$ does not depend on a…
This study presents a hierarchical mining framework for high-dimensional imbalanced data, leveraging a depth graph model to address the inherent performance limitations of conventional approaches in handling complex, high-dimensional data…