Related papers: Information loss under coarse-graining: a geometri…
The ever-increasing parameter counts of deep learning models necessitate effective compression techniques for deployment on resource-constrained devices. This paper explores the application of information geometry, the study of…
Many machine learning methods assume that the training and test data follow the same distribution. However, in the real world, this assumption is very often violated. In particular, the phenomenon that the marginal distribution of the data…
Statistical inference more often than not involves models which are non-linear in the parameters thus leading to non-Gaussian posteriors. Many computational and analytical tools exist that can deal with non-Gaussian distributions, and…
We introduce a generalized machine learning framework to probabilistically parameterize upper-scale models in the form of nonlinear PDEs consistent with a continuum theory, based on coarse-grained atomistic simulation data of mechanical…
The common idea behind complexity reduction in physical systems is separation of information into "physically meaningful" and "safely ignorable". Here we consider a generic notion of such separation -- implemented by coarse-graining the…
We make the first steps towards generalizing the theory of stochastic block models, in the sparse regime, towards a model where the discrete community structure is replaced by an underlying geometry. We consider a geometric random graph…
Data augmentation is one of the most widely used techniques to improve generalization in modern machine learning, often justified by its ability to promote invariance to label-irrelevant transformations. However, its theoretical role…
A background-independent route towards a universal continuum limit in discrete models of quantum gravity proceeds through a background-independent form of coarse graining. This review provides a pedagogical introduction to the conceptual…
Graph coarsening is a technique for solving large-scale graph problems by working on a smaller version of the original graph, and possibly interpolating the results back to the original graph. It has a long history in scientific computing…
The Renormalization Group encodes three concepts that could be key to accelerate progress in quantum gravity. First, it provides a micro-macro connection that could connect microscopic spacetime physics to phenomenology at observationally…
Extraction of structure, in particular of group symmetries, is increasingly crucial to understanding and building intelligent models. In particular, some information-theoretic models of parsimonious learning have been argued to induce…
The Marginally Rigid State is a candidate paradigm for what makes granular material a state of matter distinct from both liquid and solid. Coordination number is identified as a discriminating characteristic, and for rough-surfaced…
In this paper we focus on the development of new methods suitable for efficient and reliable coarse-graining of {\it non-equilibrium} molecular systems. In this context, we propose error estimation and controlled-fidelity model reduction…
Information geometry is used to quantify the amount of information integration within multiple terminals of a causal dynamical system. Integrated information quantifies how much information is lost when a system is split into parts and…
We present a new approach to coarse-graining of variables describing dust flow in GR. It is based on assigning quasi-local shear, twist and expansion to 2-dimensional surfaces with the help of isometric embeddings into the 3-dimensional…
We present a possible approach to the study of the renormalization group (RG) flow based entirely on the information theory. The average information loss under a single step of Wilsonian RG transformation is evaluated as a conditional…
Information geometry is a study of statistical manifolds, that is, spaces of probability distributions from a geometric perspective. Its classical information-theoretic applications relate to statistical concepts such as Fisher information,…
We show that gamma distributions provide models for departures from randomness since every neighbourhood of an exponential distribution contains a neighbourhood of gamma distributions, using an information theoretic metric topology. We…
Lossy data transformations by definition lose information. Yet, in modern machine learning, methods like data pruning and lossy data augmentation can help improve generalization performance. We study this paradox using a solvable model of…
In this work, we study generalized entropies and information geometry in a group-theoretical framework. We explore the conditions that ensure the existence of some natural properties and at the same time of a group-theoretical structure for…