Related papers: Information Theory in Density Destructors
Recently we proposed an information entropy based method for electronic structure calculations within the density-matrix functional theory(DMFT) (Phys. Rev. Lett. 128, 013001), dubbed as $i$-DMFT. Comments have been raised regarding the…
We introduce a parametric nonlinear transformation that is well-suited for Gaussianizing data from natural images. The data are linearly transformed, and each component is then normalized by a pooled activity measure, computed by…
We investigate the analogy between the renormalization group (RG) and deep neural networks, wherein subsequent layers of neurons are analogous to successive steps along the RG. In particular, we quantify the flow of information by…
The mutual information between a set of stimuli and the elicited neural responses is compared to the corresponding decoded information. The decoding procedure is presented as an artificial distortion of the joint probabilities between…
Document classification is the detection specific content of interest in text documents. In contrast to the data-driven machine learning classifiers, knowledge-based classifiers can be constructed based on domain specific knowledge, which…
Learning about density functional approximations (DFAs), or approximations for the exchange-correlation functional, can be intimidating. Density Functional Theory is now one of the primary simulation tools for the practicing chemist or…
Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this…
This paper investigates, from information theoretic grounds, a learning problem based on the principle that any regularity in a given dataset can be exploited to extract compact features from data, i.e., using fewer bits than needed to…
The cross-entropy loss commonly used in deep learning is closely related to the defining properties of optimal representations, but does not enforce some of the key properties. We show that this can be solved by adding a regularization…
We introduce a method for calculating the probability density function (PDF) of a turbulent density field in three dimensions using only information contained in the projected two-dimensional column density field. We test the method by…
The evolution of probability distribution functions (PDFs) of continuous density, velocity and velocity derivatives ( deformation tensor) fields in the theory of cosmological gravitational instability are considered. We show that in the…
Document parsing (DP) transforms unstructured or semi-structured documents into structured, machine-readable representations, enabling downstream applications such as knowledge base construction and retrieval-augmented generation (RAG).…
Denoising diffusion models (DDMs) have recently attracted increasing attention by showing impressive synthesis quality. DDMs are built on a diffusion process that pushes data to the noise distribution and the models learn to denoise. In…
Normalizing flows are generative models that provide tractable density estimation via an invertible transformation from a simple base distribution to a complex target distribution. However, this technique cannot directly model data…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
A introduction into density-functional theory and electronic structure methods is given, that aims at providing an intuitive understanding of the underlying concepts for the novice as well as an entry point towards the more advanced…
We propose a new thermodynamic, relativistic relationship between information and entropy, which is closely analogous to the classic Maxwell electro-magnetic equations. Determination of whether information resides in points of…
In this work we investigate the information loss in (nonlinear) dynamical input-output systems and provide some general results. In particular, we present an upper bound on the information loss rate, defined as the (non-negative) difference…
The one-point probability distribution function (PDF) of the matter density field in the universe is a fundamental property that plays an essential role in cosmology for estimates such as gravitational weak lensing, non-linear clustering,…
What is information? Is it physical? We argue that in a Bayesian theory the notion of information must be defined in terms of its effects on the beliefs of rational agents. Information is whatever constrains rational beliefs and therefore…