Related papers: Experimental Information and Statistical Modeling …
Kernel density estimation is a technique for approximating probability distributions. Here, it is applied to the calculation of mutual information on a metric space. This is motivated by the problem in neuroscience of calculating the mutual…
These lectures deal with the problem of inductive inference, that is, the problem of reasoning under conditions of incomplete information. Is there a general method for handling uncertainty? Or, at least, are there rules that could in…
Measures of discrepancy between probability distributions (statistical distance) are widely used in the fields of artificial intelligence and machine learning. We describe how certain measures of statistical distance can be implemented as…
Diffusion models are a class of probabilistic generative models that have been widely used as a prior for image processing tasks like text conditional generation and inpainting. We demonstrate that these models can be adapted to make…
The exploration of complex physical or technological processes usually requires exploiting available information from different sources: (i) physical laws often represented as a family of parameter dependent partial differential equations…
The uncertainty principle can be expressed in entropic terms, also taking into account the role of entanglement in reducing uncertainty. The information exclusion principle bounds instead the correlations that can exist between the outcomes…
We explore the connection between deep learning and information theory through the paradigm of diffusion models. A diffusion model converts noise into structured data by reinstating, imperfectly, information that is erased when data was…
We study relativistic scattering when one only has access to a subset of the particles, using the language of quantum measurement theory. We give an exact, non-perturbative formula for the von Neumann entanglement entropy of an apparatus…
Random matrix ensembles (RME) of quantum statistical Hamiltonian operators, {\em e.g.} Gaussian random matrix ensembles (GRME) and Ginibre random matrix ensembles (Ginibre RME), found applications in literature in study of following quantum…
The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…
The information diffusion prediction on social networks aims to predict future recipients of a message, with practical applications in marketing and social media. While different prediction models all claim to perform well, general…
A central area of research in nonlinear science is the study of instabilities that drive the emergence of extreme events. Unfortunately, experimental techniques for measuring such phenomena often provide only partial characterization. For…
Most psychophysical experiments discard half the data collected. Specifically, experiments discard reaction time data, and use binary responses (e.g. yes/no) to measure performance. Here, Shannon's information theory is used to define…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
A comparison of structural features of quantum and classical physical theories, such as the information capacity of systems subject to these theories, requires a common formal framework for the presentation of corresponding concepts (such…
Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…
An analysis of quantum measurement is presented that relies on an information-theoretic description of quantum entanglement. In a consistent quantum information theory of entanglement, entropies (uncertainties) conditional on measurement…
It is argued that the nature of probability is essentially informational rather than physical and that quantum mechanical predictions should be viewed as logical inferences made on the basis of the information content of a given…
The traditional approach of statistical physics to supervised learning routinely assumes unrealistic generative models for the data: usually inputs are independent random variables, uncorrelated with their labels. Only recently, statistical…
Complex systems are found in most branches of science. It is still argued how to best quantify their complexity and to what end. One prominent measure of complexity (the statistical complexity) has an operational meaning in terms of the…