Related papers: An intuition for physicists: information gain from…
Obtaining an accurate estimate of the underlying covariance matrix from finite sample size data is challenging due to sample size noise. In recent years, sophisticated covariance-cleaning techniques based on random matrix theory have been…
When the von Neumann entropy (VNE) of a system increases due to measurements, certain information is lost, some of which may be recoverable. We define information retrievability (IR) and information loss (IL) as functions of the density…
In a first part, we present a mathematical analysis of a general methodology of a probabilistic learning inference that allows for estimating a posterior probability model for a stochastic boundary value problem from a prior probability…
Many practical studies rely on hypothesis testing procedures applied to data sets with missing information. An important part of the analysis is to determine the impact of the missing data on the performance of the test, and this can be…
We address quantum estimation in situations where one has at disposal data from the measurement of an incomplete set of observables and some a priori information on the state itself. By expressing the a priori information in terms of a bias…
No consensus seems to exist as to what constitutes a measurement which is still considered somewhat mysterious in many respects in quantum mechanics. At successive stages mathematical theory of measure, metrology and measurement theory…
How much information about an unknown quantum state can be obtained by a measurement? We propose a model independent answer: the information obtained is equal to the minimum entropy of the outputs of the measurement, where the minimum is…
A unifying theory is put forward that entropy is equal to action. The crowning derivation is based on information theoretic methods and uses our hypothesis that "particles move via the discrete Bernoulli Process." While this hypothesis…
For generic systems exhibiting power law behaviors, and hence multiscale dependencies, we propose a new, and yet simple, tool to analyze multifractality and intermittency, after noticing that these concepts are directly related to the…
In this paper, we delve deeper into the Kullback-Leibler (KL) Divergence loss and mathematically prove that it is equivalent to the Decoupled Kullback-Leibler (DKL) Divergence loss that consists of 1) a weighted Mean Square Error (wMSE)…
A classic definition of multisensory integration (MI) has been proposed as ``the presence of a (statistically) significant change in the response to a cross-modal stimulus complex compared to unimodal stimuli''. However, this general…
This paper provides an elementary, self-contained analysis of diffusion-based sampling methods for generative modeling. In contrast to existing approaches that rely on continuous-time processes and then discretize, our treatment works…
Experimental designs are tools which can drastically reduce the number of simulations required by time-consuming computer codes. One strategy for selecting the values of the inputs, whose response is to be observed, is to choose these…
Many quantum chemical similarity measures have been derived and substantiated by applying concepts and quantities from information theory to the electron density. To justify the use of information theory, the electron density is usually…
Integrated information theory is a mathematical, quantifiable theory of conscious experience. The linchpin of this theory, the $\phi$ measure, quantifies a system's irreducibility to disjoint parts. Purely as a measure of irreducibility, we…
Deep nonlinear models pose a challenge for fitting parameters due to lack of knowledge of the hidden layer and the potentially non-affine relation of the initial and observed layers. In the present work we investigate the use of information…
This paper deals with the taking into account a given set of realizations as constraints in the Kullback-Leibler minimum principle, which is used as a probabilistic learning algorithm. This permits the effective integration of data into…
The improvement in the performance of efficient and lightweight models (i.e., the student model) is achieved through knowledge distillation (KD), which involves transferring knowledge from more complex models (i.e., the teacher model).…
The concept of varentropy has been recently introduced as a dispersion index of the reliability of measure of information. In this paper, we introduce new measures of variability for two measures of uncertainty, the Kerridge inaccuracy…
The book is structured into four main chapters. Chapter 1 introduces the foundational concepts of divergence measures, including the well-known Kullback-Leibler divergence and its limitations. It then presents a detailed exploration of…