Related papers: An intuition for physicists: information gain from…
The goal of this short note is to discuss the relation between Kullback--Leibler divergence and total variation distance, starting with the celebrated Pinsker's inequality relating the two, before switching to a simple, yet (arguably) more…
We show that quantum-to-classical channels, i.e., quantum measurements, can be asymptotically simulated by an amount of classical communication equal to the quantum mutual information of the measurement, if sufficient shared randomness is…
Knowledge distillation has been used to transfer knowledge learned by a sophisticated model (teacher) to a simpler model (student). This technique is widely used to compress model complexity. However, in most applications the compressed…
Epistemic emotions, such as curiosity and interest, drive the inquiry process. This study proposes a novel formulation of epistemic emotions such as curiosity and interest using two types of information gain generated by the principle of…
Uncertainty is an important and fundamental concept in physics education. Students are often first exposed to uncertainty in introductory labs, expand their knowledge across lab courses, and then are introduced to quantum mechanical…
The usual conjectures of quantum measurements approaches, inspired from the traditional interpretation of Heisenberg's ("uncertainty") relations, are proved as being incorrect. A group of reconsidered conjectures and a corresponding new…
A new paradigm for distributed quantum systems where information is a valuable resource is developed. After finding a unique measure for information, we construct a scheme for it's manipulation in analogy with entanglement theory. In this…
The Kullback-Leibler divergence, the Kullback-Leibler variation, and the Bernstein "norm" are used to quantify discrepancies among probability distributions in likelihood models such as nonparametric maximum likelihood and nonparametric…
By taking the need for quantum reference frames into account, it is shown that Hilbert-space factorization is a dissipative process requiring on the order of kT to reduce by one bit an observer's uncertainty in the provenance of a…
The problem is sequence prediction in the following setting. A sequence x1,..., xn,... of discrete-valued observations is generated according to some unknown probabilistic law (measure) mu. After observing each outcome, it is required to…
In quantum experiments the acquisition and representation of basic experimental information is governed by the multinomial probability distribution. There exist unique random variables, whose standard deviation becomes asymptotically…
This work introduces Knowledge-Distilled Physics-Informed Neural Networks (KD-PINN), a framework that transfers the predictive accuracy of a high-capacity teacher model to a compact student through a continuous adaptation of the…
We present a statistical mechanical framework based on the Kullback-Leibler divergence (KLD) to analyze the relativistic limits of decoding time-encoded information from a moving source. By modeling the symbol durations as…
In this paper we establish lower bounds on information divergence from a distribution to certain important classes of distributions as Gaussian, exponential, Gamma, Poisson, geometric, and binomial. These lower bounds are tight and for…
The aim of this paper is to provide new theoretical and computational understanding on two loss regularizations employed in deep learning, known as local entropy and heat regularization. For both regularized losses we introduce variational…
We propose a method to fuse posterior distributions learned from heterogeneous datasets. Our algorithm relies on a mean field assumption for both the fused model and the individual dataset posteriors and proceeds using a simple…
We investigate the information provided about a specified distributed apparatus of n units in the measurement of a quantum state. It is shown that, in contrast to such measurement of a classical state, which is bounded by log (n+1) bits,…
While a positive operator valued measure gives the probabilities in a quantum measurement, an instrument gives both the probabilities and the a posteriori states. By interpreting the instrument as a quantum channel and by using the…
The widely applicable information criterion (WAIC) has been used as a model selection criterion for Bayesian statistics in recent years. It is an asymptotically unbiased estimator of the Kullback-Leibler divergence between a Bayesian…
We characterize Martin-L\"of randomness and Schnorr randomness in terms of the merging of opinions, along the lines of the Blackwell-Dubins Theorem. After setting up a general framework for defining notions of merging randomness, we focus…