Related papers: Information transfer in generalized probabilistic …
Knowledge Transfer (KT) techniques tackle the problem of transferring the knowledge from a large and complex neural network into a smaller and faster one. However, existing KT methods are tailored towards classification tasks and they…
Generic computability has been studied in group theory and we now study it in the context of classical computability theory. A set A of natural numbers is generically computable if there is a partial computable function f whose domain has…
Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome's plausibility. Information measures based on…
We investigate the power of weak measurements in the framework of quantum state discrimination. First, we define and analyze the notion of weak consecutive measurements. Our main result is a convergence theorem whereby we demonstrate when…
In the paper we present results to develop an irreducible theory of complex systems in terms of self-organization processes of prime integer relations. Based on the integers and controlled by arithmetic only the self-organization processes…
We introduce the notion of a database system that is information theoretically "Secure In Between Accesses"--a database system with the properties that 1) users can efficiently access their data, and 2) while a user is not accessing their…
Grokking, or delayed generalization, is an intriguing learning phenomenon where test set loss decreases sharply only after a model's training set loss has converged. This challenges conventional understanding of the training dynamics in…
The non-zero minimal length arises in various theories of gravity, leading to the so-called generalized uncertainty principle (GUP). In this short paper we analyze the GUP effects on neutron interferometry, showing that the obtained phase…
A new finite form of de Finetti's representation theorem is established using elementary information-theoretic tools. The distribution of the first $k$ random variables in an exchangeable vector of $n\geq k$ random variables is close to a…
The Generative Pre-trained Transformer (GPT) represents a notable breakthrough in the domain of natural language processing, which is propelling us toward the development of machines that can understand and communicate using language in a…
The Kochen-Specker theorem states that exclusive and complete deterministic outcome assignments are impossible for certain sets of measurements, called Kochen-Specker (KS) sets. A straightforward consequence is that KS sets do not have…
Resource theories provide a general framework for the characterization of properties of physical systems in quantum mechanics and beyond. Here, we introduce methods for the quantification of resources in general probabilistic theories…
We study belief revision when information is represented by a set of probability distributions, or general information. General information extends the standard event notion while including qualitative information (A is more likely than B),…
Parameter-efficient fine-tuning approaches have recently garnered a lot of attention. Having considerably lower number of trainable weights, these methods can bring about scalability and computational effectiveness. In this paper, we look…
We present a possible approach to the study of the renormalization group (RG) flow based entirely on the information theory. The average information loss under a single step of Wilsonian RG transformation is evaluated as a conditional…
We consider the application of deep generative models in propagating uncertainty through complex physical systems. Specifically, we put forth an implicit variational inference formulation that constrains the generative model output to…
We present the first complete confirmation of Granovetter's theory of social networks using a massive dataset. For this purpose, we study a scientific collaboration network, which is considered one of the most important examples that…
Collective intelligence, which aggregates the shared information from large crowds, is often negatively impacted by unreliable information sources with the low quality data. This becomes a barrier to the effective use of collective…
We propose a formal expansion of the transfer entropy to put in evidence irreducible sets of variables which provide information for the future state of each assigned target. Multiplets characterized by a large contribution to the expansion…
Using unreliable or noisy components for reliable communication requires error correction. But which noise processes can support information transmission, and which are too destructive? For classical systems any channel whose output depends…