Related papers: Summary of Information Theoretic Quantities
Information theory has become an increasingly important research field to better understand quantum mechanics. Noteworthy, it covers both foundational and applied perspectives, also offering a common technical language to study a variety of…
An information-theoretic framework is introduced to analyze last-layer embedding, focusing on learned representations for regression tasks. We define representation-rate and derive limits on the reliability with which input-output…
Imaging systems are commonly described using resolution, contrast, and signal-to-noise ratio, but these quantities do not provide a general account of how physical transformations affect the flow of information. This paper introduces an…
General characterization of physical measurements is discussed within the framework of a classical information theory. Uncertainty relation for simultaneous measurements of two physical observables is defined in this framework for…
Finding interdependency relations between (possibly multivariate) time series provides valuable knowledge about the processes that generate the signals. Information theory sets a natural framework for non-parametric measures of several…
This explanation of what a brain is and does rests on informational first principles, because information theory, like its parent theory thermodynamics, is mathematically sacrosanct, itself resting on real-valued probability.Just as…
Entropic uncertainty relations are interesting in their own rights as well as for a lot of applications. Keeping this in mind, we try to make the corresponding inequalities as tight as possible. The use of parametrized entropies also allows…
We review of the interface between (theoretical) physics and information for non-experts. The origin of information as related to the notion of entropy is described, first in the context of thermodynamics then in the context of statistical…
Life depends as much on the flow of information as on the flow of energy. Here we review the many efforts to make this intuition precise. Starting with the building blocks of information theory, we explore examples where it has been…
In mathematics information is a number that measures uncertainty (entropy) based on a probabilistic distribution, often of an obscure origin. In real life language information is a datum, a statement, more precisely, a formula. But such a…
The tension between qualitative theorizing and quantitative methods is pervasive in the social sciences, and poses a constant challenge to empirical research. But in science studies as an interdisciplinary specialty, there are additional…
Forecasting techniques for assessing the power of future experiments to discriminate between theories or discover new laws of nature are of great interest in many areas of science. In this paper, we introduce a Bayesian forecasting method…
This chapter provides a comprehensive and self-contained discussion of the most recent developments of information theory of networks. Maximum entropy models of networks are the least biased ensembles enforcing a set of constraints and are…
Information processing in neural systems can be described and analysed at multiple spatiotemporal scales. Generally, information at lower levels is more fine-grained and can be coarse-grained in higher levels. However, information processed…
We propose a unified theoretical framework for quantifying spatio-temporal interactions in a stochastic dynamical system based on information geometry. In the proposed framework, the degree of interactions is quantified by the divergence…
The advantages of quantum information processing are in many cases obtained as consequences of quantum interactions, especially for computational tasks where two-qubit interactions are essential. In this work, we establish the framework of…
A direct connection of information entropy $S$ and kinetic energy $T$ is obtained for nuclei and atomic clusters, which establishes $T$ as a measure of the information in a distribution. It is conjectured that this is a universal property…
The participation coefficient is a widely used metric of the diversity of a node's connections with respect to a modular partition of a network. An information-theoretic formulation of this concept of connection diversity, referred to here…
We propose an information theoretic model for sociological networks. The model is a microcanonical ensemble of states and particles. The states are the possible pairs of nodes (i.e. people, sites and alike) which exchange information. The…
Information theory plays a central role in establishing fundamental limits on what any learning or estimation algorithm can -- and cannot -- achieve, regardless of computational power. In this chapter, we provide an introduction to these…