Related papers: Using Information Theory to Measure Psychophysical…
We characterize information as risk reduction between knowledge states represented by partitions of the underlying probability space. Entropy corresponds to risk reduction from no (or partial) knowledge to full knowledge about a random…
While most useful information theoretic inequalities can be deduced from the basic properties of entropy or mutual information, Shannon's entropy power inequality (EPI) seems to be an exception: available information theoretic proofs of the…
The cohesiveness of response to external stimuli depends on rapid distortion-free information transfer across the network. Aligning with the information from the network has been used to model such information transfer. Nevertheless, the…
The principles of statistical mechanics and information theory play an important role in learning and have inspired both theory and the design of numerous machine learning algorithms. The new aspect in this paper is a focus on integrating…
Information theory is a statistical theory concerned with the relative state of detectors and physical systems. As a consequence, the classical framework of Shannon needs to be extended to deal with quantum detectors, possibly moving at…
Shannon information theory provides various measures of so-called "syntactic information", which reflect the amount of statistical correlation between systems. In contrast, the concept of "semantic information" refers to those correlations…
Shannon information theory is established based on probability and bits, and the communication technology based on this theory realizes the information age. The original goal of Shannon's information theory is to describe and transmit…
Higher-order information theory has become a rapidly growing toolkit in computational neuroscience, motivated by the idea that multivariate dependencies can reveal aspects of neural computation and communication that are invisible to…
Finding interdependency relations between (possibly multivariate) time series provides valuable knowledge about the processes that generate the signals. Information theory sets a natural framework for non-parametric measures of several…
Does semantic communication require a semantic information theory parallel to Shannon's information theory, or can Shannon's work be generalized for semantic communication? This paper advocates for the latter and introduces a semantic…
Mutual information is widely used in artificial intelligence, in a descriptive way, to measure the stochastic dependence of discrete random variables. In order to address questions such as the reliability of the empirical value, one must…
Mutual information is widely used in artificial intelligence, in a descriptive way, to measure the stochastic dependence of discrete random variables. In order to address questions such as the reliability of the empirical value, one must…
I present a new approach for the interpretation of reaction time (RT) data from behavioral experiments. From a physical perspective, the entropy of the RT distribution provides a model-free estimate of the amount of processing performed by…
Information decompositions quantify how the Shannon information about a given random variable is distributed among several other random variables. Various requirements have been proposed that such a decomposition should satisfy, leading to…
We study trade-off relations in information extraction from quantum systems subject to null-result weak measurements, where the absence of a detected photon continuously updates the system state. We present a detailed analysis of qubit and…
The notion of information pervades informal descriptions of biological systems, but formal treatments face the problem of defining a quantitative measure of information rooted in a concept of fitness, which is itself an elusive notion.…
In our digital and connected societies, the development of social networks, online shopping, and reputation systems raises the question of how individuals use social information, and how it affects their decisions. We report experiments…
For biological experiments aiming at calibrating models with unknown parameters, a good experimental design is crucial, especially for those subject to various constraints, such as financial limitations, time consumption and physical…
This paper I assume that in humans the creation of knowledge depends on a discrete time, or stage, sequential decision-making process subjected to a stochastic, information transmitting environment. For each time-stage, this environment…
We relate the information exchange between two stochastic systems to the nonequilibrium entropy production in the whole system. By deriving a general formula that decomposes the total entropy production into the thermodynamic and…