Related papers: Conspiratorial beliefs observed through entropy pr…
We define correlational (von Neumann) entropy for an individual quantum state of a system whose time-independent hamiltonian contains random parameters and is treated as a member of a statistical ensemble. This entropy is representation…
Probability theory is fundamental for modeling uncertainty, with traditional probabilities being real and non-negative. Complex probability extends this concept by allowing complex-valued probabilities, opening new avenues for analysis in…
There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the…
Entropy estimation is of practical importance in information theory and statistical science. Many existing entropy estimators suffer from fast growing estimation bias with respect to dimensionality, rendering them unsuitable for…
An information-theoretic framework is developed to assess the predictability of ENSO complexity, which is a central problem in contemporary meteorology with large societal impacts. The information theory advances a unique way to quantify…
The hallmark of deterministic chaos is that it creates information---the rate being given by the Kolmogorov-Sinai metric entropy. Since its introduction half a century ago, the metric entropy has been used as a unitary quantity to measure a…
We propose an information-theoretic framework to measure narratives, providing a formalism to understand pivotal moments, cliffhangers, and plot twists. This approach offers creatives and AI researchers tools to analyse and benchmark human-…
Systematic generalization remains challenging for current language models, which are known to be both sensitive to semantically similar permutations of the input and to struggle with known concepts presented in novel contexts. Although…
Can we learn more from data than existed in the generating process itself? Can new and useful information be constructed from merely applying deterministic transformations to existing data? Can the learnable content in data be evaluated…
Most of the existing classification methods are aimed at minimization of empirical risk (through some simple point-based error measured with loss function) with added regularization. We propose to approach this problem in a more information…
We study conditional mutual information (cMI) between a pair of variables $X,Y$ given a third one $Z$ and derived quantities including transfer entropy (TE) and causation entropy (CE) in the dynamically relevant context where $X=T(Y,Z)$ is…
Traditional works on community detection from observations of information cascade assume that a single adjacency matrix parametrizes all the observed cascades. However, in reality the connection structure usually does not stay the same…
With the help of transfer entropy, we analyze information flows between communities of complex networks. We show that the transfer entropy provides a coherent description of interactions between communities, including non-linear…
The approach defines information process from probabilistic observation, emerging microprocess,qubit, encoding bits, evolving macroprocess, and extends to Observer information self-organization, cognition, intelligence and understanding…
We show why the amount of information communicated between the past and future--the excess entropy--is not in general the amount of information stored in the present--the statistical complexity. This is a puzzle, and a long-standing one,…
Transfer entropy in information theory was recently demonstrated [Phys. Rev. E 102, 012404 (2020)] to enable us to elucidate the interaction domain among interacting elements solely from an ensemble of trajectories. There, only pairs of…
Infectious diseases are studied to understand their spreading mechanisms, to evaluate control strategies and to predict the risk and course of future outbreaks. Because people only interact with a small number of individuals, and because…
Kolmogorov argued that the concept of information exists also in problems with no underlying stochastic model (as Shannon's information representation) for instance, the information contained in an algorithm or in the genome. He introduced…
This paper addresses the problem of distributed learning of average belief with sequential observations, in which a network of $n>1$ agents aim to reach a consensus on the average value of their beliefs, by exchanging information only with…
Data stream mining problem has caused widely concerns in the area of machine learning and data mining. In some recent studies, ensemble classification has been widely used in concept drift detection, however, most of them regard…