Related papers: Maximally Informative Observables and Categorical …
A practical notation can convey valuable intuitions and concisely express new ideas. Information theory is of importance to machine learning, but the notation for information-theoretic quantities is sometimes opaque. We propose a practical…
An observable canonical form is formulated for the set of rational systems on a variety each of which is a single-input-single-output, affine in the input, and a minimal realization of its response map. The equivalence relation for the…
The maximal information coefficient (MIC) is a tool for finding the strongest pairwise relationships in a data set with many variables (Reshef et al., 2011). MIC is useful because it gives similar scores to equally noisy relationships of…
An estimate of the maximal informational content available to advanced extraterrestrial or future (post)human civilizations is presented. It is shown that the fundamental thermodynamical considerations may lead to a quantitative estimate of…
When evaluating causal influence from one time series to another in a multivariate dataset it is necessary to take into account the conditioning effect of the other variables. In the presence of many variables, and possibly of a reduced…
The program of a physical concept of information is outlined in the framework of quantum theory. A proposal is made for how to avoid the introduction of axiomatic observables. The conventional (collapse) and the Everett interpretations of…
I argue that, on a judicious reading of two existing criteria--one syntactic and the other semantic--dual theories can be taken to be empirically equivalent. The judicious reading is straightforward, but leads to the surprising conclusion…
We propose an information-theoretic framework to measure narratives, providing a formalism to understand pivotal moments, cliffhangers, and plot twists. This approach offers creatives and AI researchers tools to analyse and benchmark human-…
A type theory is presented that combines (intuitionistic) linear types with type dependency, thus properly generalising both intuitionistic dependent type theory and full linear logic. A syntax and complete categorical semantics are…
We consider the physical limitations imposed on the information content of an image by the wave and quantum nature of light, when the image is obtained by illuminating a reflecting or transmitting planar object by natural---i.e., fully…
Information theory has its particles, bits and qubits, just as physics has electrons and photons. However, in physics we have a special category of objects with no clear counterparts in information theory: quasiparticles. They are…
The information conveyed by a hierarchical attractor neural network is examined. The network learns sets of correlated patterns (the examples) in the lowest level of the hierarchical tree and can categorize them at the upper levels. A way…
A measure of dependence is said to be equitable if it gives similar scores to equally noisy relationships of different types. Equitability is important in data exploration when the goal is to identify a relatively small set of strongest…
We study the problem of choosing optimal policy rules in uncertain environments using models that may be incomplete and/or partially identified. We consider a policymaker who wishes to choose a policy to maximize a particular counterfactual…
Words (phrases or symbols) play a key role in human life. Word (phrase or symbol) representation is the fundamental problem for knowledge representation and understanding. A word (phrase or symbol) usually represents a name of a category.…
Information flow properties express the capability for an agent to infer information about secret behaviours of a partially observable system. In a language-theoretic setting, where the system behaviour is described by a language, we define…
How best to quantify the information of an object, whether natural or artifact, is a problem of wide interest. A related problem is the computability of an object. We present practical examples of a new way to address this problem. By…
The theory of optimal transportation has developed into a powerful and elegant framework for comparing probability distributions, with wide-ranging applications in all areas of science. The fundamental idea of analyzing probabilities by…
The best empirical research in political science clearly defines substantive parameters of interest, presents a set of assumptions that guarantee its identification, and uses an appropriate estimator. We argue for the importance of…
The increasing prevalence of political bias in news media calls for greater public awareness of it, as well as robust methods for its detection. While prior work in NLP has primarily focused on the lexical bias captured by linguistic…