相关论文: Predictive Information
We try to establish a unified information theoretic approach to learning and to explore some of its applications. First, we define {\em predictive information} as the mutual information between the past and the future of a time series,…
We define {\em predictive information} $I_{\rm pred} (T)$ as the mutual information between the past and the future of a time series. Three qualitatively different behaviors are found in the limit of large observation times $T$: $I_{\rm…
Prediction of events is the challenge in many different disciplines, from meteorology to finance; the more this task is difficult, the more a system is {\it complex}. Nevertheless, even according to this restricted definition, a general…
We propose predictive information, that is information between a long past of duration T and the entire infinitely long future of a time series, as a universal order parameter to study phase transitions in physical systems. It can be used,…
A prediction makes a claim about a system's future given knowledge of its past. A retrodiction makes a claim about its past given knowledge of its future. The bidirectional machine is an ambidextrous hidden Markov chain that does both…
The study of complex systems has attracted widespread attention from researchers in the fields of natural sciences, social sciences, and engineering. Prediction is one of the central issues in this field. Although most related studies have…
A general notion of information-related complexity applicable to both natural and man-made systems is proposed. The overall approach is to explicitly consider a rational agent performing a certain task with a quantifiable degree of success.…
Despite the obvious advantage of simple life forms capable of fast replication, different levels of cognitive complexity have been achieved by living systems in terms of their potential to cope with environmental uncertainty. Against the…
Construction methods for prior densities are investigated from a predictive viewpoint. Predictive densities for future observables are constructed by using observed data. The simultaneous distribution of future observables and observed data…
Reliable estimation of predictive uncertainty is crucial for machine learning applications, particularly in high-stakes scenarios where hedging against risks is essential. Despite its significance, there is no universal agreement on how to…
Information theory is widely accepted as a powerful tool for analyzing complex systems and it has been applied in many disciplines. Recently, some central components of information theory - multivariate information measures - have found…
We introduce an information theoretic measure of statistical structure, called 'binding information', for sets of random variables, and compare it with several previously proposed measures including excess entropy, Bialek et al.'s…
Information Theory provides a fundamental basis for analysis, and for a variety of subsequent methodological approaches, in relation to uncertainty quantification. The transversal character of concepts and derived results justifies its…
Traditional statistical theory assumes that the analysis to be performed on a given data set is selected independently of the data themselves. This assumption breaks downs when data are re-used across analyses and the analysis to be…
Complex systems are found in most branches of science. It is still argued how to best quantify their complexity and to what end. One prominent measure of complexity (the statistical complexity) has an operational meaning in terms of the…
We present a theory of information expressed solely in terms of which transformations of physical systems are possible and which are impossible - i.e. in constructor-theoretic terms. Although it includes conjectured laws of physics that are…
We propose and study a system whose dynamics are governed by predictions of its future states. General formalism and concrete examples are presented. We find that the dynamical characteristics depend on both how to shape predictions as well…
We review possible measures of complexity which might in particular be applicable to situations where the complexity seems to arise spontaneously. We point out that not all of them correspond to the intuitive (or "naive") notion, and that…
We introduce an ambidextrous view of stochastic dynamical systems, comparing their forward-time and reverse-time representations and then integrating them into a single time-symmetric representation. The perspective is useful theoretically,…
The concept of information has emerged as a language in its own right, bridging several disciplines that analyze natural phenomena and man-made systems. Integrated information has been introduced as a metric to quantify the amount of…