相关论文: Intensional Inheritance Between Concepts: An Infor…
In this paper, we present an alternative approach to multiple inheritance for typed feature structures. In our approach, a feature structure can be associated with several types coming from different hierarchies (dimensions). In case of…
Integrated information theory (IIT) argues that the substrate of consciousness is a maximally irreducible complex of units. Together, subsets of the complex specify a cause-effect structure, composed of distinctions and their relations,…
I introduce a new approach to semantic information based upon the influence of erasure operations (interventions) upon distributions of a system's future trajectories through its phase space. Semantic (meaningful) information is…
The explicit link between Promise Theory and Information Theory, while perhaps obvious, is laid out explicitly here. It's shown how causally related observations of promised behaviours relate to the probabilistic formulation of causal…
Interpretability is an elusive but highly sought-after characteristic of modern machine learning methods. Recent work has focused on interpretability via $\textit{explanations}$, which justify individual model predictions. In this work, we…
Information propagation characterizes how input correlations evolve across layers in deep neural networks. This framework has been well studied using mean-field theory, which assumes infinitely wide networks. However, these assumptions…
This article introduces the physics of information in the context of molecular biology and genomics. Entropy and information, the two central concepts of Shannon's theory of information and communication, are often confused with each other…
This paper represents an extended version of an earlier note [10]. The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. We analyse analogs of…
Interpretability is a pressing issue for machine learning. Common approaches to interpretable machine learning constrain interactions between features of the input, rendering the effects of those features on a model's output comprehensible…
Information Theory concepts and methodologies conform the background of how communication systems are studied and understood. They are mainly focused on the source-channel-receiver problem and on the asymptotic limits of accuracy and…
We try to establish a unified information theoretic approach to learning and to explore some of its applications. First, we define {\em predictive information} as the mutual information between the past and the future of a time series,…
The weighted entropy $H^{\rm w}_\phi (X)=H^{\rm w}_\phi (f)$ of a random variable $X$ with values $x$ and a probability-mass/density function $f$ is defined as the mean value ${\mathbb E} I^{\rm w}_\phi(X)$ of the weighted information…
To build intelligent machine learning systems, there are two broad approaches. One approach is to build inherently interpretable models, as endeavored by the growing field of causal representation learning. The other approach is to build…
A new interpretation for the wavelet analysis is reported, which can is viewed as an information processing technique. It was recently proposed that every basic wavelet could be associated with a proper probability density, allowing…
It is high time to openly and without finalism define the dangerous but needed term 'purposeful information', whose quantity is an Eigen information value. Using the term 'biological information' in its stead forces one into an…
In the 21st century, many of the crucial scientific and technical issues facing humanity can be understood as problems associated with understanding, modelling, and ultimately controlling complex systems: systems comprised of a large number…
This overview of integrated information theory (IIT) emphasizes IIT's "consciousness-first" approach to what exists. Consciousness demonstrates to each of us that something exists--experience--and reveals its essential properties--the…
Conventional machine learning methods are predominantly designed to predict outcomes based on a single data type. However, practical applications may encompass data of diverse types, such as text, images, and audio. We introduce…
The notion of information pervades informal descriptions of biological systems, but formal treatments face the problem of defining a quantitative measure of information rooted in a concept of fitness, which is itself an elusive notion.…
We introduce instancewise feature selection as a methodology for model interpretation. Our method is based on learning a function to extract a subset of features that are most informative for each given example. This feature selector is…