Related papers: Information Loss in Static Nonlinearities
We propose an entropy measure for the analysis of chaotic attractors through recurrence networks which are un-weighted and un-directed complex networks constructed from time series of dynamical systems using specific criteria. We show that…
In communications, unknown variables are usually modelled as random variables, and concepts such as independence, entropy and information are defined in terms of the underlying probability distributions. In contrast, control theory often…
In biological systems, information is frequently transferred with Poisson like spike processes (shot noise) modulated in time by information-carrying signals. How then to quantify information transfer for the output for such nonstationary…
This paper considers the subject of information losses arising from the finite datasets used in the training of neural classifiers. It proves a relationship between such losses as the product of the expected total variation of the estimated…
The information shared among observables representing processes of interest is traditionally evaluated in terms of macroscale measures characterizing aggregate properties of the underlying processes and their interactions. Traditional…
Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…
Despite the wide usage of information as a concept in science, we have yet to develop a clear & concise scientific definition. This paper is aimed at laying the foundations for a new theory concerning the mechanics of information alongside…
The Tsallis entropy and Fisher information entropy (matrix) are very important quantities expressing information measures in nonextensive systems. Stationary and dynamical properties of the information entropies have been investigated in…
Renewal processes are broadly used to model stochastic behavior consisting of isolated events separated by periods of quiescence, whose durations are specified by a given probability law. Here, we identify the minimal sufficient statistic…
In the study of open quantum systems, information exchange between system and its surrounding environment plays an eminent and important role in analysing the dynamics of open quantum system. In this work, by making use of the quantum…
We propose a partial information decomposition based on the newly introduced framework of causal tensors, i.e., multilinear stochastic maps that transform source data into destination data. This framework enables us to express an indirect…
We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and…
Transfer entropy is a widely used measure for quantifying directed information flows in complex systems. While the challenges of estimating transfer entropy for continuous data are well known, it has two major shortcomings for data of…
Artificial intelligence models and methods commonly lack causal interpretability. Despite the advancements in interpretable machine learning (IML) methods, they frequently assign importance to features which lack causal influence on the…
Entropy notions for $\varepsilon$-incremental practical stability and incremental stability of deterministic nonlinear systems under disturbances are introduced. The entropy notions are constructed via a set of points in state space which…
We describe how to analyze the wide class of non stationary processes with stationary centered increments using Shannon information theory. To do so, we use a practical viewpoint and define ersatz quantities from time-averaged probability…
We study the problem of system identification for stochastic continuous-time dynamics, based on a single finite-length state trajectory. We present a method for estimating the possibly unstable open-loop matrix by employing properly…
We study the information loss of a class of inference strategies that is solely based on time averaging. For an array of independent binary sensors (e.g., receptors, single electron transistors) measuring a weak random signal (e.g., ligand…
A method for correcting for detector smearing effects using machine learning techniques is presented. Compared to the standard approaches the method can use more than one reconstructed variable to infere the value of the unsmeared quantity…
Information theory provides tools to predict the performance of a learning algorithm on a given dataset. For instance, the accuracy of learning an unknown parameter can be upper bounded by reducing the learning task to hypothesis testing…