Related papers: Information Measures for Deterministic Input-Outpu…

Obtaining meaningful quantitative descriptions of the statistical dependence within multivariate systems is a difficult open problem. Recently, the Partial Information Decomposition (PID) was proposed to decompose mutual information (MI)…

Trustworthy artificial intelligence increasingly relies on probabilistic computation to achieve robustness, interpretability, security and privacy. In practical systems, such workloads interleave deterministic data access with repeated…

Constraints on entropies are considered to be the laws of information theory. Even though the pursuit of their discovery has been a central theme of research in information theory, the algorithmic aspects of constraints on entropies remain…

The quantification of aleatoric and epistemic uncertainty in terms of conditional entropy and mutual information, respectively, has recently become quite common in machine learning. While the properties of these measures, which are rooted…

Information theory has become an increasingly important research field to better understand quantum mechanics. Noteworthy, it covers both foundational and applied perspectives, also offering a common technical language to study a variety of…

There is a vast body of recent literature on the reliability of communication through noisy channels, the recovery of community structures in the stochastic block model, the limiting behavior of the free entropy in spin glasses and the…

The matrix-based R\'enyi's entropy allows us to directly quantify information measures from given data, without explicit estimation of the underlying probability distribution. This intriguing property makes it widely applied in statistical…

We present a possible approach to the study of the renormalization group (RG) flow based entirely on the information theory. The average information loss under a single step of Wilsonian RG transformation is evaluated as a conditional…

Rather than measuring NP search in terms of Turing-machine time, we reinterpret witness recovery as an information-acquisition process: the hidden witness is the sole source of uncertainty, and identification requires sufficient reduction…

In this paper we derive information theoretic performance bounds to sensing and reconstruction of sparse phenomena from noisy projections. We consider two settings: output noise models where the noise enters after the projection and input…

Much of uncertainty quantification to date has focused on determining the effect of variables modeled probabilistically, and with a known distribution, on some physical or engineering system. We develop methods to obtain information on the…

The characterisation of information processing is an important task in complex systems science. Information dynamics is a quantitative methodology for modelling the intrinsic information processing conducted by a process represented as a…

We show that a certain type of quasi finite, conservative, ergodic, measure preserving transformation always has a maximal zero entropy factor, generated by predictable sets. We also construct a conservative, ergodic, measure preserving…

In the dynamics of open quantum systems, the backflow of information to the reduced system under study has been suggested as the actual physical mechanism inducing memory and thus leading to non-Markovian quantum dynamics. To this aim, the…

Information theory, rooted in computer science, and many-body physics, have traditionally been studied as (almost) independent fields. Only recently has this paradigm started to shift, with many-body physics being studied and characterized…

Avoiding overfitting is a central challenge in machine learning, yet many large neural networks readily achieve zero training loss. This puzzling contradiction necessitates new approaches to the study of overfitting. Here we quantify…

Here, we investigate the uncertainty of dynamical observables in classical systems manipulated by repeated measurements and feedback control; the precision should be enhanced in the presence of an external controller but limited by the…

Knowing and guessing, these are two essential epistemological pillars in the theory of quantum-mechanical measurement. As formulated quantum mechanics is a statistical theory. In general, a priori unknown states can be completely determined…

Making decisions freely presupposes that there is some indeterminacy in the environment and in the decision making engine. The former is reflected on the behavioral changes due to communicating: few changes indicate rigid environments;…

(abridged) In this paper, we present the issues we consider as essential as far as the statistical mechanics of finite systems is concerned. In particular, we emphasis our present understanding of phase transitions in the framework of…