Related papers: Information Measures for Deterministic Input-Outpu…
Configurational information is generated when three or more sources of variance interact. The variations not only disturb each other relationally, but by selecting upon each other, they are also positioned in a configuration. A…
A unified combinatorial definition of the information content and entropy of different types of patterns, compatible with the traditional concepts of information and entropy, going beyond the limitations of Shannon information interpretable…
The information reconstruction problem on an infinite tree, is to collect and analyze massive data samples at the $n$th level of the tree to identify whether there is non-vanishing information of the root, as $n$ goes to infinity. This…
In this paper, we analyze the monotonicity of information aging in a remote estimation system, where historical observations of a Gaussian autoregressive AR(p) process are used to predict its future values. We consider two widely used loss…
Entropy governs molecular self-assembly, phase transitions, and material stability, yet remains challenging to quantify and directly control in molecular systems. Here, we demonstrate that the computable information density (CID), a data…
Understanding a complex system entails capturing the non-trivial collective phenomena that arise from interactions between its different parts. Information theory is a flexible and robust framework to study such behaviours, with several…
The purpose of this paper is to formalize the concept that best synthesizes our intuitive understanding of quantum mechanics -- that the information carried by a system is limited -- and, from this principle, to construct the foundations of…
An information theory description of finite systems explicitly evolving in time is presented. We impose a MaxEnt variational principle on the Shannon entropy at a given time while the constraints are set at a former time. The resulting…
In many real-world dynamical systems, obtaining precise models of system uncertainty remains a challenge. It may be difficult to estimate noise distributions or robustness bounds, especially when the distributions/robustness bounds vary…
The achievable information rate of finite-state input two-dimensional (2-D) channels with memory is an open problem, which is relevant, e.g., for inter-symbol-interference (ISI) channels and cellular multiple-access channels. We propose a…
Given a universe of discourse X-a domain of possible outcomes-an experiment may consist of selecting one of its elements, subject to the operation of chance, or of observing the elements, subject to imprecision. A priori uncertainty about…
This paper addresses a new approach to find a spectrum of information measures for the process of digital circuit synthesis. We consider the problem from the information engine point of view. The circuit synthesis as a whole and different…
In this paper, we expand the theory of depth-unbiased source localization to unbiased parameter estimation and signal reconstruction of an arbitrary number of non-zero parameters to be recovered. The topic touches on the concept of exact…
Complex systems are characterised by a tight, nontrivial interplay of their constituents, which gives rise to a multi-scale spectrum of emergent properties. In this scenario, it is practically and conceptually difficult to identify those…
We address the problem of uncertainty quantification and propose measures of total, aleatoric, and epistemic uncertainty based on a known decomposition of (strictly) proper scoring rules, a specific type of loss function, into a divergence…
Structures are abundant in both natural and human-made environments and usually studied in the form of images or scattering patterns. To characterize structures a huge variety of descriptors is available spanning from porosity to radial and…
This paper applies the recently axiomatized Optimum Information Principle (minimize the Kullback-Leibler information subject to all relevant information) to nonparametric density estimation, which provides a theoretical foundation as well…
The generalization error of a learning algorithm refers to the discrepancy between the loss of a learning algorithm on training data and that on unseen testing data. Various information-theoretic bounds on the generalization error have been…
In this essay, a general case of information systems contains quantum information systems is considered. By presenting an algorithmic method a new kind of information topology is defined and considered. Continuous maps between two…
Information-theoretic quantities like entropy and mutual information have found numerous uses in machine learning. It is well known that there is a strong connection between these entropic quantities and submodularity since entropy over a…