Related papers: Information density, structure and entropy in equi…
Organizations generate vast amounts of interconnected content across various platforms. While language models enable sophisticated reasoning for use in business applications, retrieving and contextualizing information from organizational…
We investigate fundamental connections between thermodynamics and quantum information theory. First, we show that the operational framework of thermal operations is nonequivalent to the framework of Gibbs-preserving maps, and we comment on…
Entropy is a classical measure to quantify the amount of information or complexity of a system. Various entropy-based measures such as functional and spectral entropies have been proposed in brain network analysis. However, they are less…
The ordinal approach to evaluate time series due to innovative works of Bandt and Pompe has increasingly established itself among other techniques of nonlinear time series analysis. In this paper, we summarize and generalize the theory of…
Most of the existing classification methods are aimed at minimization of empirical risk (through some simple point-based error measured with loss function) with added regularization. We propose to approach this problem in a more information…
The relationship between entropy and information is reviewed, taking into account that information is stored in macroscopic degrees of freedom, such as the order parameter in a system exhibiting spontaneous symmetry breaking. It is shown…
We develop the framework of classical Observational entropy, which is a mathematically rigorous and precise framework for non-equilibrium thermodynamics, explicitly defined in terms of a set of observables. Observational entropy can be seen…
The application of the Shannon entropy to study the relationship between information and structures has yielded insights into molecular and material systems. However, the difficulty in directly observing and manipulating atoms and molecules…
Entropy can signify different things: For instance, heat transfer in thermodynamics or a measure of information in data analysis. Many entropies have been introduced and it can be difficult to ascertain their different importance and…
Accurate information processing is crucial both in technology and in nature. To achieve it, any information processing system needs an initial supply of resources away from thermal equilibrium. Here we establish a fundamental limit on the…
Entropy production is arguably the most universally applicable measure of non-equilibrium behavior, particularly for systems coupled to a heat bath. This setting encompasses driven soft matter as well as biomolecular, biochemical, and…
Observational entropy is interpreted as the uncertainty an observer making measurements associates with a system. So far, properties that make such an interpretation possible rely on the assumption of ideal projective measurements. We show…
In this work we investigate the information loss in (nonlinear) dynamical input-output systems and provide some general results. In particular, we present an upper bound on the information loss rate, defined as the (non-negative) difference…
In inverse problems, one attempts to infer spatially variable functions from indirect measurements of a system. To practitioners of inverse problems, the concept of "information" is familiar when discussing key questions such as which parts…
There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the…
We propose a new perspective on Turbulence using Information Theory. We compute the entropy rate of a turbulent velocity signal and we particularly focus on its dependence on the scale. We first report how the entropy rate is able to…
Entropy and information provide natural measures of correlation among elements in a network. We construct here the information theoretic analog of connected correlation functions: irreducible $N$--point correlation is measured by a decrease…
Finding interdependency relations between (possibly multivariate) time series provides valuable knowledge about the processes that generate the signals. Information theory sets a natural framework for non-parametric measures of several…
We study nonequilibrium thermodynamics of complex information flows induced by interactions between multiple fluctuating systems. Characterizing nonequilibrium dynamics by causal networks (i.e., Bayesian networks), we obtain novel…
In this article, we review a general theoretical framework of thermodynamics of information on the basis of Bayesian networks. This framework can describe a broad class of nonequilibrium dynamics of multiple interacting systems with complex…