Related papers: Information temperature as a parameter of random s…
Quantum complexity measures the difficulty of realizing a quantum process, such as preparing a state or implementing a unitary. We present an approach to quantifying the thermodynamic resources required to implement a process if the…
In communications, unknown variables are usually modelled as random variables, and concepts such as independence, entropy and information are defined in terms of the underlying probability distributions. In contrast, control theory often…
In the analysis of any type of system, granting maximum information extraction from its data is non-trivial. Confidence in successful information extraction typically builds on prior knowledge of the studied system or on the user's…
The information spectrum approach gives general formulae for optimal rates of codes in many areas of information theory. In this paper the quantum spectral divergence rates are defined and properties of the rates are derived. The entropic…
While Kolmogorov complexity is the accepted absolute measure of information content in an individual finite object, a similarly absolute notion is needed for the information distance between two individual objects, for example, two…
Diversity is an essential metric for evaluating the creativity of outputs generated by language models. Temperature-based sampling is a common strategy to increase diversity. However, for tasks that require high precision, e.g.,…
Information distance is a parameter-free similarity measure based on compression, used in pattern recognition, data mining, phylogeny, clustering, and classification. The notion of information distance is extended from pairs to multiples…
The mutual information between stimulus and spike-train response is commonly used to monitor neural coding efficiency, but neuronal computation broadly conceived requires more refined and targeted information measures of input-output joint…
Markov random fields area popular model for high-dimensional probability distributions. Over the years, many mathematical, statistical and algorithmic problems on them have been studied. Until recently, the only known algorithms for…
Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…
Information capacity of a symbol sequence is a measure of the unexpectedness of a continuation of given string of symbols. Continuation of a string is determined through the maximum entropy of the reconstructed frequency dictionary; the…
Stochastic chains represent a wide and key variety of phenomena in many branches of science within the context of Information Theory and Thermodynamics. They are typically approached by a sequence of independent events or by a memoryless…
Quantitative measure of disorder or randomness based on the entropy production characterizes thermodynamical irreversibility, which is relevant to the conventional second law of thermodynamics. Here we report, in a quantum mechanical…
We show a relationship between the entropy production in stochastic thermodynamics and the stochastic interaction in the information integrated theory. To clarify this relationship, we newly introduce an information geometric interpretation…
There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a…
Information thermodynamics relates the rate of change of mutual information between two interacting subsystems to their thermodynamics when the joined system is described by a bipartite stochastic dynamics satisfying local detailed balance.…
We use a formal correspondence between thermodynamics and inference, where the number of samples can be thought of as the inverse temperature, to study a quantity called ``learning capacity'' which is a measure of the effective…
We present a new method of estimating the dispersion of a distribution which is based on the surprising property of a function that measures information processing intensity. It turns out that this function has a maximum at its fixed point.…
Complex systems are found in most branches of science. It is still argued how to best quantify their complexity and to what end. One prominent measure of complexity (the statistical complexity) has an operational meaning in terms of the…
We translate the problem of calculating the entropy of a set of binary configurations/signals into a sequence of supervised classification tasks. Subsequently, one can use virtually any machine learning classification algorithm for…