相关论文: Measuring complexity with zippers

We define a one-parameter family of entropies, each assigning a real number to any probability measure on a compact metric space (or, more generally, a compact Hausdorff space with a notion of similarity between points). These entropies…

We present a simple adaptation of the Lempel Ziv 78' (LZ78) compression scheme ({\em IEEE Transactions on Information Theory, 1978}) that supports efficient random access to the input string. Namely, given query access to the compressed…

After reviewing unnormalized and normalized information distances based on incomputable notions of Kolmogorov complexity, we discuss how Kolmogorov complexity can be approximated by data compression algorithms. We argue that optimal…

In signal analysis and synthesis, linear approximation theory considers a linear decomposition of any given signal in a set of atoms, collected into a so-called dictionary. Relevant sparse representations are obtained by relaxing the…

Information theory on a time-discrete setting in the framework of time series analysis is generalized to the time-continuous case. Considerations of the Roessler and Lorenz dynamics as well as the Ornstein-Uhlenbeck process yield for…

We review the properties of the quantum relative entropy function and discuss its application to problems of classical and quantum information transfer and to quantum data compression. We then outline further uses of relative entropy to…

Entropy can signify different things: For instance, heat transfer in thermodynamics or a measure of information in data analysis. Many entropies have been introduced and it can be difficult to ascertain their different importance and…

There are three ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a…

In this paper we present a general method for information extraction that exploits the features of data compression techniques. We first define and focus our attention on the so-called "dictionary" of a sequence. Dictionaries are…

We describe a universal information compression scheme that compresses any pure quantum i.i.d. source asymptotically to its von Neumann entropy, with no prior knowledge of the structure of the source. We introduce a diagonalisation…

A key tenet of the Transactional Interpretation of Quantum Mechanics is the idea that photon absorption localizes the absorbing material system. In doing so, it measures the location of the absorber and hence reduces information entropy,…

This paper is a review of a particular approach to the method of maximum entropy as a general framework for inference. The discussion emphasizes the pragmatic elements in the derivation. An epistemic notion of information is defined in…

Entropy is a fundamental property of a repertoire. Here, we present an efficient algorithm to estimate the entropy of types with the help of Zhang's estimator. The algorithm takes advantage of the fact that the number of different…

By introducing Hilbert space and operators, we show how probabilities, approximations and entropy encoding from signal and image processing allow precise formulas and quantitative estimates. Our main results yield orthogonal bases which…

We generalize the Point information gain (PIG) and derived quantities, i.e. Point information entropy (PIE) and Point information entropy density (PIED), for the case of R\'enyi entropy and simulate the behavior of PIG for typical…

In this article we present a methodology to estimate the Transfer Entropy Rate between two systems through the Lempel-Ziv complexity. This methodology carries a set of practical advantages: it can be estimated from two single discrete…

The rapid scaling of artificial intelligence models has revealed a fundamental tension between model capacity (storage) and inference efficiency (computation). While classical information theory focuses on transmission and storage limits,…

The aim of this note is to provide some reference facts for LZW---mostly from Thomas and Cover \cite{Cover:2006aa} and provide a reference for some metrics that can be derived from it. LZW is an algorithm to compute a Kolmogorov Complexity…

The notion of entropy appears in many fields and this paper is a survey about entropies in several branches of Mathematics. We are mainly concerned with the topological and the algebraic entropy in the context of continuous endomorphisms of…

Entropy has emerged as a dynamic, interdisciplinary, and widely accepted quantitative measure of uncertainty across different disciplines. A unified understanding of entropy measures, supported by a detailed review of their theoretical…