Related papers: How Compressible are Innovation Processes?
In compressed sensing one measures sparse signals directly in a compressed form via a linear transform and then reconstructs the original signal. However, it is often the case that the linear transform itself is known only approximately, a…
Compressive Sensing (CS) exploits the surprising fact that the information contained in a sparse signal can be preserved in a small number of compressive, often random linear measurements of that signal. Strong theoretical guarantees have…
The aim of this note is to introduce a notion of dynamical entropy, which we call infinite-product entropy, for probability measures on (countable) infinite cartesian product of any measurable space with itself. The idea behind the…
We introduce the concept of {\em information compressibility}, $K_I$, which measures the relative change of number of available microstates of an open system in response to an energy variation. We then prove that at the time in which the…
Recovering sparse signals from linear measurements has demonstrated outstanding utility in a vast variety of real-world applications. Compressive sensing is the topic that studies the associated raised questions for the possibility of a…
Entropy and information can be considered dual: entropy is a measure of the subspace defined by the information constraining the given ambient space. Negative entropies, arising in na\"ive extensions of the definition of entropy from…
A new line of research uses compression methods to measure the similarity between signals. Two signals are considered similar if one can be compressed significantly when the information of the other is known. The existing compression-based…
Sparsity and entropy are pillar notions of modern theories in signal processing and information theory. However, there is no clear consensus among scientists on the characterization of these notions. Previous efforts have contributed to…
The characteristic functional is the infinite-dimensional generalization of the Fourier transform for measures on function spaces. It characterizes the statistical law of the associated stochastic process in the same way as a characteristic…
The notion of metric entropy dimension is introduced to measure the complexity of entropy zero dynamical systems. For measure preserving systems, we define entropy dimension via the dimension of entropy generating sequences. This…
Noise is conventionally viewed as a severe problem in diverse fields, e.g., engineering, learning systems. However, this paper aims to investigate whether the conventional proposition always holds. It begins with the definition of task…
Compressed sensing allows perfect recovery of sparse signals (or signals sparse in some basis) using only a small number of random measurements. Existing results in compressed sensing literature have focused on characterizing the achievable…
We propose a new way to measure the balance between freedom and coherence in a dynamical system and a new measure of its internal variability. Based on the concept of entropy and ideas from neuroscience and information theory, we define…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
Complex systems are found in most branches of science. It is still argued how to best quantify their complexity and to what end. One prominent measure of complexity (the statistical complexity) has an operational meaning in terms of the…
Subshifts of deterministic substitutions are ubiquitous objects in dynamical systems and aperiodic order (the mathematical theory of quasicrystals). Two of their most striking features are that they have low complexity (zero topological…
This article extends the concept of compressed sensing to signals that are not sparse in an orthonormal basis but rather in a redundant dictionary. It is shown that a matrix, which is a composition of a random matrix of certain type and a…
We extend the notion of estimation entropy of autonomous dynamical systems proposed by Liberzon and Mitra [1] to nonlinear dynamical systems with uncertain inputs with bounded variation. We call this new notion the {$\epsilon$}-estimation…
Information entropy is applied to the state of knowledge of reaction amplitudes in pseudoscalar meson photoproduction, and a scheme is developed that quantifies the information content of a measured set of polarization observables. It is…
The Random Permutation Set (RPS) is a new type of set proposed recently, which can be regarded as the generalization of evidence theory. To measure the uncertainty of RPS, the entropy of RPS and its corresponding maximum entropy have been…