Related papers: Information entropy re-defined in a category theor…
The matrix-based Renyi's \alpha-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the current theory in the…
Bounds on information combining are entropic inequalities that determine how the information, or entropy, of a set of random variables can change when they are combined in certain prescribed ways. Such bounds play an important role in…
Algebraic statistics is a recently evolving field, where one would treat statistical models as algebraic objects and thereby use tools from computational commutative algebra and algebraic geometry in the analysis and computation of…
These lecture notes concern information-theoretic notions of entropy. They are intended for, and have been successfully taught to, undergraduate students interested inresearch careers. Besides basic notions of analysis related to…
Entropy might be a not well defined concept if the system can undergo transformations involving stationary nonequilibria. It might be analogous to the heat content (once called ``caloric'') in transformations that are not isochoric (i.e.…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…
The thermodynamic definition of entropy can be extended to nonequilibrium systems based on its relation to information. To apply this definition in practice requires access to the physical system's microstates, which may be prohibitively…
We introduce a new class of locally compact groups, namely the strongly compactly covered groups, which are the Hausdorff topological groups $G$ such that every element of $G$ is contained in a compact open normal subgroup of $G$. For…
Shannon's metric of "Entropy" of information is a foundational concept of information theory. This article is a primer for novices that presents an intuitive way of understanding, remembering, and/or reconstructing Shannon's Entropy metric…
Entropy is a very useful concept from physics that tries to explain how a system behaves from a point of view of the thermodynamics. However, there are two ways to explain entropy, and it depends on if we are studying a microsystem or a…
A recently explored interesting quantity in AdS/CFT, dubbed 'residual entropy', characterizes the amount of collective ignorance associated with either boundary observers restricted to finite time duration, or bulk observers who lack access…
The entropy of an orthogonal matrix is defined. It provides a new interpretation of Hadamard matrices as those that saturate the bound for entropy.It appears to be a useful Morse function on the group manifold. It has sharp maxima and other…
Predictive inference requires balancing statistical accuracy against informational complexity, yet the choice of complexity measure is usually imposed rather than derived. We treat econometric objects as predictive rules, mappings from…
We shall prove that the celebrated R\'enyi entropy is the first example of a new family of infinitely many multi-parametric entropies. We shall call them the $Z$-entropies. Each of them, under suitable hypotheses, generalizes the celebrated…
In this note I take the opportunity to correct the last statement of Part I of same title and continue the study of uniform purity of epimorphisms in order to derive the main result, which states that--provided $R_R\in \langle\cal…
We present a unifying approach to the study of entropies in Mathematics, such as measure entropy, topological entropy, algebraic entropy, set-theoretic entropy. We take into account discrete dynamical systems, that is, pairs $(X,T)$, where…
We introduce the category of information structures, whose objects are suitable diagrams of measurable sets that encode the possible outputs of a given family of observables and their mutual relationships of refinement; they serve as…
In this paper, we develop a framework for information theoretic learning based on infinitely divisible matrices. We formulate an entropy-like functional on positive definite matrices based on Renyi's axiomatic definition of entropy and…
We define {\em predictive information} $I_{\rm pred} (T)$ as the mutual information between the past and the future of a time series. Three qualitatively different behaviors are found in the limit of large observation times $T$: $I_{\rm…
Entropy rate is a real valued functional on the space of discrete random sources which lacks a closed formula even for subclasses of sources which have intuitive parameterizations. A good way to overcome this problem is to examine its…