Related papers: Incomplete descriptions and relevant entropies
Depending on context, the term entropy is used for a thermodynamic quantity, a~measure of available choice, a quantity to measure information, or, in the context of statistical inference, a maximum configuration predictor. For systems in…
Discovering relevant, but possibly hidden, variables is a key step in constructing useful and predictive theories about the natural world. This brief note explains the connections between three approaches to this problem: the recently…
We deal with a generalized statistical description of nonequilibrium complex systems based on least biased distributions given some prior information. A maximum entropy principle is introduced that allows for the determination of the…
Learning is a distinctive feature of intelligent behaviour. High-throughput experimental data and Big Data promise to open new windows on complex systems such as cells, the brain or our societies. Yet, the puzzling success of Artificial…
A physical law is represented by the probability distribution of a measured variable. The probability density is described by measured data using an estimator whose kernel is the instrument scattering function. The experimental information…
In this paper, we analyze the relationship between entropy and information in the context of the mixing process of two identical ideal gases. We will argue that entropy has a special information-based feature that is enfolded in the…
The uncertainty principle can be expressed in entropic terms, also taking into account the role of entanglement in reducing uncertainty. The information exclusion principle bounds instead the correlations that can exist between the outcomes…
The gap in statistics between multi-variate and time-series analysis can be bridged by using entropy statistics and recent developments in multi-dimensional scaling. For explaining the evolution of the sciences as non-linear dynamics, the…
Decision procedures aggregating the preferences of multiple agents can produce cycles and hence outcomes which have been described heuristically as `chaotic'. We make this description precise by constructing an explicit dynamical system…
In nonlinear dynamics, basins of attraction link a given set of initial conditions to its corresponding final states. This notion appears in a broad range of applications where several outcomes are possible, which is a common situation in…
Thermodynamics makes definite predictions about the thermal behavior of macroscopic systems in and out of equilibrium. Statistical mechanics aims to derive this behavior from the dynamics and statistics of the atoms and molecules making up…
Much of statistics relies upon four key elements: a law of large numbers, a calculus to operationalize stochastic convergence, a central limit theorem, and a framework for constructing local approximations. These elements are…
An algorithmic limit of compressed sensing or related variable-selection problems is analytically evaluated when a design matrix is given by an overcomplete random matrix. The replica method from statistical mechanics is employed to derive…
A consistent generalization of statistical mechanics is obtained by applying the maximum entropy principle to a trace-form entropy and by requiring that physically motivated mathematical properties are preserved. The emerging…
Statistical thermodynamics is valuable as a conceptual structure that shapes our thinking about equilibrium thermodynamic states. A cloud of unresolved questions surrounding the foundations of the theory could lead an impartial observer to…
The Shannon entropy, one of the cornerstones of information theory, is widely used in physics, particularly in statistical mechanics. Yet its characterization and connection to physics remain vague, leaving ample room for misconceptions and…
We propose a new type of entropic descriptor that is able to quantify the statistical complexity (a measure of complex behaviour) by taking simultaneously into account the average departures of a system's entropy S from both its maximum…
The exploration of complex physical or technological processes usually requires exploiting available information from different sources: (i) physical laws often represented as a family of parameter dependent partial differential equations…
The essence of the second law of classical thermodynamics is the `entropy principle' which asserts the existence of an additive and extensive entropy function, S, that is defined for all equilibrium states of thermodynamic systems and whose…
Entropy is a measure of heterogeneity widely used in applied sciences, often when data are collected over space. Recently, a number of approaches has been proposed to include spatial information in entropy. The aim of entropy is to…