Related papers: Incomplete descriptions and relevant entropies
Entropy notions for $\varepsilon$-incremental practical stability and incremental stability of deterministic nonlinear systems under disturbances are introduced. The entropy notions are constructed via a set of points in state space which…
Non-deductive reasoning systems are often {\em representation dependent}: representing the same situation in two different ways may cause such a system to return two different answers. Some have viewed this as a significant problem. For…
Simplified mechanistic models in ecology have been criticized for the fact that a good fit to data does not imply the mechanism is true: pattern does not equal process. In parallel, the maximum entropy principle (MaxEnt) has been applied in…
In this paper an alternative approach to statistical mechanics based on the maximum information entropy principle (MaxEnt) is examined, specifically its close relation with the Gibbs method of ensembles. It is shown that the MaxEnt…
We introduce a new information-theoretic formulation of quantum measurement uncertainty relations, based on the notion of relative entropy between measurement probabilities. In the case of a finite-dimensional system and for any approximate…
Complex systems are found in most branches of science. It is still argued how to best quantify their complexity and to what end. One prominent measure of complexity (the statistical complexity) has an operational meaning in terms of the…
The thermodynamic definition of entropy can be extended to nonequilibrium systems based on its relation to information. To apply this definition in practice requires access to the physical system's microstates, which may be prohibitively…
We show that the naive application of the maximum entropy principle can yield answers which depend on the level of description, i.e. the result is not invariant under coarse-graining. We demonstrate that the correct approach, even for…
The paper moves a step towards the full integration of statistical mechanics and information theory. Starting from the assumption that the thermodynamical system is composed by particles whose quantized energies can be modelled as…
Many complex systems are characterized by non-Boltzmann distribution functions of their statistical variables. If one wants to -- justified or not -- hold on to the maximum entropy principle for complex statistical systems (non-Boltzmann)…
Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…
In the scientific and engineering literature, the second law of thermodynamics is expressed in terms of the behavior of entropy in reversible and irreversible processes. According to the prevailing statistical mechanics interpretation the…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
Statistical mechanics can only be ultimately justified in terms of microscopic dynamics (classical, quantum, relativistic, or any other). It is known that Boltzmann-Gibbs statistics is based on the hypothesis of exponential sensitivity to…
A colloquial interpretation of entropy is that it is the knowledge gained upon learning the outcome of a random experiment. Conditional entropy is then interpreted as the knowledge gained upon learning the outcome of one random experiment…
Recovering dynamical equations from observed noisy data is the central challenge of system identification. We develop a statistical mechanics approach to analyze sparse equation discovery algorithms, which typically balance data fit and…
As physics searches for invariants in observations, this paper looks for invariants of probabilistic observation without assuming physical structure. Structure emerges from the basic assumption of science that new information shall lead to…
Thermodynamics can be formulated in either of two approaches, the phenomenological approach, which refers to the macroscopic properties of systems, and the statistical approach, which describes systems in terms of their microscopic…
In the previous papers (Kui\'{c} et al. in Found Phys 42:319-339, 2012; Kui\'{c} in arXiv:1506.02622, 2015), it was demonstrated that applying the principle of maximum information entropy by maximizing the conditional information entropy,…
Here we deconstruct, and then in a reasoned way reconstruct, the concept of "entropy of a system," paying particular attention to where the randomness may be coming from. We start with the core concept of entropy as a COUNT associated with…