Related papers: Memory and entropy
A system responding to a stochastic driving signal can be interpreted as computing, by means of its dynamics, an implicit model of the environmental variables. The system's state retains information about past environmental fluctuations,…
The major problem in information theoretic analysis of neural responses and other biological data is the reliable estimation of entropy--like quantities from small samples. We apply a recently introduced Bayesian entropy estimator to…
Here we deconstruct, and then in a reasoned way reconstruct, the concept of "entropy of a system," paying particular attention to where the randomness may be coming from. We start with the core concept of entropy as a COUNT associated with…
It has recently been emphasized again that the very existence of stationary stable localized structures with short range interactions might allow to store information in non-equilibrium media, opening new perspectives on information…
Free energy and entropy are examined in detail from the standpoint of classical thermodynamics. The approach is logically based on the fact that thermodynamic work is mediated by thermal energy through the tendency for nonthermal energy to…
Based on the data gained from a full-scale experiment, the order/disorder characteristics of the compartment fire temperatures are analyzed. Among the known permutation/encoding type entropies used to analyze time series, we look for those…
Observational entropy captures both the intrinsic uncertainty of a thermodynamic state and the lack of knowledge due to coarse-graining. We demonstrate two interpretations of observational entropy, one as the statistical deficiency…
An estimate of the maximal informational content available to advanced extraterrestrial or future (post)human civilizations is presented. It is shown that the fundamental thermodynamical considerations may lead to a quantitative estimate of…
The paper moves a step towards the full integration of statistical mechanics and information theory. Starting from the assumption that the thermodynamical system is composed by particles whose quantized energies can be modelled as…
Dissipative quantum systems are frequently described within the framework of the so-called "system-plus-reservoir" approach. In this work we assign their description to the Maximum Entropy Formalism and compare the resulting thermodynamic…
A communication theory for a transmitter broadcasting to many receivers is presented. In this case energetic considerations cannot be neglected as in Shannon theory. It is shown that, when energy is assigned to the information bit,…
Classical thermodynamics is unrivalled in its range of applications and relevance to everyday life. It enables a description of complex systems, made up of microscopic particles, in terms of a small number of macroscopic quantities, such as…
Entropic dynamics is a framework for defining dynamical systems that is aligned with the principles of information theory. In an entropic dynamics model for motion on a statistical manifold, we find that the rate of changes for expected…
Our everyday descriptions of the universe are highly coarse-grained, following only a tiny fraction of the variables necessary for a perfectly fine-grained description. Coarse graining in classical physics is made natural by our limited…
We formulate a geometric framework in which physical laws emerge from restricted access to microscopic information. Measurement constraints are modeled as a gauge symmetry acting on density operators, inducing a gauge-reduced space of…
Information-theoretic quantities, such as entropy, are used to quantify the amount of information a given variable provides. Entropies can be used together to compute the mutual information, which quantifies the amount of information two…
We show that transport in the presence of entropic barriers exhibits peculiar characteristics which makes it distinctly different from that occurring through energy barriers. The constrained dynamics yields a scaling regime for the particle…
There is a relation between the irreversibility of thermodynamic processes as expressed by the breaking of time-reversal symmetry, and the entropy production in such processes. We explain on an elementary mathematical level the relations…
We investigate thermodynamics of feedback processes driven by measurement. Regarding system and memory device as a composite system, mutual information as a measure of correlation between the two constituents contributes to the entropy of…
The transmission of information can couple two entities of very different nature, one of them serving as a memory for the other. Here we study the situation in which information is stored in a wave field and serves as a memory that pilots…