Related papers: Thermodynamics of Information Retrieval
This paper provides a perspective on applying the concepts of information thermodynamics, developed recently in non-equilibrium statistical physics, to problems in theoretical neuroscience. Historically, information and energy in…
Thermodynamics and information have intricate interrelations. Often thermodynamics is considered to be the logical premise to justify that information is physical - through Landauer's principle -, thereby also linking information and…
The distribution of frequency counts of distinct words by length in a language's vocabulary will be analyzed using two methods. The first, will look at the empirical distributions of several languages and derive a distribution that…
Connections between information theory and thermodynamics have proven to be very useful to establish bounding limits for physical processes. Ideas such as Landauer's erasure principle and information assisted work extraction have greatly…
Statistical thermodynamics delivers the probability distribution of the equilibrium state of matter through the constrained maximization of a special functional, entropy. Its elegance and enormous success have led to numerous attempts to…
Several tasks in information retrieval (IR) rely on assumptions regarding the distribution of some property (such as term frequency) in the data being processed. This thesis argues that such distributional assumptions can lead to incorrect…
While ubiquitous, textual sources of information such as company reports, social media posts, etc. are hardly included in prediction algorithms for time series, despite the relevant information they may contain. In this work, openly…
The deep connection between thermodynamics, computation, and information is now well established both theoretically and experimentally. Here, we extend these ideas to show that thermodynamics also places fundamental constraints on…
Scientific discovery can be framed as a thermodynamic process in which an agent invests physical work to acquire information about an environment under a finite work budget. Using established results about the thermodynamics of computing,…
Information dynamics is an emerging description of information processing in complex systems which describes systems in terms of intrinsic computation, identifying computational primitives of information storage and transfer. In this paper…
As language models such as GPT-3 become increasingly successful at generating realistic text, questions about what purely text-based modeling can learn about the world have become more urgent. Is text purely syntactic, as skeptics argue? Or…
We study the informational underpinnings of thermodynamics and statistical mechanics, using an abstract framework, general probabilistic theories, capable of describing arbitrary physical theories. This allows one to abstract the…
We present a method to geometrize massive data sets from search engines query logs. For this purpose, a macrodynamic-like quantitative model of the Information Retrieval (IR) process is developed, whose paradigm is inspired by basic…
The significance of statistical physics concepts such as entropy extends far beyond classical thermodynamics. We interpret the similarity between partitions in statistical mechanics and partitions in Bayesian inference as an articulation of…
The statistical mechanical description of small systems staying in thermal equilibrium with an environment can be achieved by means of the Hamiltonian of mean force. In contrast to the reduced density matrix of an open quantum system, or…
For the system with inhomogeneous distribution of macroscopic parameters we obtain thermodynamic relation which depends on the spatial point (coordinate). In our approach, to obtain such a relation we use the basic ideas of the method of…
We introduce a generalised multivariate Polya process for document language modelling. The framework outlined here generalises a number of statistical language models used in information retrieval for modelling document generation. In…
Thousands of documents are made available to the users via the web on a daily basis. One of the most extensively studied problems in the context of such document streams is burst identification. Given a term t, a burst is generally…
Much work has been done on feature selection. Existing methods are based on document frequency, such as Chi-Square Statistic, Information Gain etc. However, these methods have two shortcomings: one is that they are not reliable for…
A system responding to a stochastic driving signal can be interpreted as computing, by means of its dynamics, an implicit model of the environmental variables. The system's state retains information about past environmental fluctuations,…