Related papers: Thermodynamics of Information Retrieval
We develop a statistical mechanical interpretation of algorithmic information theory by introducing the notion of thermodynamic quantities, such as free energy, energy, statistical mechanical entropy, and specific heat, into algorithmic…
We study random words in a weighted regular language that achieve the maximal free energy using thermodynamics formalism. In particular, typical words in the language are algorithmically generated which have applications in computer…
In this article we present a model of human written text based on statistical mechanics approach by deriving the potential energy for different parts of the text using large text corpus. We have checked the results numerically and found…
We present algorithms for topic modeling based on the geometry of cross-document word-frequency patterns. This perspective gains significance under the so called separability condition. This is a condition on existence of novel-words that…
This letter exposes a tight connection between the thermodynamic efficiency of information processing and predictive inference. A generalized lower bound on dissipation is derived for partially observable information engines which are…
We have formulated a family of machine learning problems as the time evolution of Parametric Probabilistic Models (PPMs), inherently rendering a thermodynamic process. Our primary motivation is to leverage the rich toolbox of thermodynamics…
Thermodynamics (in concert with its sister discipline, statistical physics) can be regarded as a data reduction scheme based on partitioning a total system into a subsystem and a bath that weakly interact with each other. The ubiquity and…
In recent years, the unified theory of information and thermodynamics has been intensively discussed in the context of stochastic thermodynamics. The unified theory reveals that information theory would be useful to understand…
Text Categorization is traditionally done by using the term frequency and inverse document frequency.This type of method is not very good because, some words which are not so important may appear in the document .The term frequency of…
The concept of temperature is one of the key ideas in describing the thermodynamical properties of a physical system. In classical statistical mechanics of ideal gases, the notion of temperature can be described in two different ways, the…
In the theoretical modelling of a physical system a crucial step consists in the identification of those degrees of freedom that enable a synthetic, yet informative representation of it. While in some cases this selection can be carried out…
We present a theoretical and empirical investigation of the statistical behaviour of the words in a text produced by human language. To this aim, we analyse the word distribution of various texts of Italian language selected from a specific…
This Colloquium is a fast journey through the build-up of key thermodynamical concepts, i.e. work, heat and irreversibility -- and how they relate to information. Born at the time of industrial revolution to optimize the exploitation of…
In traditional thermodynamical and statistical-mechanical approaches one has (some) detailed knowledge of the principles governing the microdynamics of a system. However in many instances we may not have a Hamiltonian or good information…
Information thermodynamics relates the rate of change of mutual information between two interacting subsystems to their thermodynamics when the joined system is described by a bipartite stochastic dynamics satisfying local detailed balance.…
Search engines perform the task of retrieving information related to the user-supplied query words. This task has two parts; one is finding "featured words" which describe an article best and the other is finding a match among these words…
Three different approaches to derive mutual information via thermodynamics are presented where the temperature-dependent energy is given by: (a) $\beta \mathcal{E} = -\ln[P(X,Y)]$, (b) $\beta \mathcal{E} =-\ln[P(Y|X)]$ or (c) $\beta…
Just like everything in the nature, scientific topics flourish and perish. While existing literature well captures article's life-cycle via citation patterns, little is known about how scientific popularity and impact evolves for a specific…
We formulate thermodynamics of economic systems in terms of an arbitrary probability distribution for a conserved economic quantity. As in statistical physics, thermodynamic macroeconomic variables emerge as the mean value of microeconomic…
Thermodynamics and information have intricate inter-relations. The justification of the fact that information is physical, is done by inter-linking information and thermodynamics - through Landauer's principle. This modern approach towards…