Related papers: Thermodynamics of Information Retrieval
Temperature is a widely used hyperparameter in various tasks involving neural networks, such as classification or metric learning, whose choice can have a direct impact on the model performance. Most of existing works select its value using…
In natural scenes and documents, we can find the correlation between a text and its color. For instance, the word, "hot", is often printed in red, while "cold" is often in blue. This correlation can be thought of as a feature that…
Traditional topic models do not account for semantic regularities in language. Recent distributional representations of words exhibit semantic consistency over directional metrics such as cosine similarity. However, neither categorical nor…
We construct a generic model for a heat engine using information theory concepts, attributing irreversible energy dissipation to the information transmission channels. Using several forms for the channel capacity, classical and quantum, we…
Topic models, such as latent Dirichlet allocation (LDA), can be useful tools for the statistical analysis of document collections and other discrete data. The LDA model assumes that the words of each document arise from a mixture of topics,…
Document indexing is a key component for efficient information retrieval (IR). After preprocessing steps such as stemming and stop-word removal, document indexes usually store term-frequencies (tf). Along with tf (that only reflects the…
I argue here that both the first and second laws of thermodynamics, generally understood to be quintessentially physical in nature, can be equally well described as being about certain types of information without the need to invoke…
The thermodynamic and retrieval properties of fully connected Blume-Emery-Griffiths networks, storing ternary patterns, are studied using replica mean-field theory. Capacity-temperature phase diagrams are derived for several values of the…
Continuous word representation (aka word embedding) is a basic building block in many neural network-based models used in natural language processing tasks. Although it is widely accepted that words with similar semantics should be close to…
Word embeddings predict a word from its neighbours by learning small, dense embedding vectors. In practice, this prediction corresponds to a semantic score given to the predicted word (or term weight). We present a novel model that, given a…
In this paper, we study the thermodynamics of short-range central potentials, namely, the Lee-Wick potential, and the Plasma potential. In the first part of the paper we obtain the numerical solution for the orbits equation for these…
We present an efficient sampling method for computing a partition function and accelerating configuration sampling. The method performs a random walk in the $\lambda$ space, with $\lambda$ being any thermodynamic variable that characterizes…
The main contribution of this paper is to design an Information Retrieval (IR) technique based on Algorithmic Information Theory (using the Normalized Compression Distance- NCD), statistical techniques (outliers), and novel organization of…
Ensuring a satisfactory statistical convergence of anharmonic thermodynamic properties requires sampling of many atomic configurations, however the methods to obtain those necessarily produce correlated samples, thereby reducing the…
Information based thermodynamic logic is revisited. It consists of two parts: Part A applies the modern theory of probability in which an arbitrary convex function \phi is employed as an analytic "device" to express information as…
All statistical information about the heat can be obtained with the probability distribution of the heat functional. This paper derives analytically the expression for the distribution of the heat, through path integral, for a diffusive…
New formulas are given for the grand partition function of paraboson systems of order p with n orbitals and parafermion systems of order p with m orbitals. These formulas allow the computation of statistical and thermodynamic functions for…
In this paper, we explore and evaluate the use of ranking-based objective functions for learning simultaneously a word string and a word image encoder. We consider retrieval frameworks in which the user expects a retrieval list ranked…
Using the frequency of keywords is a classic approach in the formal analysis of text, but has the drawback of glossing over the relationality of word meanings. Word embedding models overcome this problem by constructing a standardized and…
Compounding is a highly productive word-formation process in some languages that is often problematic for natural language processing applications. In this paper, we investigate whether distributional semantics in the form of word…