Related papers: Thermodynamics of Information Retrieval
In this paper, we explore the usage of Word Embedding semantic resources for Information Retrieval (IR) task. This embedding, produced by a shallow neural network, have been shown to catch semantic similarities between words (Mikolov et…
This article presents a probabilistic generative model for text based on semantic topics and syntactic classes called Part-of-Speech LDA (POSLDA). POSLDA simultaneously uncovers short-range syntactic patterns (syntax) and long-range…
A general problem of optimal information acquisition for its use in decision making problems is considered. This motivates the need for developing quantitative measures of information sources' capabilities for supplying accurate information…
Complex systems produce high-dimensional signals that lack macroscopic variables analogous to entropy, temperature, or free energy. This work introduces a thermoinformational formulation that derives entropy, internal energy, temperature,…
Thermodynamic properties can be in principle derived from the partition function, which, in many-atom systems, is hard to evaluate as it involves a sum on the accessible microscopic states. Recently, the partition function has been computed…
We develop an axiomatic reconstruction of thermodynamics based entirely on two primitive components: a description of what aspects of a system are observed and a reference measure that encodes the underlying descriptive convention. These…
To make an interactive guidance mechanism for document retrieval systems, we developed a user-interface which presents users the visualized map of topics at each stage of retrieval process. Topic words are automatically extracted by…
Humans often communicate by using imprecise language, suggesting that fuzzy concepts with unclear boundaries are prevalent in language use. In this paper, we test the extent to which models trained to capture the distributional statistics…
We investigate the probability distribution of the quantum fluctuations of thermodynamic functions of finite, ballistic, phase-coherent Fermi gases. Depending on the chaotic or integrable nature of the underlying classical dynamics, on the…
We investigate fundamental connections between thermodynamics and quantum information theory. First, we show that the operational framework of thermal operations is nonequivalent to the framework of Gibbs-preserving maps, and we comment on…
Exact and approximate expressions for thermodynamic characteristics of heated matter, which consists of particles with finite mass-widths, are constructed. They are expressed in terms of Fermi/Bose distributions and spectral functions,…
In a physical system, changing parameters such as temperature can induce a phase transition: an abrupt change from one state of matter to another. Analogous phenomena have recently been observed in large language models. Typically, the task…
Most Information Retrieval models compute the relevance score of a document for a given query by summing term weights specific to a document or a query. Heuristic approaches, like TF-IDF, or probabilistic models, like BM25, are used to…
A new set of parameters to describe the word frequency behavior of texts is proposed. The analogy between the word frequency distribution and the Bose-distribution is suggested and the notion of "temperature" is introduced for this case.…
Thermodynamics could be seen as an expression of physics at a high epistemic level. As such, its potential as an inductive bias to help machine learning procedures attain accurate and credible predictions has been recently realized in many…
The word-stock of a language is a complex dynamical system in which words can be created, evolve, and become extinct. Even more dynamic are the short-term fluctuations in word usage by individuals in a population. Building on the recent…
In this study, we continue our exploration of the concept of information temperature as a characteristic of random sequences. We describe methods for introducing the information temperature in the context of binary high-order Markov chain…
Word frequency-based methods for extractive summarization are easy to implement and yield reasonable results across languages. However, they have significant limitations - they ignore the role of context, they offer uneven coverage of…
Finding rare but useful solutions in very large candidate spaces is a recurring practical challenge across language generation, planning, and reinforcement learning. We present a practical framework, \emph{Inverted Causality Focusing…
Short text messages such as tweets are very noisy and sparse in their use of vocabulary. Traditional textual representations, such as tf-idf, have difficulty grasping the semantic meaning of such texts, which is important in applications…