Related papers: The flow of ideas in word embeddings
In many applications, ideas that are described by a set of words often flow between different groups. To facilitate users in analyzing the flow, we present a method to model the flow behaviors that aims at identifying the lead-lag…
Network embedding which encodes all vertices in a network as a set of numerical vectors in accordance with it's local and global structures, has drawn widespread attention. Network embedding not only learns significant features of a…
Random walkers characterized by random positions and random velocities lead to normal diffusion. A random walk was originally proposed by Einstein to model Brownian motion and to demonstrate the existence of atoms and molecules. Such a…
Word embeddings are ubiquitous in NLP and information retrieval, but it is unclear what they represent when the word is polysemous. Here it is shown that multiple word senses reside in linear superposition within the word embedding and…
Random walks are ubiquitous in the sciences, and they are interesting from both theoretical and practical perspectives. They are one of the most fundamental types of stochastic processes; can be used to model numerous phenomena, including…
The development of plot or story in novels is reflected in the content and the words used. The flow of sentiments, which is one aspect of writing style, can be quantified by analyzing the flow of words. This study explores literary works as…
We address this work to investigate some statistical properties of symbolic sequences generated by a numerical procedure in which the symbols are repeated following a power law probability density. In this analysis, we consider that the sum…
Word embeddings are a popular way to improve downstream performances in contemporary language modeling. However, the underlying geometric structure of the embedding space is not well understood. We present a series of explorations using…
Understanding how large language models (LLMs) represent natural language is a central challenge in natural language processing (NLP) research. Many existing methods extract word embeddings from an LLM, visualise the embedding space via…
Mathematical models of motility are often based on random-walk descriptions of discrete individuals that can move according to certain rules. It is usually the case that large masses concentrated in small regions of space have a great…
We examine the line-wrap feature of text processors and show that adding characters to previously formatted lines leads to the cascading of words to subsequent lines and forms a state of self-organized criticality. We show the connection to…
We study memory based random walk models to understand diffusive motion in crowded heterogeneous environment. The models considered are non-Markovian as the current move of the random walk models is determined by randomly selecting a move…
Random walks provide a simple conventional model to describe various transport processes, for example propagation of heat or diffusion of matter through a medium. However, in many practical cases the medium is highly irregular due to…
Distributed language representation has become the most widely used technique for language representation in various natural language processing tasks. Most of the natural language processing models that are based on deep learning…
Graph embedding based on random-walks supports effective solutions for many graph-related downstream tasks. However, the abundance of embedding literature has made it increasingly difficult to compare existing methods and to identify…
Word embeddings allow natural language processing systems to share statistical information across related words. These embeddings are typically based on distributional statistics, making it difficult for them to generalize to rare or unseen…
In this paper, I present a novel method to detect intellectual influence across a large corpus. Taking advantage of the unique affordances of large language models in encoding semantic and structural meaning while remaining robust to…
Random walks are studied on disordered cellular networks in 2-and 3-dimensional spaces with arbitrary curvature. The coefficients of the evolution equation are calculated in term of the structural properties of the cellular system. The…
Word embeddings are a fixed, distributional representation of the context of words in a corpus learned from word co-occurrences. Despite their proven utility in machine learning tasks, word embedding models may capture uneven semantic and…
The random walk process underlies the description of a large number of real world phenomena. Here we provide the study of random walk processes in time varying networks in the regime of time-scale mixing; i.e. when the network connectivity…