相关论文: General Erased-Word Processes: Product-Type Filtra…

Most existing word embedding methods can be categorized into Neural Embedding Models and Matrix Factorization (MF)-based methods. However some models are opaque to probabilistic interpretation, and MF-based methods, typically solved using…

A word-valued source $\mathbf{Y} = Y_1,Y_2,...$ is discrete random process that is formed by sequentially encoding the symbols of a random process $\mathbf{X} = X_1,X_2,...$ with codewords from a codebook $\mathscr{C}$. These processes…

Random substitutions are a natural generalisation of their classical `deterministic' counterpart, whereby at every step of iterating the substitution, instead of replacing a letter with a predetermined word, every letter is independently…

The present work analyzes the redundancy of sets of combinatorial objects produced by a weighted random generation algorithm proposed by Denise et al. This scheme associates weights to the terminals symbols of a weighted context-free…

We consider the off-policy evaluation problem in Markov decision processes with function approximation. We propose a generalization of the recently introduced \emph{emphatic temporal differences} (ETD) algorithm \citep{SuttonMW15}, which…

This work traces the evolution of word-embedding techniques within the natural language processing (NLP) literature. We collect and analyze 149 research articles spanning the period from 1954 to 2025, providing both a comprehensive…

It has been established under very general conditions that the ergodic properties of Markov processes are inherited by their conditional distributions given partial information. While the existing theory provides a rather complete picture…

Kingman derived the Ewens sampling formula for random partitions from the genealogy model defined by a Poisson process of mutations along lines of descent governed by a simple coalescent process. M\"ohle described the recursion which…

We introduce a new approach for smoothing and improving the quality of word embeddings. We consider a method of fusing word embeddings that were trained on the same corpus but with different initializations. We project all the models to a…

Kingman derived the Ewens sampling formula for random partitions describing the genetic variation in a neutral mutation model defined by a Poisson process of mutations along lines of descent governed by a simple coalescent process, and…

We develop a Perron-Frobenius type theory for products of random quantum channels acting on finite-dimensional matrix algebras sampled from a stationary and ergodic stochastic process, which, in keeping with the literature, we call ergodic…

While neural networks have been successfully applied to many natural language processing tasks, they come at the cost of interpretability. In this paper, we propose a general methodology to analyze and interpret decisions from a neural…

This paper presents a model-based, unsupervised algorithm for recovering word boundaries in a natural-language text from which they have been deleted. The algorithm is derived from a probability model of the source that generated the text.…

Gaussian elimination with partial pivoting (GEPP) is a widely used method to solve dense linear systems. Each GEPP step uses a row transposition pivot movement if needed to ensure the leading pivot entry is maximal in magnitude for the…

Distributed word representation (a.k.a. word embedding) is a key focus in natural language processing (NLP). As a highly successful word embedding model, Word2Vec offers an efficient method for learning distributed word representations on…

Motivated by problems of statistical language modeling, we consider probability measures on infinite sequences over two countable alphabets of a different cardinality, such as letters and words. We introduce an invertible mapping between…

We consider random walks on the set of all words over a finite alphabet such that in each step only the last two letters of the current word may be modified and only one letter may be adjoined or deleted. We assume that the transition…

We present a new proof of an Ergodic theorem for Wide-Sense Stationary Random Processes added with a new canonical sampling theorem of mine for finite time duration signals in the frequency domain (periodograms) which is free from the…

De Finetti's theorem, also called the de Finetti-Hewitt-Savage theorem, is a foundational result in probability and statistics. Roughly, it says that an infinite sequence of exchangeable random variables can always be written as a mixture…

Pre-trained language models such as BERT have become a more common choice of natural language processing (NLP) tasks. Research in word representation shows that isotropic embeddings can significantly improve performance on downstream tasks.…