Related papers: Typical Sequences for Polish Alphabets
Written language is a complex communication signal capable of conveying information encoded in the form of ordered sequences of words. Beyond the local order ruled by grammar, semantic and thematic structures affect long-range patterns in…
Bayesian probability theory is used to analyze the oft-made assumption that humans are typical observers in the universe. Some theoretical calculations make the {\it selection fallacy} that we are randomly chosen from a class of objects by…
Query answering routinely employs knowledge graphs to assist the user in the search process. Given a knowledge graph that represents entities and relationships among them, one aims at complementing the search with intuitive but effective…
A central question in the era of 'big data' is what to do with the enormous amount of information. One possibility is to characterize it through statistics, e.g., averages, or classify it using machine learning, in order to understand the…
If $A$ and $B$ are sets such that $A \subset B$, generalisation may be understood as the inference from $A$ of a hypothesis sufficient to construct $B$. One might infer any number of hypotheses from $A$, yet only some of those may…
This paper establishes an abstract Korovkin-type approximation theorem in general spaces, extending the framework of approximation theory to accommodate broader contexts. A critical result supporting this theorem is the proof that any…
We study types that appear in ultraproducts that have distributions which can be thought of as a sequence of graphs. The property of having distributions that are captured by graphs is motivated by a commonality of $\mathrm{SOP}_2$-types…
Information theory is a mathematical theory of learning with deep connections with topics as diverse as artificial intelligence, statistical physics, and biological evolution. Many primers on information theory paint a broad picture with…
Universal compression of patterns of sequences generated by independently identically distributed (i.i.d.) sources with unknown, possibly large, alphabets is investigated. A pattern is a sequence of indices that contains all consecutive…
In previous work we defined and studied a notion of typicality, originated with B. Russell, for properties and objects in the context of general infinite first-order structures. In this paper we consider this notion in the context of finite…
We develop a technique for normalization for $\infty$-type theories. The normalization property helps us to prove a coherence theorem: the initial model of a given $\infty$-type theory is $0$-truncated. The coherence theorem justifies…
The characteristic sequence of hypergraphs $<P_n : n<\omega>$ associated to a formula $\phi(x;y)$, introduced in [arXiv:0908.4111], is defined by $P_n(y_1,... y_n) = (\exists x) \bigwedge_{i\leq n} \phi(x;y_i)$. This paper continues the…
Building on research arguing for the possibility of conceptual and categorical knowledge acquisition through statistics contained in language, we evaluate predictive language models (LMs) -- informed solely by textual input -- on a…
Out-of-distribution generalization capabilities of sequence-to-sequence models can be studied from the lens of two crucial forms of generalization: length generalization -- the ability to generalize to longer sequences than ones seen during…
We study the binary Ehrenfeucht Mycielski sequence seeking a balance between the number of occurrences of different binary strings. There have been numerous attempts to prove the balance conjecture of the sequence, which roughly states that…
Philosophers now seem to agree that frequentism is an untenable strategy to explain the meaning of probabilities. Nevertheless, I want to revive frequentism, and I will do so by grounding probabilities on typicality in the same way as the…
In this paper two hypotheses are developed. The first hypothesis is the existence of random phenomena/experiments in which the events cannot generally be assigned a definite probability but that nevertheless admit a class of nearly certain…
Information theory is built on probability measures and by definition a probability measure has total mass 1. Probability measures are used to model uncertainty, and one may ask how important it is that the total mass is one. We claim that…
The performance of ``typical set (pairs) decoding'' for ensembles of Gallager's linear code is investigated using statistical physics. In this decoding, error happens when the information transmission is corrupted by an untypical noise or…
Based on Euclid's algorithm, we find a kind of special sequences which play an interesting role in the study of primes. We call them W Sequences. They not only ties up the distribution of primes in short interval but also enables us to give…