Related papers: Compression of enumerations and gain
Despite it being the cornerstone of BPE, the most common tokenization algorithm, the importance of compression in the tokenization process is still unclear. In this paper, we argue for the theoretical importance of compression, that can be…
Our main objective is the COMPRESSED enumeration (based on wildcards) of all minimal hitting sets of general hypergraphs. To the author's best knowledge the only previous attempt towards compression, due to Toda, is based on BDD's and much…
We present an informal survey (meant to accompany another paper) on graph compression methods. We focus on lossless methods, briefly list available pproaches, and compare them where possible or give some indicators on their compression…
How can we compress language models without sacrificing accuracy? The number of compression algorithms for language models is rapidly growing to benefit from remarkable advances of recent language models without side effects due to the…
Can we use machine learning to compress graph data? The absence of ordering in graphs poses a significant challenge to conventional compression algorithms, limiting their attainable gains as well as their ability to discover relevant…
As suggested by the title, this paper is a survey of recent results and questions on the collection of computably enumerable sets under inclusion. This is not a broad survey but one focused on the author's and a few others' current…
Embedding tables are used by machine learning systems to work with categorical features. In modern Recommendation Systems, these tables can be very large, necessitating the development of new methods for fitting them in memory, even during…
We investigate the complexity of isomorphism relations for classes of finitely generated and n-generated computably enumerable (c.e.) algebras, presented via c.e. presentations -- that is, as quotients of term algebras over decidable sets…
In prefix coding over an infinite alphabet, methods that consider specific distributions generally consider those that decline more quickly than a power law (e.g., Golomb coding). Particular power-law distributions, however, model many…
Compressed sensing is a signal processing technique that allows for the reconstruction of a signal from a small set of measurements. The key idea behind compressed sensing is that many real-world signals are inherently sparse, meaning that…
We recast move generators for solving board games as operations on compressed sets of strings. We aim for compressed representations with space sublinear in the number of game positions for interesting sets of positions, move generation in…
Neural compression is the application of neural networks and other machine learning methods to data compression. Recent advances in statistical machine learning have opened up new possibilities for data compression, allowing compression…
Random Number Generators play a critical role in a number of important applications. In practice, statistical testing is employed to gather evidence that a generator indeed produces numbers that appear to be random. In this paper, we…
Variational inequalities as an effective tool for solving applied problems, including machine learning tasks, have been attracting more and more attention from researchers in recent years. The use of variational inequalities covers a wide…
A set $X \subseteq 2^\omega$ with positive measure contains a perfect subset. We study such perfect subsets from the viewpoint of computability and prove that these sets can have weak computational strength. Then we connect the existence of…
Various algorithms have been proposed to enumerate all connected induced subgraphs of a graph $G=(V,E)$. As a variation we enumerate all "packings of connected sets", i.e. partitions $\Pi$ of $V$ with the property that each part of $\Pi$…
Kolmogorov complexity is often used as a convenient language for counting and/or probabilistic existence proofs. However, there are some applications where Kolmogorov complexity is used in a more subtle way. We provide one (somehow)…
A compression algorithm is presented that uses the set of prime numbers. Sequences of numbers are correlated with the prime numbers, and labeled with the integers. The algorithm can be iterated on data sets, generating factors of doubles on…
Compressed file formats are the corner stone of efficient data storage and transmission, yet their potential for representation learning remains largely underexplored. We introduce TEMPEST (TransformErs froM comPressed rEpreSenTations), a…
The incompressibility method is a counting argument in the framework of algorithmic complexity that permits discovering properties that are satisfied by most objects of a class. This paper gives a preliminary insight into Kolmogorov's…