Related papers: Multihead Finite-State Compression
This paper examines information-theoretic questions regarding the difficulty of compressing data versus the difficulty of decompressing data and the role that information loss plays in this interaction. Finite-state compression and…
We extend Ziv and Lempel's model of finite-state encoders to the realm of lossy compression of individual sequences. In particular, the model of the encoder includes a finite-state reconstruction codebook followed by an information lossless…
Stochastic finite-state generators are compressed descriptions of infinite time series. Alternatively, compressed descriptions are given by quantum finite- state generators [K. Wiesner and J. P. Crutchfield, Physica D 237, 1173 (2008)].…
In 2004, Dai, Lathrop, Lutz, and Mayordomo defined and investigated the finite-state dimension (a finite-state version of algorithmic dimension) of a sequence $S \in \Sigma^\infty$ and, in 2018, Case and Lutz defined and investigated the…
As they become more capable, large language models (LLMs) have continued to rapidly increase in size. This has exacerbated the difficulty in running state of the art LLMs on small, edge devices. Standard techniques advocate solving this…
We show how to calculate the finite-state dimension (equivalently, the finite-state compressibility) of a saturated sets $X$ consisting of {\em all} infinite sequences $S$ over a finite alphabet $\Sigma_m$ satisfying some given condition…
We describe a method for lossless quantum compression if the output of the information source is not known. We compute the best possible compression rate, minimizing the expected base length of the output quantum bit string (the base length…
This article describes lossless compression algorithms for multisets of sequences, taking advantage of the multiset's unordered structure. Multisets are a generalisation of sets where members are allowed to occur multiple times. A multiset…
This thesis concerns sequential-access data compression, i.e., by algorithms that read the input one or more times from beginning to end. In one chapter we consider adaptive prefix coding, for which we must read the input character by…
We consider the problem of compression of the quantum information carried by ensemble of mixed states. We prove that for arbitrary coding schemes the least number of qubits needed to convey the signal states asymptotically faithfully is…
How can we compress language models without sacrificing accuracy? The number of compression algorithms for language models is rapidly growing to benefit from remarkable advances of recent language models without side effects due to the…
We describe an image compression method, consisting of a nonlinear analysis transformation, a uniform quantizer, and a nonlinear synthesis transformation. The transforms are constructed in three successive stages of convolutional linear…
The rapid growth of large models' size has far outpaced that of computing resources. To bridge this gap, encouraged by the parsimonious relationship between genotype and phenotype in the brain's growth and development, we propose the…
Consider the case where consecutive blocks of N letters of a semi-infinite individual sequence X over a finite-alphabet are being compressed into binary sequences by some one-to-one mapping. No a-priori information about X is available at…
We present a systematic study of quantum system compression for the evolution of generic many-body problems. The necessary numerical simulations of such systems are seriously hindered by the exponential growth of the Hilbert space dimension…
The study of sums of finite sets of integers has mostly concentrated on sets with small sumsets (Freiman's theorem and related work) and on sets with large sumsets (Sidon sets and $B_h$-sets). This paper considers the sets ${\mathcal…
In this paper we present an end-to-end meta-learned system for image compression. Traditional machine learning based approaches to image compression train one or more neural network for generalization performance. However, at inference…
Model compression has gained significant popularity as a means to alleviate the computational and memory demands of machine learning models. Each compression technique leverages unique features to reduce the size of neural networks.…
We consider the problem of successive-refinement coding for lossy compression of individual sequences, namely, compression in two stages, where in the first stage, a coarse description at a relatively low rate is sent from the encoder to…
We design quantum compression algorithms for parametric families of tensor network states. We first establish an upper bound on the amount of memory needed to store an arbitrary state from a given state family. The bound is determined by…