Related papers: Statistical Physics of Coding for the Integers
The aim of the present paper is to provide a preliminary investigation of the thermodynamics of particles obeying monotone statistics. To render the potential physical applications realistic, we propose a modified scheme called…
Recent investigations show that the statistical mechanics of a finite number of particles in ideal harmonic systems predicts different results for the same physical properties, depending on the ensemble under consideration. Path integral…
We study the large deviations behavior of systems that admit a certain form of a product distribution, which is frequently encountered both in Physics and in various information system models. First, to fix ideas, we demonstrate a simple…
This paper focuses on the ultimate limit theory of image compression. It proves that for an image source, there exists a coding method with shapes that can achieve the entropy rate under a certain condition where the shape-pixel ratio in…
The partition function pertaining to finite--temperature decoding of a (typical) randomly chosen code is known to have three types of behavior, corresponding to three phases in the plane of rate vs. temperature: the {\it ferromagnetic…
We study a random code ensemble with a hierarchical structure, which is closely related to the generalized random energy model with discrete energy values. Based on this correspondence, we analyze the hierarchical random code ensemble by…
We examine the classical joint source--channel coding problem from the viewpoint of statistical physics and demonstrate that in the random coding regime, the posterior probability distribution of the source given the channel output is…
Partition functions $Z(x)$ of statistical mechanics are generally approximated by integrals. The approximation fails in small cavities or at very low temperature, when the ratio $x$ between the energy quantum and thermal energy is larger or…
The number partitioning problem can be interpreted physically in terms of a thermally isolated non-interacting Bose gas trapped in a one-dimensional harmonic oscillator potential. We exploit this analogy to characterize, by means of a…
Relative entropy coding (REC) algorithms encode a random sample following a target distribution $Q$, using a coding distribution $P$ shared between the sender and receiver. Sadly, general REC algorithms suffer from prohibitive encoding…
We compute the full probability distribution of the moment of inertia $I \propto \sum_{i=1}^N \vec{r}_i^{\,2}$ of a gas of $N$ noninteracting bosons trapped in a harmonic potential $V(r) = (1/2)\, m\, \omega^2 r^2$, in all dimensions and at…
In this paper, we propose a source coding scheme that represents data from unknown distributions through frequency and support information. Existing encoding schemes often compress data by sacrificing computational efficiency or by assuming…
Natural languages exhibit striking regularities in their statistical structure, including notably the emergence of Zipf's and Heaps' laws. Despite this, it remains broadly unclear how these properties relate to the modern tokenisation…
In this paper we derive accurate numerical methods for the quantum Boltzmann equation for a gas of interacting bosons. The schemes preserve the main physical features of the continuous problem, namely conservation of mass and energy, the…
This paper proposes a novel entropy encoding technique for lossless data compression. Representing a message string by its lexicographic index in the permutations of its symbols results in a compressed version matching Shannon entropy of…
Many data compressors regularly encode probability distributions for entropy coding - requiring minimal description length type of optimizations. Canonical prefix/Huffman coding usually just writes lengths of bit sequences, this way…
In the thermodynamic limit the ratio of system size to thermal de Broglie wavelength tends to infinity and the volume per particle of the system is constant. Our familiar Bose-Einstein statistics is absolutely valid in the thermodynamic…
In this paper will be presented methodology of encoding information in valuations of discrete lattice with some translational invariant constrains in asymptotically optimal way. The method is based on finding statistical description of such…
We consider an isolated system in an arbitrary state and provide a general formulation using first principles for an additive and non-negative statistical quantity that is shown to reproduce the equilibrium thermodynamic entropy of the…
Over the last few years, machine learning unlocked previously infeasible features for compression, such as providing guarantees for users' privacy or tailoring compression to specific data statistics (e.g., satellite images or audio…