Related papers: Moser-Tardos resampling algorithm, entropy compres…
As access to high-quality, domain-specific data grows increasingly scarce, multi-epoch training has become a practical strategy for adapting large language models (LLMs). However, autoregressive models often suffer from performance…
We make use of an entropic property to establish a convergence theorem (Main Theorem), which reveals that the conditional entropy measures the asymptotic Gaussianity. As an application, we establish the {\it entropic conditional central…
We consider the problem of learned transform compression where we learn both, the transform as well as the probability distribution over the discrete codes. We utilize a soft relaxation of the quantization operation to allow for…
End-to-end deep trainable models are about to exceed the performance of the traditional handcrafted compression techniques on videos and images. The core idea is to learn a non-linear transformation, modeled as a deep neural network,…
We provide new estimates of an asymptotic upper bound on the entropy of English using the large language model LLaMA-7B as a predictor for the next token given a window of past tokens. This estimate is significantly smaller than currently…
Some results on cross-diffusion systems with entropy structure are reviewed. The focus is on local-in-time existence results for general systems with normally elliptic diffusion operators, due to Amann, and global-in-time existence theorems…
For a gas-solid interfacial system where chemical species undergo reversible adsorption, we develop a mesoscopic stochastic modeling method that simulates both gas-phase hydrodynamics and surface coverage dynamics by coupling the Langmuir…
We investigate lossy compression (source coding) of data in the form of permutations. This problem has direct applications in the storage of ordinal data or rankings, and in the analysis of sorting algorithms. We analyze the rate-distortion…
We compare the behaviour of a small truncated coupled map lattice with random inputs at the boundaries with that of a large deterministic lattice essentially at the thermodynamic limit. We find exponential convergence for the probability…
We study ground-state quantum entanglement in the one-dimensional Bose-Hubbard model in the presence of a harmonic trap. We focus on two transitions that occur upon increasing the characteristic particle density: the formation of a…
We compute the low-temperature configurational entropy of a two-dimensional supercooled liquid. Our method, based on a higher-dimensional version of the Grassberger--Procaccia algorithm, can be implemented in a manner that is entirely…
The Markov entropy decomposition (MED) is a recently-proposed, cluster-based simulation method for finite temperature quantum systems with arbitrary geometry. In this paper, we detail numerical algorithms for performing the required steps…
We propose an end-to-end trainable image compression framework with a multi-scale and context-adaptive entropy model, especially for low bitrate compression. Due to the success of autoregressive priors in probabilistic generative model, the…
This paper proposes a novel method of algorithmic subsampling (data sketching) for multiway cluster dependent data. We establish a new uniform weak law of large numbers and a new central limit theorem for the multiway algorithmic subsample…
We investigate the performance of entropy estimation methods, based either on block entropies or compression approaches, in the case of bidimensional sequences. We introduce a validation dataset made of images produced by a large number of…
We analyze the ground-state and low-temperature properties of a one-dimensional Bose gas in a harmonic trapping potential using the numerical density matrix renormalization group. Calculations cover the whole range from the Bogoliubov limit…
The aim of this paper is to provide several novel upper bounds on the excess risk with a primal focus on classification problems. We suggest two approaches and the obtained bounds are represented via the distribution dependent local…
This note extends the modulated entropy and free energy methods for proving mean-field limits/propagation of chaos to the whole space without any confining potential, in contrast to previous work limited to the torus or requiring…
Grammar compression is a general compression framework in which a string $T$ of length $N$ is represented as a context-free grammar of size $n$ whose language contains only $T$. In this paper, we focus on studying the limitations of…
We give a number of approximation metatheorems for monotone maximization problems expressible in the first-order logic, in substantially more general settings than the previously known. We obtain * constant-factor approximation algorithm in…