Related papers: Condensed Unpredictability
We derive a lower bound on the smallest output entropy that can be achieved via vector quantization of a $d$-dimensional source with given expected $r$th-power distortion. Specialized to the one-dimensional case, and in the limit of…
The exponential growth of data transmission and processing speeds in modern digital infrastructure requires entropy sources capable of producing large volumes of true randomness for information security. Chaotic emissions from semiconductor…
We show that unconverged stochastic gradient descent can be interpreted as a procedure that samples from a nonparametric variational approximate posterior distribution. This distribution is implicitly defined as the transformation of an…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible constrained to match empirical data, for instance, feature expectations. We seek to generalize…
Large reasoning models have demonstrated remarkable performance on complex reasoning tasks, yet the excessive length of their chain-of-thought outputs remains a major practical bottleneck due to high computation cost and poor deployability.…
Machine learning has had a major impact on data compression over the last decade and inspired many new, exciting theoretical and applied questions. This paper describes one such direction -- relative entropy coding -- which focuses on…
Herding is a deterministic algorithm used to generate data points that can be regarded as random samples satisfying input moment conditions. The algorithm is based on the complex behavior of a high-dimensional dynamical system and is…
Here we deconstruct, and then in a reasoned way reconstruct, the concept of "entropy of a system," paying particular attention to where the randomness may be coming from. We start with the core concept of entropy as a COUNT associated with…
We simply construct a quantum universal variable-length source code in which, independent of information source, both of the average error and the probability that the coding rate is greater than the entropy rate $H(rho_p)$, tend to 0. If…
We discuss further our proposed modification of the Susskind-Horowitz-Polchinski scenario in which black hole entropy goes over to string entropy as one scales the string length scale up and the string coupling constant down, keeping…
We introduce the Clifford entropy, a measure of how close an arbitrary unitary is to a Clifford unitary, which generalizes the stabilizer entropy for states. We show that this quantity vanishes if and only if a unitary is Clifford, is…
In this paper, we propose a source coding scheme that represents data from unknown distributions through frequency and support information. Existing encoding schemes often compress data by sacrificing computational efficiency or by assuming…
There are several inequalities in physics which limit how well we can process physical systems to achieve some intended goal, including the second law of thermodynamics, entropy bounds in quantum information theory, and the uncertainty…
Ultrafast physical random bit generation at hundreds of Gb/s rates, with verified randomness, is a crucial ingredient in secure communication and have recently emerged using optics based physical systems. Here we examine the inverse problem…
This paper addresses a fundamental problem in random variate generation: given access to a random source that emits a stream of independent fair bits, what is the most accurate and entropy-efficient algorithm for sampling from a discrete…
Over the last few years, machine learning unlocked previously infeasible features for compression, such as providing guarantees for users' privacy or tailoring compression to specific data statistics (e.g., satellite images or audio…
Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…
This paper proposes a novel entropy encoding technique for lossless data compression. Representing a message string by its lexicographic index in the permutations of its symbols results in a compressed version matching Shannon entropy of…
The generic crease set of an event horizon possesses anisotropic structure though most of black holes are dynamically stable. This fact suggests that a generic almost spherical black hole has a very crumpled crease set in a microscopic…
We derive a new entropic quantum uncertainty relation involving min-entropy. The relation is tight and can be applied in various quantum-cryptographic settings. Protocols for quantum 1-out-of-2 Oblivious Transfer and quantum Bit Commitment…