相关论文: On nonlinear compression costs: when Shannon meets…

Relative entropy coding (REC) algorithms encode a sample from a target distribution $Q$ using a proposal distribution $P$, such that the expected codelength is $\mathcal{O}(D_{KL}[Q \,||\, P])$. REC can be seamlessly integrated with…

This paper presents a one shot analysis of the lossy compression problem under average distortion constraints. We calculate the exact expected distortion of a random code. The result is given as an integral formula using a newly defined…

This paper addresses a fundamental problem in random variate generation: given access to a random source that emits a stream of independent fair bits, what is the most accurate and entropy-efficient algorithm for sampling from a discrete…

We show that a simple telescoping sum trick, together with the triangle inequality and a tensorisation property of expected-contractive coefficients of random channels, allow us to achieve general simultaneous decoupling for multiple users…

This paper shows that the probability that the error exponent of a given code randomly generated from a pairwise independent ensemble being smaller than a lower bound on the typical random-coding exponent tends to zero as the codeword…

A classical random variable can be faithfully compressed into a sequence of bits with its expected length lies within one bit of Shannon entropy. We generalize this variable-length and faithful scenario to the general quantum source…

Grammar compression represents a string as a context free grammar. Achieving compression requires encoding such grammar as a binary string; there are a few commonly used encodings. We bound the size of practically used encodings for several…

We address the problem of applying the Kolmogorov-Sinai method of entropic analysis, expressed in a generalized non-extensive form, to the dynamics of the logistic map at the chaotic threshold, which is known to be characterized by a power…

Upper and lower bounds are obtained for the joint entropy of a collection of random variables in terms of an arbitrary collection of subset joint entropies. These inequalities generalize Shannon's chain rule for entropy as well as…

Let $W$ be a binary-input memoryless symmetric (BMS) channel with Shannon capacity $I(W)$ and fix any $\alpha > 0$. We construct, for any sufficiently small $\delta > 0$, binary linear codes of block length $O(1/\delta^{2+\alpha})$ and rate…

This paper proposes a new lossless data compression coding scheme named an asymmetric encoding-decoding scheme (AEDS), which can be considered as a generalization of tANS (tabled variant of asymmetric numeral systems). In the AEDS, a data…

Recently, a new decoding rule called jar decoding was proposed; under jar decoding, a non-asymptotic achievable tradeoff between the coding rate and word error probability was also established for any discrete input memoryless channel with…

We discuss the possibility of estimating experimentally the von Neumann entanglement entropy $S_{A}^{vN}$ of a symmetric bi-partite quantum system $AB$ by using the basic measurement counts (bitstrings) for a $single$ copy of a prepared…

Knowing the dynamics of neuromorphic photonic schemes would allow their optimization for controlled data-processing capability in possibly simplified designs and minimized energy consumption levels. In nonlinear substrates such as optical…

Tensors are a fundamental operation in distributed computing, \emph{e.g.,} machine learning, that are commonly distributed into multiple parallel tasks for large datasets. Stragglers and other failures can severely impact the overall…

We introduce a universal quantization scheme based on random coding, and we analyze its performance. This scheme consists of a source-independent random codebook (typically_mismatched_ to the source distribution), followed by optimal…

Sharpness (of the loss minima) is a common measure to investigate the generalization of neural networks. Intuitively speaking, the flatter the landscape near the minima is, the better generalization might be. Unfortunately, the correlation…

It has recently been a common practice to maximize the deformed entropies through the escort averaging scheme. However, whatever averaging procedure is employed, one should recover the ordinary Shannon maximization results in the…

The well known maximum-entropy principle due to Jaynes, which states that given mean parameters, the maximum entropy distribution matching them is in an exponential family, has been very popular in machine learning due to its "Occam's…

We consider an abstraction of computational security in password protected systems where a user draws a secret string of given length with i.i.d. characters from a finite alphabet, and an adversary would like to identify the secret string…