Related papers: An Extended Fano's Inequality for the Finite Block…
While the channel capacity reflects a theoretical upper bound on the achievable information transmission rate in the limit of infinitely many bits, it does not characterise the information transfer of a given encoding routine with finitely…
Looped Transformers provide advantages in parameter efficiency, computational capabilities, and generalization for reasoning tasks. However, their expressive power regarding function approximation remains underexplored. In this paper, we…
We investigate the longest common substring problem for encoded sequences and its asymptotic behaviour. The main result is a strong law of large numbers for a re-scaled version of this quantity, which presents an explicit relation with the…
This paper studies random-coding error exponents of randomised list decoding, in which the decoder randomly selects $L$ messages with probabilities proportional to the decoding metric of the codewords. The exponents (or bounds) are given…
This paper proposes a unifying variational approach for proving and extending some fundamental information theoretic inequalities. Fundamental information theory results such as maximization of differential entropy, minimization of Fisher…
It is shown that some well-known and some new cyclic codes with orthogonal parity-check equations can be constructed in the finite-field transform domain. It is also shown that, for some binary linear cyclic codes, the performance of the…
Proximal constraints are fundamental to the stability of the Large Language Model reinforcement learning. While the canonical clipping mechanism in PPO serves as an efficient surrogate for trust regions, we identify a critical bottleneck:…
Constrained codes are used to eliminate error-prone patterns in various practical systems. Recently, we introduced efficient binary symmetric lexicographically-ordered constrained (LOCO) codes and asymmetric LOCO (A-LOCO) codes to increase…
Low-latency communication plays an increasingly important role in delay-sensitive applications by ensuring the real-time information exchange. However, due to the constraint on the maximum instantaneous power, guaranteeing bounded latency…
The design of block codes for short information blocks (e.g., a thousand or less information bits) is an open research problem that is gaining relevance thanks to emerging applications in wireless communication networks. In this paper, we…
We explore the relation between entanglement entropy of quantum many body systems and the distribution of corresponding, properly selected, observables. Such a relation is necessary to actually measure the entanglement entropy. We show that…
The likelihood encoder with a random codebook is demonstrated as an effective tool for source coding. Coupled with a soft covering lemma (associated with channel resolvability), likelihood encoders yield simple achievability proofs for…
A scheme to achieve dense quantum coding for the quadrature amplitudes of the electromagnetic field is presented. The protocol utilizes shared entanglement provided by nondegenerate parametric down conversion in the limit of large gain to…
Two channels are equivalent if their maximum likelihood (ML) decoders coincide for every code. We show that this equivalence relation partitions the space of channels into a generalized hyperplane arrangement. With this, we define a coding…
This paper studies physical-layer secure transmissions from a transmitter to a legitimate receiver against an eavesdropper over slow fading channels, taking into account the impact of finite blocklength secrecy coding. A comprehensive…
In this letter, we evaluate the finite-length performance of network coding when using either random or structured encoding matrices. First, we present our novel construction of structured network codes over Fq (q = 2^m) using Pascal…
This paper is motivated by the problem of error control in network coding when errors are introduced in a random fashion (rather than chosen by an adversary). An additive-multiplicative matrix channel is considered as a model for random…
We examine a new class of channel coding strategies, and hypothesis tests referred to as almost-fixed-length strategies that have little flexibility in the stopping time over fixed-length strategies. The stopping time of these strategies is…
The weak law of large numbers implies that, under mild assumptions on the source, the Renyi entropy per produced symbol converges (in probability) towards the Shannon entropy rate. This paper quantifies the speed of this convergence for…
This paper examines the maximum code rate achievable by a data-driven communication system over some unknown discrete memoryless channel in the finite blocklength regime. A class of channel codes, called learning-based channel codes, is…