Related papers: Information-Theoretic Limits on Compression of Sem…
Semantic communications, aiming at ensuring the successful delivery of the meaning of information, are expected to be one of the potential techniques for the next generation communications. However, the knowledge forming and synchronizing…
In this paper, the problem of joint transmission and computation resource allocation for a multi-user probabilistic semantic communication (PSC) network is investigated. In the considered model, users employ semantic information extraction…
Transformer-based Large Language Models (LLMs) often impose limitations on the length of the text input to ensure the generation of fluent and relevant responses. This constraint restricts their applicability in scenarios involving long…
One of the primary goals of information theory is to provide limits on the amount of information it is possible to send through various types of communication channels, and to understand the encoding methods that will allow one to achieve…
This paper studies the joint data and semantics lossy compression problem, i.e., an extension of the hidden lossy source coding problem that entails recovering both the hidden and observable sources. We aim to study the nonasymptotic and…
Recently, a secrecy measure based on list-reconstruction has been proposed [2], in which a wiretapper is allowed to produce a list of $2^{mR_{L}}$ reconstruction sequences and the secrecy is measured by the minimum distortion over the…
Our capacity to process information depends on the computational power at our disposal. Information theory captures our ability to distinguish states or communicate messages when it is unconstrained with unrivaled beauty and elegance. For…
Learning disentangled representations of natural language is essential for many NLP tasks, e.g., conditional text generation, style transfer, personalized dialogue systems, etc. Similar problems have been studied extensively for other forms…
A classical random variable can be faithfully compressed into a sequence of bits with its expected length lies within one bit of Shannon entropy. We generalize this variable-length and faithful scenario to the general quantum source…
Lossy compression algorithms are typically designed and analyzed through the lens of Shannon's rate-distortion theory, where the goal is to achieve the lowest possible distortion (e.g., low MSE or high SSIM) at any given bit rate. However,…
Recent contributions of semantic information theory reveal the set-element relationship between semantic and syntactic information, represented as synonymous relationships. In this paper, we propose a synonymous variational inference (SVI)…
This paper analyzes the fundamental limits of strate- gic communication in network settings. Strategic communication differs from the conventional communication paradigms in in- formation theory since it involves different objectives for…
Locally decodable channel codes form a special class of error-correcting codes with the property that the decoder is able to reconstruct any bit of the input message from querying only a few bits of a noisy codeword. It is well known that…
We consider the problem of joint source and channel coding of structured data such as natural language over a noisy channel. The typical approach to this problem in both theory and practice involves performing source coding to first…
We consider lossy compression of an information source when the decoder has lossless access to a correlated one. This setup, also known as the Wyner-Ziv problem, is a special case of distributed source coding. To this day, real-world…
We study a new class of codes for lossy compression with the squared-error distortion criterion, designed using the statistical framework of high-dimensional linear regression. Codewords are linear combinations of subsets of columns of a…
In this paper, we consider the one-shot version of the classical Wyner-Ziv problem where a source is compressed in a lossy fashion when only the decoder has access to a correlated side information. Following the entropy-constrained…
This paper examines the theory pertaining to lossless compression of correlated sources located at the edge of a network. Importantly, communication between nodes is prohibited. In particular, a method that combines correlated source coding…
Visual data compression is shifting from human-centered reconstruction to machine-oriented representation coding. In this setting, an image is often mapped to a compact semantic embedding, which is then compressed and transmitted for…
In this work, we address the lossy quantum-classical source coding with the quantum side-information (QC-QSI) problem. The task is to compress the classical information about a quantum source, obtained after performing a measurement while…