相关论文: On the Role of Feedback in LT Codes

Transformer encoders contextualize token representations by attending to all other tokens at each layer, leading to quadratic increase in compute effort with the input length. In practice, however, the input text of many NLP tasks can be…

Large Language Models trained on code corpora (code-LLMs) have demonstrated impressive performance in various coding assistance tasks. However, despite their increased size and training dataset, code-LLMs still have limitations such as…

The standard approach to the design of individual space-time codes is based on optimizing diversity and coding gains. This geometric approach leads to remarkable examples, such as perfect space-time block codes, for which the complexity of…

As the mobile application landscape expands, wireless networks are tasked with supporting various connection profiles, including real-time communications and delay-sensitive traffic. Among many ensuing engineering challenges is the need to…

The success of RL for LLM post-training stems from an unreasonably uninformative source: a single bit of information per rollout as binary reward or preference label. At the other extreme, distillation offers dense supervision but requires…

We propose two application-layer coding schemes for delay-constrained point-to-point packet communications with restrictions on the transmitter's maximum duty-cycle. The schemes operate over GF(2) and utilize intermittently available…

In this chapter, we show how the use of differential coding and the presence of phase slips in the transmission channel affect the total achievable information rates and capacity of a system. By means of the commonly used QPSK modulation,…

Multilingual Large Language Models (LLMs) can process many languages, yet how they internally represent this diversity remains unclear. Do they form shared multilingual representations with language-specific decoding, and if so, why does…

In this paper we demonstrate how code-switching patterns can be utilised to improve various downstream NLP applications. In particular, we encode different switching features to improve humour, sarcasm and hate speech detection tasks. We…

In this paper we study the long-term throughput performance of repetition protocols coupled with power control for multiple access block-fading channels. We propose to use the feedback bits to inform the transmitter about the decoding…

Code snippet adaptation is a fundamental activity in the software development process. Unlike code generation, code snippet adaptation is not a "free creation", which requires developers to tailor a given code snippet in order to fit…

Aligning large language models (LLMs) with human values and intents critically involves the use of human or AI feedback. While dense feedback annotations are expensive to acquire and integrate, sparse feedback presents a structural design…

This paper introduces the design and development of a framework that integrates a large language model (LLM) with a retrieval-augmented generation (RAG) approach leveraging both a knowledge graph and user interaction history. The framework…

A new deep-neural-network (DNN) based error correction encoder architecture for channels with feedback, called Deep Extended Feedback (DEF), is presented in this paper. The encoder in the DEF architecture transmits an information message…

Negative feedback is a powerful approach capable of improving several aspects of a system. In linear electronics, it has been critical for allowing invariance to device properties. Negative feedback is also known to enhance linearity in…

We introduce a novel mechanism, called timid/bold coding, by which feedback can be used to improve coding performance. For a certain class of DMCs, called compound-dispersion channels, we show that timid/bold coding allows for an improved…

This paper presents a decentralized relay selection protocol for a dense wireless network and describes channel feedback strategies that improve its performance. The proposed selection protocol supports hybrid automatic-repeat-request…

Feedback is one of the most crucial components to facilitate effective learning. With the rise of large language models (LLMs) in recent years, research in programming education has increasingly focused on automated feedback generation to…

Recent work has questioned whether large language models (LLMs) can perform genuine in-context learning (ICL) for scientific experimental design, with prior studies suggesting that LLM-based agents exhibit no sensitivity to experimental…

Linear precoded discrete multitone modulation (LP-DMT) system has been already proved advantageous with adaptive resource allocation algorithm in a power line communication (PLC) context. In this paper, we investigate the bit and energy…