Related papers: The Capacity of Random Linear Coding Networks as S…
In this paper, we make use of channel symmetry properties to determine the capacity region of three types of two-way networks: (a) two-user memoryless two-way channels (TWCs), (b) two-user TWCs with memory, and (c) three-user…
Consider any discrete memoryless channel (DMC) with arbitrarily but finite input and output alphabets X, Y respectively. Then, for any capacity achieving input distribution all symbols occur less frequently than 1-1/e$. That is, \[…
In this paper, we introduce the notion of liquid time-constant (LTC) recurrent neural networks (RNN)s, a subclass of continuous-time RNNs, with varying neuronal time-constant realized by their nonlinear synaptic transmission model. This…
Recurrent neural networks (RNNs) have drawn interest from machine learning researchers because of their effectiveness at preserving past inputs for time-varying data processing tasks. To understand the success and limitations of RNNs, it is…
Numerical evaluations of the memory capacity (MC) of recurrent neural networks reported in the literature often contradict well-established theoretical bounds. In this paper, we study the case of linear echo state networks, for which the…
Let $X^n\in\mathcal{X}^n$ be a sequence drawn from a discrete memoryless source, and let $Y^n\in\mathcal{Y}^n$ be the corresponding reconstruction sequence that is output by a good rate-distortion code. This paper establishes a property of…
We construct a channel coding scheme to achieve the capacity of any discrete memoryless channel based solely on the techniques of polar coding. In particular, we show how source polarization and randomness extraction via polarization can be…
Network coding has been successfully used in the past for efficient broadcasting in wireless multi-hop networks. Two coding approaches are suitable for mobile networks; Random Linear Network Coding (RLNC) and XOR-based coding. In this work,…
Randomized network coding (RNC) greatly reduces the complexity of implementing network coding in large-scale, heterogeneous networks. This paper examines two tradeoffs in applying RNC: The first studies how the performance of RNC varies…
We consider the discrete memoryless asymmetric broadcast channels. We prove that the error probability of decoding tends to one exponentially for rates outside the capacity region and derive an explicit lower bound of this exponent…
The deep linear network (DLN) is a model for implicit regularization in gradient based optimization of overparametrized learning architectures. Training the DLN corresponds to a Riemannian gradient flow, where the Riemannian metric is…
In an unreliable single-hop broadcast network setting, we investigate the throughput and decoding-delay performance of random linear network coding as a function of the coding window size and the network size. Our model consists of a source…
We consider the problem of multicasting information from a source to a set of receivers over a network where intermediate network nodes perform randomized network coding operations on the source packets. We propose a channel model for the…
Layer normalization (LN) is a ubiquitous technique in deep learning but our theoretical understanding to it remains elusive. This paper investigates a new theoretical direction for LN, regarding to its nonlinearity and representation…
A single-letter characterization is provided for the capacity region of finite-state multiple-access channels, when the channel state process is an independent and identically distributed sequence, the transmitters have access to partial…
Traditional recurrent neural networks (RNNs) have a fixed, finite number of memory cells. In theory (assuming bounded range and precision), this limits their formal language recognition power to regular languages, and in practice, RNNs have…
Coded caching (CC) can transform cache memory at network devices into an active communication resource. Prior studies have shown that CC can significantly enhance the achievable Degrees of Freedom (DoF) in multi-input multi-output (MIMO)…
Sparse random linear network coding (SRLNC) used as a class of erasure codes to ensure the reliability of multicast communications has been widely investigated. However, an exact expression for the decoding success probability of SRLNC is…
In this paper, we consider some long-standing problems in communication systems with access to noisy feedback. We introduce a new notion, the residual directed information, to capture the effective information flow (i.e. mutual information…
We derive a lower and upper bound on the reliability function of discrete memoryless multiple-access channel (MAC) with noiseless feedback and variable-length codes (VLCs). For the upper-bound, we use proof techniques of Burnashev for the…