Related papers: Information Density in Multi-Layer Resistive Memor…
Neural networks can be compressed to reduce memory and computational requirements, or to increase accuracy by facilitating the use of a larger base architecture. In this paper we focus on pruning individual neurons, which can simultaneously…
Motivated by applications in distributed storage, the storage capacity of a graph was recently defined to be the maximum amount of information that can be stored across the vertices of a graph such that the information at any vertex can be…
In this work we study the capacity of interference-limited channels with memory. These channels model non-orthogonal communications scenarios, such as the non-orthogonal multiple access (NOMA) scenario and underlay cognitive communications,…
Two novel views are presented on the trapdoor channel. First, by deriving the underlying iterated function system (IFS), it is shown that the trapdoor channel with input blocks of length $n$ can be regarded as the $n$th element of a…
Data hiding is the procedure of encoding desired information into a certain types of cover media (e.g. images) to resist potential noises for data recovery, while ensuring the embedded image has few perceptual perturbations. Recently, with…
We analyze physical-layer security based on the premise that the coding mechanism for secrecy over noisy channels is tied to the notion of channel resolvability. Instead of considering capacity-based constructions, which associate to each…
Secure data compression in the presence of side information at both a legitimate receiver and an eavesdropper is explored. A noise-free, limited rate link between the source and the receiver, whose output can be perfectly observed by the…
Responsive, adaptive and intelligent are widely used but inconsistently defined descriptors of soft matter. A conceptual framework is proposed in which the three classes are information channels of increasing architectural complexity: a…
The exploitation of multimodality in different degrees of freedom is one of the most promising ways to increase the rate of heralded entanglement between distant quantum nodes. In this paper, we realize a spatially-multiplexed solid-state…
Learning-based lossy image compression usually involves the joint optimization of rate-distortion performance. Most existing methods adopt spatially invariant bit length allocation and incorporate discrete entropy approximation to constrain…
Motivated by biological considerations, we study sparse neural maps from an input layer to a target layer with sparse activity, and specifically the problem of storing $K$ input-target associations $(x,y)$, or memories, when the target…
The capacity of non-coherent stationary Gaussian fading channels with memory under a peak-power constraint is studied in the asymptotic weak-signal regime. It is assumed that the fading law is known to both transmitter and receiver but that…
Two potential bottlenecks on the expressiveness of recurrent neural networks (RNNs) are their ability to store information about the task in their parameters, and to store information about the input history in their units. We show…
This article introduces a novel, low-cost technique for hiding data in commercially available resistive-RAM (ReRAM) chips. The data is kept hidden in ReRAM cells by manipulating its analog physical properties through switching…
In this paper, we study a model of communication under adversarial noise. In this model, the adversary makes online decisions on whether to corrupt a transmitted bit based on only the value of that bit. Like the usual binary symmetric…
The object of this article is to review the development of ultrahigh-density, nanoscale data storage, i.e., nanostorage. As a fundamentally new type of storage system, the recording mechanisms of nanostorage may be completely different to…
Very deep convolutional neural networks (CNNs) yield state of the art results on a wide variety of visual recognition problems. A number of state of the the art methods for image recognition are based on networks with well over 100 layers…
The problem of determining the best achievable performance of arbitrary lossless compression algorithms is examined, when correlated side information is available at both the encoder and decoder. For arbitrary source-side information pairs,…
We present the information-ordered bottleneck (IOB), a neural layer designed to adaptively compress data into latent variables ordered by likelihood maximization. Without retraining, IOB nodes can be truncated at any bottleneck width,…
We present two sequences of ensembles of non-systematic irregular repeat-accumulate codes which asymptotically (as their block length tends to infinity) achieve capacity on the binary erasure channel (BEC) with bounded complexity per…