Related papers: Using Variable Threshold to Increase Capacity in a…
Learning from human feedback is an effective way to improve robotic learning in exploration-heavy tasks. Compared to the wide application of binary human feedback, scalar human feedback has been used less because it is believed to be noisy…
Current methods for estimating the required neural-network size for a given problem class have focused on methods that can be computationally intensive, such as neural-architecture search and pruning. In contrast, methods that add capacity…
An information measure based on fractional partitions of a set is used to derive a general dependence balance inequality for communication. This inequality is used to obtain new upper bounds on reliable and secret rates for multiterminal…
We propose two new criteria to understand the advantage of deepening neural networks. It is important to know the expressivity of functions computable by deep neural networks in order to understand the advantage of deepening neural…
It is well known that, in general, feedback may enlarge the capacity region of Gaussian broadcast channels. This has been demonstrated even when the feedback is noisy (or partial-but-perfect) and only from one of the receivers. The only…
Attentional Neural Network is a new framework that integrates top-down cognitive bias and bottom-up feature extraction in one coherent architecture. The top-down influence is especially effective when dealing with high noise or difficult…
We develop a novel framework for proving converse theorems for channel coding, which is based on the analysis technique of multicast transmission with an additional auxiliary receiver, which serves as a genie to the original receiver. The…
Deep neural networks generally involve some layers with mil- lions of parameters, making them difficult to be deployed and updated on devices with limited resources such as mobile phones and other smart embedded systems. In this paper, we…
Large-scale deep neural networks (DNNs) such as convolutional neural networks (CNNs) have achieved impressive performance in audio classification for their powerful capacity and strong generalization ability. However, when training a DNN…
We characterize the symmetric capacity to within 1.7075 bits/s/Hz for the two-user Gaussian interference channel with feedback. The result makes use of a deterministic model to provide insights into the Gaussian channel. We derive a new…
Binarized neural networks, or BNNs, show great promise in edge-side applications with resource limited hardware, but raise the concerns of reduced accuracy. Motivated by the complex neural networks, in this paper we introduce complex…
Artificial neural networks (NN) are instrumental in realizing highly-automated driving functionality. An overarching challenge is to identify best safety engineering practices for NN and other learning-enabled components. In particular,…
Adversarial training has proven to be effective in hardening networks against adversarial examples. However, the gained robustness is limited by network capacity and number of training samples. Consequently, to build more robust models, it…
In an ever expanding set of research and application areas, deep neural networks (DNNs) set the bar for algorithm performance. However, depending upon additional constraints such as processing power and execution time limits, or…
The theoretic capacity of a communication system constituted of several transmitting/receiving elements is determined by the singular values of its transfer matrix. Results based on an independent identically distributed channel model,…
The capacity of multiple-input multiple-output additive white Gaussian noise channels is investigated under peak amplitude constraints on the norm of the input vector. New insights on the capacity-achieving input distribution are presented.…
This paper addresses how a recursive neural network model can automatically leave out useless information and emphasize important evidence, in other words, to perform "weight tuning" for higher-level representation acquisition. We propose…
The utility of limited feedback for coding over an individual sequence of DMCs is investigated. This study complements recent results showing how limited or noisy feedback can boost the reliability of communication. A strategy with fixed…
Neural networks are composed of multiple layers arranged in a hierarchical structure jointly trained with a gradient-based optimization, where the errors are back-propagated from the last layer back to the first one. At each optimization…
Soft-thresholding has been widely used in neural networks. Its basic network structure is a two-layer convolution neural network with soft-thresholding. Due to the network's nature of nonlinearity and nonconvexity, the training process…