Related papers: Using Variable Threshold to Increase Capacity in a…

Recent efforts to learn reward functions from human feedback have tended to use deep neural networks, whose lack of transparency hampers our ability to explain agent behaviour or verify alignment. We explore the merits of learning…

We develop an approach to efficiently grow neural networks, within which parameterization and optimization strategies are designed by considering their effects on the training dynamics. Unlike existing growing methods, which follow simple…

The method presented extends a given regression neural network to make its performance improve. The modification affects the learning procedure only, hence the extension may be easily omitted during evaluation without any change in…

In this paper we propose the capacity optimization over sensing threshold for sensing-based cognitive radio networks. The objective function of the proposed optimization is to maximize the capacity at the secondary user subject to the…

In Shannon information theory the capacity of a memoryless communication channel cannot be increased by the use of feedback. In quantum information theory the no-cloning theorem means that noiseless copying and feedback of quantum…

Variable-length feedback coding has the potential to significantly enhance communication reliability in finite block length scenarios by adapting coding strategies based on real-time receiver feedback. Designing such codes, however, is…

In this paper we study the effect of rate-limited feedback on the sum-rate capacity of the deterministic interference channel. We characterize the sum-rate capacity of this channel in the symmetric case and show that having feedback links…

Graph neural networks (GNNs) are effective machine learning models for various graph learning problems. Despite their empirical successes, the theoretical limitations of GNNs have been revealed recently. Consequently, many GNN models have…

Binary neural networks (BNNs), where both weights and activations are binarized into 1 bit, have been widely studied in recent years due to its great benefit of highly accelerated computation and substantially reduced memory footprint that…

A long standing open problem in the theory of neural networks is the development of quantitative methods to estimate and compare the capabilities of different architectures. Here we define the capacity of an architecture by the binary…

Transformers for language modeling usually rely on deterministic internal computation, with uncertainty expressed mainly at the output layer. We introduce variational neurons into Transformer feed-forward computation so that uncertainty…

We consider the problem of characterizing network capacity in the presence of adversarial errors on network links,focusing in particular on the effect of low-capacity feedback links cross network cuts.

Modularity has been widely studied as a mechanism to improve the capabilities of neural networks through various techniques such as hand-crafted modular architectures and automatic approaches. While these methods have sometimes shown…

The notion of memory capacity, originally introduced for echo state and linear networks with independent inputs, is generalized to nonlinear recurrent networks with stationary but dependent inputs. The presence of dependence in the inputs…

Binary Neural Networks (BNNs) have emerged as a promising solution for reducing the memory footprint and compute costs of deep neural networks, but they suffer from quality degradation due to the lack of freedom as activations and weights…

The expressive power of artificial neural networks crucially depends on the nonlinearity of their activation functions. Though a wide variety of nonlinear activation functions have been proposed for use in artificial neural networks, a…

Recent research revealed a considerable lack of reliability for user feedback when interacting with adaptive systems, often denoted as user noise or human uncertainty. Moreover, this lack of reliability holds striking impacts for the…

The effect of inaccuracies in the parameters of a dynamic Bayesian network can be investigated by subjecting the network to a sensitivity analysis. Having detailed the resulting sensitivity functions in our previous work, we now study the…

The correct specification of reward models is a well-known challenge in reinforcement learning. Hand-crafted reward functions often lead to inefficient or suboptimal policies and may not be aligned with user values. Reinforcement learning…

Traditional approaches to understanding the brain's resilience to neuropathology have identified neurophysiological variables, often described as brain or cognitive 'reserve,' associated with better outcomes. However, mechanisms of function…