Related papers: Innovation Capacity of Dynamical Learning Systems
Predictive coding (PC) is a biologically plausible alternative to standard backpropagation (BP) that minimises an energy function with respect to network activities before updating weights. Recent work has improved the training stability of…
Current deep learning architectures suffer from catastrophic forgetting, a failure to retain knowledge of previously learned classes when incrementally trained on new classes. The fundamental roadblock faced by deep learning methods is that…
Fundamental limits of the cognitive interference channel (CIC) with two pairs of transmitter-receiver has been under exploration for several years. In this paper, we study the discrete memoryless cognitive interference channel (DM-CIC) in…
While information theory has been introduced to characterize the fundamental limitations of control and filtering for a few decades, the existing information-theoretic methods are indirect and cumbersome for analyzing the limitations of…
In distributed model predictive control (DMPC), where a centralized optimization problem is solved in distributed fashion using dual decomposition, it is important to keep the number of iterations in the solution algorithm, i.e. the amount…
Retrieving data from large-scale source code archives is vital for AI training, neural-based software analysis, and information retrieval, to cite a few. This paper studies and experiments with the design of a compressed key-value store for…
This paper is about output-feedback control problems for general linear systems in the presence of given state-, control-, disturbance-, and measurement error constraints. Because the traditional separation theorem in stochastic control is…
Quantum Reservoir Computing (QRC) exploits the dynamics of quantum ensemble systems for machine learning. Numerical experiments show that quantum systems consisting of 5-7 qubits possess computational capabilities comparable to conventional…
We investigate the capacity region of multi-user interference channels (IC), where each user encodes multiple sub-user components. By unifying chain-rule decomposition with the Entropy Power Inequality (EPI), we reason that single-user…
In this paper, a new and general version of Gaussian channel in presence of two-sided state information correlated to the channel input and noise is considered. Determining a general achievable rate for the channel and obtaining the…
Robust learning from noisy demonstrations is a practical but highly challenging problem in imitation learning. In this paper, we first theoretically show that robust imitation learning can be achieved by optimizing a classification risk…
Despite their great success, there is still no comprehensive theoretical understanding of learning with Deep Neural Networks (DNNs) or their inner organization. Previous work proposed to analyze DNNs in the \textit{Information Plane}; i.e.,…
Previous Knowledge Distillation based efficient image retrieval methods employs a lightweight network as the student model for fast inference. However, the lightweight student model lacks adequate representation capacity for effective…
In this work we study the capacity of interference-limited channels with memory. These channels model non-orthogonal communications scenarios, such as the non-orthogonal multiple access (NOMA) scenario and underlay cognitive communications,…
Reservoir computing (RC) is a machine learning algorithm that can learn complex time series from data very rapidly based on the use of high-dimensional dynamical systems, such as random networks of neurons, called "reservoirs." To implement…
The quest for energy-efficient, scalable neuromorphic computing has elevated compute-in-memory (CIM) architectures to the forefront of hardware innovation. While memristive memories have been extensively explored for synaptic implementation…
We study the propagation and distribution of information-carrying signals injected in dynamical systems serving as a reservoir computers. A multivariate correlation analysis in tailored replica tests reveals consistency spectra and…
Energy-harvesting-powered computing offers intriguing and vast opportunities to dramatically transform the landscape of the Internet of Things (IoT) devices by utilizing ambient sources of energy to achieve battery-free computing. In order…
Noise on quantum devices is much more complex than it is commonly given credit. Far from usual models of decoherence, nearly all quantum devices are plagued both by a continuum of environments and temporal instabilities. These induce noisy…
Large language models improve with scale, yet feedback-based alignment still exhibits systematic deviations from intended behavior. Motivated by bounded rationality in economics and cognitive science, we view judgment as resource-limited…