Multiscale nonlinear integration drives accurate encoding of input information
Abstract
Biological and artificial systems encode information through several complex nonlinear operations, making their exact study a formidable challenge. These internal mechanisms often take place across multiple timescales and process external signals to enable functional output responses. In this work, we focus on two widely implemented paradigms: nonlinear summation, where signals are first processed independently and then combined; and nonlinear integration, where they are combined first and then processed. We study a general model where the input signal is propagated to an output unit through a processing layer via nonlinear activation functions. Further, we distinguish between the two cases of fast and slow processing timescales. We demonstrate that integration and fast-processing capabilities systematically enhance input-output mutual information over a wide range of parameters and system sizes, while simultaneously enabling tunable input discrimination. Moreover, we reveal that high-dimensional embeddings and low-dimensional projections emerge naturally as optimal competing strategies. Our results uncover the foundational features of nonlinear information processing with profound implications for both biological and artificial systems.
Cite
@article{arxiv.2411.11710,
title = {Multiscale nonlinear integration drives accurate encoding of input information},
author = {Giorgio Nicoletti and Daniel M. Busiello},
journal= {arXiv preprint arXiv:2411.11710},
year = {2024}
}