Related papers: Neural Decision Boundaries for Maximal Information…
Spiking neural networks can compensate for quantization error by encoding information either in the temporal domain, or by processing discretized quantities in hidden states of higher precision. In theory, a wide dynamic range state-space…
Selective rationalization improves neural network interpretability by identifying a small subset of input features -- the rationale -- that best explains or supports the prediction. A typical rationalization criterion, i.e. maximum mutual…
Neurons in the nervous system convey information to higher brain regions by the generation of spike trains. An important question in the field of computational neuroscience is how these sensory neurons encode environmental information in a…
Living cells must control the reading out or "expression" of information encoded in their genomes, and this regulation often is mediated by transcription factors--proteins that bind to DNA and either enhance or repress the expression of…
The theory of Balanced Neural Networks is a very popular explanation for the high degree of variability and stochasticity in the brain's activity. Roughly speaking, it entails that typical neurons receive many excitatory and inhibitory…
Capacity of conscious agents to perform genuine choices among future alternatives is a prerequisite for moral responsibility. Determinism that pervades classical physics, however, forbids free will, undermines the foundations of ethics, and…
In this Letter we analyze the benefit of digital compensation of fiber nonlinearity, where the digital signal processing is divided between the transmitter and receiver. The application of the Gaussian noise model indicates that, where…
Bayesian inference offers a principled account of information processing in natural agents. However, it remains an open question how neural mechanisms perform their abstract operations. We investigate a hypothesis where a distributed form…
Neural dynamics is triggered by discrete synaptic inputs of finite amplitude. However, the neural response is usually obtained within the diffusion approximation (DA) representing the synaptic inputs as Gaussian noise. We derive a…
We consider the information fiber optical channel modeled by the nonlinear Schrodinger equation with additive Gaussian noise. Using path-integral approach and perturbation theory for the small dimensionless parameter of the second…
Word embeddings provide point representations of words containing useful semantic information. We introduce multimodal word distributions formed from Gaussian mixtures, for multiple word meanings, entailment, and rich uncertainty…
In this paper, we investigate the signal shaping in a two-user discrete time memoryless Gaussian multiple-access channel (MAC) with computation. It is shown that by optimizing input probability distribution, the transmission rate per…
The Bayesian and Akaike information criteria aim at finding a good balance between under- and over-fitting. They are extensively used every day by practitioners. Yet we contend they suffer from at least two afflictions: their penalty…
A new field of research is rapidly expanding at the crossroad between statistical physics, information theory and combinatorial optimization. In particular, the use of cutting edge statistical physics concepts and methods allow one to solve…
Path planning is an important component of au- tonomous mobile sensing systems. This paper studies upper and lower bounds of communication performance over Gaussian sen- sor networks, to drive power-distortion metrics for path planning…
Information needs to be appropriately encoded to be reliably transmitted over physical media. Similarly, neurons have their own codes to convey information in the brain. Even though it is well-known that neurons exchange information using a…
Since proposed, spiking neural networks (SNNs) gain recognition for their high performance, low power consumption and enhanced biological interpretability. However, while bringing these advantages, the binary nature of spikes also leads to…
We show that two dynamical systems exhibiting very different deterministic behaviours possess very similar stationary distributions when stabilized by a multiplicative Gaussian white noise. We also discuss practical aspects of numerically…
We examine receiver-optimal mechanisms for aggregating information divided across many biased senders. Each sender privately observes an unconditionally independent signal about an unknown state, so no sender can verify another's report. A…
The nonlinear Shannon capacity limit has been identified as the fundamental barrier to the maximum rate of transmitted information in optical communications. In long-haul high-bandwidth optical networks, this limit is mainly attributed to…