Related papers: Neural Network Entropy (NNetEn): Entropy-Based EEG…
Optimization results are one method for understanding neural computation from Nature's perspective and for defining the physical limits on neuron-like engineering. Earlier work looks at individual properties or performance criteria and…
Claude Shannon coined entropy to quantify the uncertainty of a random distribution for communication coding theory. We observe that the uncertainty nature of entropy also limits its direct usage in mathematical modeling. Therefore we…
Convolutional Neural Networks (CNNs) specialize in feature extraction rather than function mapping. In doing so they form complex internal hierarchical feature representations, the complexity of which gradually increases with a…
How does the information flow between different brain regions during various stimuli? This is the question we aim to address by studying complex cognitive paradigms in terms of Information Theory. To assess creativity and the emergence of…
The success of deep convolutional neural network (CNN) in computer vision especially image classification problems requests a new information theory for function of image, instead of image itself. In this article, after establishing a deep…
This paper demonstrates that in classification problems, fully connected neural networks (FCNs) and residual neural networks (ResNets) cannot be approximated by kernel logistic regression based on the Neural Tangent Kernel (NTK) under…
Understanding how network function constrains neural connectivity is a central challenge in neuroscience. An influential approach is to train neural networks with gradient descent on cognitive tasks and characterize the resulting…
A measure is derived to quantify directed information transfer between pairs of vertices in a weighted network, over paths of a specified maximal length. Our approach employs a general, probabilistic model of network traffic, from which the…
In this work, several random Boolean networks (RBN) are generated and analyzed from two characteristics: their time evolution diagram and their transition diagram. For this purpose, its randomness is estimated using three measures, of which…
Entropy and mutual information in neural networks provide rich information on the learning process, but they have proven difficult to compute reliably in high dimensions. Indeed, in noisy and high-dimensional data, traditional estimates in…
A method for classification of complex time series using coarse-grained entropy rates (CER's) is presented. The CER's, which are computed from information-theoretic functionals -- redundancies, are relative measures of regularity and…
The purpose of this study was to investigate how the entropy of a neuron cell can be measured using membrane ion current signals, which were recorded from neurons in the mouse medial prefrontal cortex (mPFC). The sample entropy and the…
Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy…
A well-interpretable measure of information has been recently proposed based on a partition obtained by intersecting a random sequence with its moving average. The partition yields disjoint sets of the sequence, which are then ranked…
Conditional differential entropy provides an intuitive measure for relatively ranking time-series complexity by quantifying uncertainty in future observations given past context. However, its direct computation for high-dimensional…
On account of a greater need for understanding the complexity of time series like physiological time series, financial time series, and many more that enter into picture for their inculpation with real-world problems, several complexity…
Elucidating exocytosis processes provide insights into cellular neurotransmission mechanisms, and may have potential in neurodegenerative diseases research. Amperometry is an established electrochemical method for the detection of…
A networked time series (NETS) is a family of time series on a given graph, one for each node. It has a wide range of applications from intelligent transportation, environment monitoring to smart grid management. An important task in such…
Quantum Neural Networks (QNN) are considered a candidate for achieving quantum advantage in the Noisy Intermediate Scale Quantum computer (NISQ) era. Several QNN architectures have been proposed and successfully tested on benchmark datasets…
Ensembles of Deep Neural Networks (DNNs) have achieved qualitative predictions but they are computing and memory intensive. Therefore, the demand is growing to make them answer a heavy workload of requests with available computational…