Related papers: Information-theoretic analysis of multivariate sin…
Exchange of molecules allows cells to exchange information. How robust is the information to changes in cell parameters? We use a mapping between the stochastic dynamics of two cells sharing a stimulatory molecule, and parameters akin to an…
Real-space mutual information (RSMI) was shown to be an important quantity, formally and from a numerical standpoint, in finding coarse-grained descriptions of physical systems. It very generally quantifies spatial correlations, and can…
Current methods for pattern analysis in time series mainly rely on statistical features or probabilistic learning and inference methods to identify patterns and trends in the data. Such methods do not generalize well when applied to…
Finding interdependency relations between (possibly multivariate) time series provides valuable knowledge about the processes that generate the signals. Information theory sets a natural framework for non-parametric measures of several…
Analysis of data without labels is commonly subject to scrutiny by unsupervised machine learning techniques. Such techniques provide more meaningful representations, useful for better understanding of a problem at hand, than by looking only…
This paper discusses the implementation of a tactical network simulation tool. The tool is called Tactical Network Modeller (TNM). TNM uses some novel techniques to simplify the building of the network model using graph theory constrained…
The principles of statistical mechanics and information theory play an important role in learning and have inspired both theory and the design of numerous machine learning algorithms. The new aspect in this paper is a focus on integrating…
Machine learning has emerged as the dominant tool for implementing complex cognitive tasks that require supervised, unsupervised, and reinforcement learning. While the resulting machines have demonstrated in some cases even super-human…
Despite the remarkable capabilities of Language Models (LMs) across diverse tasks, no single model consistently outperforms others, necessitating efficient methods to combine their strengths without expensive retraining. Existing model…
In high-dimensional and high-stakes contexts, ensuring both rigorous statistical guarantees and interpretability in feature extraction from complex tabular data remains a formidable challenge. Traditional methods such as Principal Component…
A framework integrating information theory and network science is proposed, giving rise to a potentially new area. By incorporating and integrating concepts such as complexity, coding, topological projections and network dynamics, the…
This manuscript contains preprint of a chapter under consideration for inclusion in the forthcoming third edition of {\em Cover and Thomas's Elements of Information Theory}, posted with permission from Wiley. The table of contents EIT-3 ToC…
Recent progress in large language models (LLMs) has focused on test-time scaling to improve reasoning via increased inference computation, but often at the cost of efficiency. We revisit test-time behavior and uncover a simple yet…
While neural networks have been successfully applied to the full-spectrum k-distribution (FSCK) method at a large range of thermodynamics with k-values predicted by a trained multilayer perceptron (MLP) model, the required a-values still…
The correlation and interactions among different biological entities comprise the biological system. Although already revealed interactions contribute to the understanding of different existing systems, researchers face many questions…
Molecular-scale computation is crucial for smart materials and nanoscale devices, yet creating single-molecule systems capable of complex computations remains challenging. We present a theoretical framework for a single-molecule computer…
In this contribution, we consider a two-tiered network and focus on the coexistence between the two tiers at physical layer. We target our efforts on a long term evolution advanced (LTE-A) orthogonal frequency division multiple access…
Across the scientific literature, information measurement in the nervous system is posed as a problem of information processing internal to the brain by constructs such as neuronal populations, sensory surprise, or cognitive models.…
We give an overview of the role of information theory in statistics, and particularly in biostatistics. We recall the basic quantities in information theory; entropy, cross-entropy, conditional entropy, mutual information and…
The nervous system represents time-dependent signals in sequences of discrete action potentials or spikes, all spikes are identical so that information is carried only in the spike arrival times. We show how to quantify this information, in…