Related papers: Signalling Entropy: a novel network-theoretical fr…
Systematic generalization remains challenging for current language models, which are known to be both sensitive to semantically similar permutations of the input and to struggle with known concepts presented in novel contexts. Although…
The profile of a sample is the multiset of its symbol frequencies. We show that for samples of discrete distributions, profile entropy is a fundamental measure unifying the concepts of estimation, inference, and compression. Specifically,…
This report addresses state inference for hidden Markov models. These models rely on unobserved states, which often have a meaningful interpretation. This makes it necessary to develop diagnostic tools for quantification of state…
The thermodynamic entropy of coarse-grained (CG) models stands as one of the most important properties for quantifying the missing information during the CG process and for establishing transferable (or extendible) CG interactions. However,…
A transversal topic of my research has been the development and application of computational methods for DNA sequence analysis. The methods I have been developing aim at improving our understanding of the regulation processes happening in…
The recent extension of permutation entropy and its derivatives to graph signals has opened up new horizons for the analysis of complex, high-dimensional systems evolving on networks. However, these measures are all fundamentally rooted in…
Deep neural networks have demonstrated promising performance on image recognition tasks. However, they may heavily rely on confounding factors, using irrelevant artifacts or bias within the dataset as the cue to improve performance. When a…
The dynamics of molecular collisions in a macroscopic body are encoded by the parameter Thermodynamic entropy - a statistical measure of the number of molecular configurations that correspond to a given macrostate. Directionality in the…
Persistent entropy (PE) is an information-theoretic summary statistic of persistence barcodes that has been widely used to detect regime changes in complex systems. Despite its empirical success, a general theoretical understanding of when…
The electrical activity of external anal sphincter can be registered with surface electromyography. This signals are known to be highly complex and nonlinear. This work aims in characterisation of the information carried in the signals by…
This work presents a novel means for understanding learning dynamics and scaling relations in neural networks. We show that certain measures on the spectrum of the empirical neural tangent kernel, specifically entropy and trace, yield…
Biological cells encode information about their environment through biochemical signaling networks that control their internal state and response. This information is often encoded in the dynamical patterns of the signaling molecules,…
Sensitivity analysis is an effective tool for systematically identifying specific perturbations in parameters that have significant effects on the behavior of a given biosystem, at the scale investigated. In this work, using a…
Multivariate entropy quantification algorithms are becoming a prominent tool for the extraction of information from multi-channel physiological time-series. However, in the analysis of physiological signals from heterogeneous organ systems,…
Many branches of theoretical and applied mathematics require a quantifiable notion of complexity. One such circumstance is a topological dynamical system - which involves a continuous self-map on a metric space. There are many notions of…
Signal processing (SP) techniques convert DNA and protein sequences into information that lead to successful drug discovery. One must, however, be aware about the difference between information and entropy1. Eight other physical properties…
Efficient and accurate learning of constitutive laws is crucial for accurately predicting the mechanical behavior of materials under complex loading conditions. Accurate model calibration hinges on a delicate interplay between the…
During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…
Entropy metrics (for example, permutation entropy) are nonlinear measures of irregularity in time series (one-dimensional data). Some of these entropy metrics can be generalised to data on periodic structures such as a grid or lattice…
We explore the relation between entanglement entropy of quantum many body systems and the distribution of corresponding, properly selected, observables. Such a relation is necessary to actually measure the entanglement entropy. We show that…