Related papers: Variational Embedding Multiscale Sample Entropy:co…

Despite the popular of multimodal statistical models, there lacks rigorous statistical inference tools for inferring the significance of a single modality within a multimodal model, especially in high-dimensional models. For…

We propose utilizing entropy as a diagnostic tool to distinguish between constant and dynamical dark energy models. Entropy, a measure of the system's disorder or information content, captures the complexity and evolution of the universe.…

Background: Transitioning from an old medical coding system to a new one can be challenging, especially when the two coding systems are significantly different. The US experienced such a transition in 2015. Objective: This research aims to…

Network inference algorithms are valuable tools for the study of large-scale neuroimaging datasets. Multivariate transfer entropy is well suited for this task, being a model-free measure that captures nonlinear and lagged dependencies…

Entropy measures quantify the amount of information and correlation present in a quantum system. In practice, when the quantum state is unknown and only copies thereof are available, one must resort to the estimation of such entropy…

Several studies demonstrate that there are critical differences between real wireless networks and simulation models. This finding has permitted to extract spatial and temporal properties for links and to provide efficient methods as biased…

An experiment to study the entropy method for an anomaly detection system has been performed. The study has been conducted using real data generated from the distributed sensor networks at the Intel Berkeley Research Laboratory. The…

Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be…

Given the variety of the visual world there is not one true scale for recognition: objects may appear at drastically different sizes across the visual field. Rather than enumerate variations across filter channels or pyramid levels, dynamic…

Data transformation, e.g. feature transformation and selection, is an integral part of any machine learning procedure. In this paper we introduce an information-theoretic model and tools to assess the quality of data transformations in…

Data stream mining problem has caused widely concerns in the area of machine learning and data mining. In some recent studies, ensemble classification has been widely used in concept drift detection, however, most of them regard…

Multifractal analysis is one of the important approaches that enables us to measure the complexity of various data via the scaling properties. We compare the most common techniques used for multifractal exponents estimation from both…

Entropy is a fundamental concept in quantum information theory that allows to quantify entanglement and investigate its properties, for example its monogamy over multipartite systems. Here, we derive variational formulas for relative…

The multiscale entropy assesses the complexity of a signal across different timescales. It originates from the biomedical domain and was recently successfully used to characterize light curves as part of a supervised machine learning…

In many high-impact applications, it is important to ensure the quality of output of a machine learning algorithm as well as its reliability in comparison with the complexity of the algorithm used. In this paper, we have initiated a…

As large language models continue to scale, their growing computational and storage demands pose significant challenges for real-world deployment. In this work, we investigate redundancy within Transformer-based models and propose an…

Envelope methodology is succinctly pitched as a class of procedures for increasing efficiency in multivariate analyses without altering traditional objectives \citep[first sentence of page 1]{cook2018introduction}. This description is true…

Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…

Machine learning algorithms are designed to capture complex relationships between features. In this context, the high dimensionality of data often results in poor model performance, with the risk of overfitting. Feature selection, the…

We propose a new perspective on Turbulence using Information Theory. We compute the entropy rate of a turbulent velocity signal and we particularly focus on its dependence on the scale. We first report how the entropy rate is able to…