Related papers: Transfer Entropy as a Log-likelihood Ratio
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a…
This thesis applies entropy as a model independent measure to address three research questions concerning financial time series. In the first study we apply transfer entropy to drawdowns and drawups in foreign exchange rates, to study their…
We propose a unified theoretical framework for quantifying spatio-temporal interactions in a stochastic dynamical system based on information geometry. In the proposed framework, the degree of interactions is quantified by the divergence…
We prove that under certain mild assumptions, the entropy rate of a hidden Markov chain, observed when passing a finite-state stationary Markov chain through a discrete-time continuous-output channel, is jointly analytic as a function of…
Relative entropy is a measure of distinguishability for quantum states, and plays a central role in quantum information theory. The family of Renyi entropies generalizes to Renyi relative entropies that include as special cases most entropy…
This study investigates entropy's potential for analyzing scientific research patterns across disciplines. Originating from thermodynamics, entropy now measures uncertainty and diversity in information systems. We examine Shannon Entropy,…
While correlation measures are used to discern statistical relationships between observed variables in almost all branches of data-driven scientific inquiry, what we are really interested in is the existence of causal dependence. Designing…
We propose a new estimator to measure directed dependencies in time series. The dimensionality of data is first reduced using a new non-uniform embedding technique, where the variables are ranked according to a weighted sum of the amount of…
In statistical physics, if we successively divide an equilibrium system into two parts, we will face a situation that, within a certain length $\xi$, the physics of a subsystem is no longer the same as the original system. Then the…
Granger causal inference is a contentious but widespread method used in fields ranging from economics to neuroscience. The original definition addresses the notion of causality in time series by establishing functional dependence…
Predictability of behavior has emerged an an important characteristic in many fields including biology, medicine, and marketing. Behavior can be recorded as a sequence of actions performed by an individual over a given time period. This…
The question of deriving general force/flux relationships that apply out of the linear response regime is a central topic of theories for nonequilibrium statistical mechanics. This work applies an information theory perspective to compute…
In this work, conditional entropy is used to quantify the information loss induced by passing a continuous random variable through a memoryless nonlinear input-output system. We derive an expression for the information loss depending on the…
In networks of independent entities that face similar predictive tasks, transfer machine learning enables to re-use and improve neural nets using distributed data sets without the exposure of raw data. As the number of data sets in business…
Information flow or information transfer is an important concept in dynamical systems which has applications in a wide variety of scientific disciplines. In this study, we show that a rigorous formalism can be established in the context of…
Ordinal measures provide a valuable collection of tools for analyzing correlated data series. However, using these methods to understand the information interchange in networks of dynamical systems, and uncover the interplay between…
We introduce the term smooth entanglement entropy transfer, a phenomenon that is a consequence of the causality-cancellation property of the quantum gravity environment. The causality-cancellation of the quantum gravity space removes the…
We study the sample complexity of entropic optimal transport in high dimensions using computationally efficient plug-in estimators. We significantly advance the state of the art by establishing dimension-free, parametric rates for…
Modern statistical analysis often encounters high dimensional models but with limited sample sizes. This makes the target data based statistical estimation very difficult. Then how to borrow information from another large sized source data…
Shannon entropy, a cornerstone of information theory, statistical physics and inference methods, is uniquely identified by the Shannon-Khinchin or Shore-Johnson axioms. Generalizations of Shannon entropy, motivated by the study of…