Related papers: Epistasis and Entropy
According to E.T. Jaynes and E.P. Wigner, entropy is an anthropomorphic concept in the sense that in a physical system correspond many thermodynamic systems. The physical system can be examined from many points of view each time examining…
An information theoretic measure is derived that quantifies the statistical coherence between systems evolving in time. The standard time delayed mutual information fails to distinguish information that is actually exchanged from shared…
The concept of entropy in statistical physics is related to the existence of irreversible macroscopic processes. In this work, we explore a recently introduced entropy formula for a class of stochastic processes with more than one absorbing…
Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order…
'Causal' direction is of great importance when dealing with complex systems. Often big volumes of data in the form of time series are available and it is important to develop methods that can inform about possible causal connections between…
The principle of maximum entropy is a broadly applicable technique for computing a distribution with the least amount of information possible while constrained to match empirically estimated feature expectations. However, in many real-world…
Transfer entropy provides a general tool for analyzing the magnitudes and directions---but not the \emph{kinds}---of information transfer in a system. We extend transfer entropy in two complementary ways. First, we distinguish…
Maximum entropy estimation is of broad interest for inferring properties of systems across many different disciplines. In this work, we significantly extend a technique we previously introduced for estimating the maximum entropy of a set of…
Informational entropy is often identified as physical entropy. This is surprising because the two quantities are differently defined and furthermore the former is a subjective quantity while the latter is an objective one. We describe the…
A sofic approximation to a countable group is a sequence of partial actions on finite sets that asymptotically approximates the action of the group on itself by left-translations. A group is sofic if it admits a sofic approximation. Sofic…
Radical-ion-pair reactions were recently shown to manifest a host of non-trivial quantum effects accounted for by quantum measurement theory. An alternative approach purporting to describe the fundamental quantum dynamics of spin-selective…
Transfer entropy is capable of capturing nonlinear source-destination relations between multi-variate time series. It is a measure of association between source data that are transformed into destination data via a set of linear…
For deterministic continuous time nonlinear control systems, epsilon-practical stabilization entropy and practical stabilization entropy are introduced. Here the rate of attraction is specified by a KL-function. Upper and lower bounds for…
Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields. This problem has many applications in various domains, including machine learning, statistics and data compression. Over…
This paper introduces the basic concepts of information theory. Based on these concepts, we regard the states in the state space and the types of ideal gases as the symbols in a symbol set to calculate the mixing entropy of ideal gas…
Potentially influential spaces in the spatial networks of cities can be detected by means of the entropy participation ratios. Local (connectivity) and global (centrality) entropies are considered. While the connectivity entropy has a…
Estimating information-theoretic quantities such as entropy and mutual information is central to many problems in statistics and machine learning, but challenging in high dimensions. This paper presents estimators of entropy via inference…
Maximum Entropy is a powerful concept that entails a sharp separation between relevant and irrelevant variables. It is typically invoked in inference, once an assumption is made on what the relevant variables are, in order to estimate a…
Tsallis' non-extensive entropy is extended to incorporate the dependence on affinities between the microstates of a system. At the core of our construction of the extended entropy ($\mathcal{H}$) is the concept of the effective number of…
Machine-type communications (MTC) are crucial in the evolution of mobile communication systems. Within this context, we distinguish the so-called massive MTC (mMTC), where a large number of devices coexist in the same geographical area. In…